Editing Openai/692db884-c2d8-800c-a357-aa2a0e684fbe (section)

==== ### ====
* Effective VRAM expansion By storing a lot of tensors, states, and buffers in compressed form and only inflating them when actually needed, you get the effect of: - Running larger models than the card should realistically hold. - Or running the same model with: - Bigger batch sizes - Longer context windows - More concurrent sessions.
* Fewer out-of-memory (OOM) crashes Because the “true” footprint is smaller, you: - Hit the physical VRAM limit less often. - Avoid the usual cascade of “retry, fragment, die” that can happen when you’re sitting on the edge of VRAM.
* More room for “extras” The saved space can go to: - Larger KV caches - Extra intermediate activations - More monitoring, logging, or side models on the same card.

In human terms:

: