Ever since large language models (LLMs) started running locally, the thin line between available VRAM and disk space has separated usable inference from unbearable waiting. In communities like LocalLLaMA, “disk spillover” — the moment a model exceeds video memory and starts using system RAM or disk — has always been a point of no return: performance plummeted from 4–5 tokens per second to roughly 0.5 tokens per second. Now, with the arrival of inference boosters — dSpark, dFlash, Multi-Token Prediction (MTP), Quantization-Aware Training (QAT), and others — the question arises: has that cliff become less steep?

These techniques, ranging from speculative decoding to FlashAttention variants, from multi-token generation to quantization-aware training, accelerate inference under optimal conditions by reducing the number of operations or memory footprint. But the real test happens when VRAM runs out. And here the answer leaves little room for optimism: the bottleneck shifts, but it doesn’t disappear. Even NVMe disks have latency and bandwidth orders of magnitude lower than VRAM; no compute optimization can compensate for slow weight loading. Disk spillover remains a performance disaster.

Yet the frustration bar is creeping upward. Techniques like QAT and dFlash can shrink memory usage and delay spillover, possibly allowing larger models to run without touching the disk at all. When spillover does occur, some users report marginal gains — from 0.5 to 0.7–0.8 tokens/s — still far from acceptable for an interactive chatbot, but enough to prompt Reddit questions like: “Are we becoming tolerant?”. The mere fact the question is being asked signals a perceptual shift, the result of incremental improvements that, while not revolutionizing the hardware, widen the maneuvering room by a few centimeters.

For those planning on-premise deployments, the message is twofold. Software optimizations can extend the life of a GPU with limited VRAM, but they don’t turn an undersized asset into a performant machine. Hardware remains the foundation. Knowing that QAT and dFlash save a few gigabytes and push spillover further away is useful data for anyone evaluating Total Cost of Ownership (TCO), but it doesn’t flip the balance of power. The cost of a GPU with more VRAM must be weighed against the guarantee of a predictable user experience, without the sword of Damocles of a single-digit token-per-second drop. In this light, analytical frameworks for on-premise deployment help navigate the trade-offs, reminding us that TCO isn’t just about peak speed, but also about perceived stability.

The technical direction is promising — speculative decoding, tensor parallelism, and distribution across multiple GPUs might one day relegate the disk to a remote backup role. But today, anyone doing local inference would do well to treat these accelerations as a valuable cushion, not as the launch ramp that erases the need for VRAM.