Going from 9 to 48 tokens per second on the same 8 GB GPU is the kind of jump that changes how people think about local inference. The report comes from a user testing Qwen 3.8 Flash Next with 3-bit quantization via Unsloth; the shift came after adopting STRATA on CachyOS, a Linux distribution with ready-to-use NVIDIA drivers instead of the usual manual setup. The average sits around 30 tokens/s, with peaks up to 48 tokens/s: enough for interactive use, even if the comparison with Claude should be read as a user perception rather than a benchmark.

The limit, however, is not speed. With 8 GB of VRAM, a 32k-token context works, but pushing further becomes a matter of megabytes. The user managed to load 42k tokens while staying about 130 MB away from a crash; the hoped-for 64k does not fit. That is a structural constraint: during inference, context occupies memory for the key-value cache, and every additional token takes space away from the model and compute buffers. The fact that 32k is stable and 42k is the practical ceiling shows how the advertised context window rarely matches what consumer hardware can actually use.

The case signals two things. First, software matters as much as hardware: the same model and GPU go from modest to solid performance just by changing the system and using ready drivers. Second, the race toward larger models hides the real bottleneck in local deployments: memory available for context, not parameter count. For anyone evaluating self-hosted setups, the trade-off becomes generation speed, perceived quality, and context length. Consumer 8 GB GPUs can run aggressively quantized models at interesting speeds, but they remain far from the long contexts that many enterprise workflows require. That tension is why compact on-premise inference is used for targeted tasks, while workloads with long documents or persistent memory still gravitate toward machines with more VRAM or toward the cloud. It is not about software loyalty; it is about bytes.