A GeForce RTX 5070 with 12GB of VRAM, a Ryzen 5 7600, 64GB of DDR5 RAM, and Windows. It is a desktop, not an inference machine. Yet according to data published on the Strata GitHub repository, it is enough to run Qwen3.8-Flash-Next at 65.1 tokens/s output and 543 tokens/s prompt processing on a 128K context. The credit does not go to the GPU alone, but to a custom inference engine written for this specific model and this class of hardware: one-click install, CUDA-only support, and a model published on Hugging Face by ISTA-DASLab.

The recent comparison gives the scale of the jump. With llama.cpp and IQ3_XXS quantization, the author recorded 15 tokens/s output and 100-120 tokens/s prompt processing. With the new engine, IQ3_XXS reaches 44.8 tokens/s, IQ2_XS 52.0, and Q2_0 65.1. Prompt processing moves to 414, 472, and 543 tokens/s respectively. The 2-bit versions, produced with RCO-GSQ, run faster than the heavier ones: they take less space and, in this context, beat IQ3_XXS on speed. The repository labels them as equivalent to unsloth Q3, Q4, and Q5, a useful reference for those already familiar with those numerical precisions. It is a counterintuitive detail for those who associate aggressive precision reduction only with quality loss, because here the constraint is not the model but available memory.

The minimum combined RAM and VRAM requirements tell the story: 37.6GB for Q2_0, 39.2GB for IQ2_XS, 47GB for IQ3_XXS, plus 0.91GB for the vision encoder. The 12GB card does not contain everything; system RAM becomes an active part of execution. The point is not magic but orchestration: specialized CUDA kernels, memory management, and a model designed to be pushed to the limit.

This is not an isolated benchmark. It shows that for local workloads and long contexts, a runtime specialized on a single model can outperform a generalist framework like llama.cpp when the target hardware is defined. Strata is not a universal platform: it works on CUDA, for this model, with a simplified but constrained install path. Users are not buying versatility; they are buying efficiency within a narrow perimeter. For a technical team serving a specific use case on hardware it already owns, the potential saving is not about buying a larger GPU but about reusing a desktop. For generalist framework vendors, it is a signal: the last mile of optimization can end up in ad hoc projects, hard to replicate but hard to ignore when the gain multiplies output speed by three or four times on the same card.

Finally, data sovereignty. If a model with a vision encoder and a 128K context can run on an office machine, the push to send documents and prompts to an external API weakens. The need for skills to maintain the runtime does not disappear, but the cost of staying on-premise changes qualitatively. Self-hosted stops being a data center-only choice and becomes a matter of software engineering, numerical precision, and risk tolerance. For those evaluating on-premise deployments, AI-RADAR offers analytical frameworks at /llm-onpremise to assess these trade-offs without turning a demo into a universal recommendation. The next frontier is not the card, but how much the software can squeeze before model quality becomes the real bottleneck.