6,250 tokens per second on a 32GB RTX 5090. That is the figure attached to the new GGUF NVFP4 quant for Qwen3.8 27B, reported on Reddit and presented as the fastest NVFP4 quant available for that model. The benchmark focuses on prefill with 2048-token prompts: the NVFP4 version delivers 6,250 tokens/s, compared with 4,130 tokens/s for a Q4_0 quant of the same memory footprint and 3,210 tokens/s for Q6_K. Against another NVFP4 quant, the poster reports a 4-7% advantage.

The comparison with Q4_0 deserves attention for a specific reason: the two formats occupy the same amount of VRAM, but NVFP4 exploits Blackwell tensor cores more directly, at least on compatible hardware. Q4_0 is a traditional integer format in the GGUF world; NVFP4 is a 4-bit floating-point format designed for Blackwell. The 50% difference therefore does not come from more aggressive compression, but from a different use of the silicon. For anyone serving LLMs on a single GPU, prefill is the phase that determines initial latency and the ability to process long prompts or more queued requests: a 50% gain in this phase matters more than an equivalent improvement in generation, which remains more tied to memory bandwidth.

The GGUF also includes a quantized MTP draft head. Multi-Token Prediction, used as a draft head for speculative decoding, accelerates generation by accepting multiple tokens in parallel. Having it in the same file simplifies local deployment: there is no need to manage a separate draft model. The poster points to specific settings to get up to 15% faster MTP, a practical detail when the goal is reducing perceived latency.

This story signals something broader. After years in which GGUF quantizations moved toward relative portability, NVFP4 reintroduces a hardware dependency. The measured advantage exists only on compatible hardware; on non-Blackwell GPUs the comparison with Q4_0 does not hold. For anyone building self-hosted stacks on heterogeneous hardware, this is a new tension: on one side, native formats like NVFP4 promise higher peaks without increasing VRAM; on the other, they fragment the ecosystem between those who own the latest GPU generation and those who do not. This is not an abstract issue for anyone evaluating the TCO of an on-premise fleet: a hardware-specific format can shorten the useful life of infrastructure or push toward more frequent purchases, even when throughput numbers seem to justify them. For those evaluating on-premise deployment, AI-RADAR offers analytical frameworks on /llm-onpremise to weigh exactly these trade-offs between peak performance and portability.

The result remains tied to a benchmark on a 32GB RTX 5090 and to a user post, so it should be read as a directional signal rather than a guarantee. But the direction is clear: compression is returning to a closer dialogue with silicon, and those who manage models locally will need to consider not only tokens/s, but also the compatibility of the format with the GPU fleet they own.