A 27-billion-parameter model dropping from 55.6 GB to 19.7 GB without collapsing on benchmarks: that is the claim from the QUASAR-QAT team for the Qwen3.8-27B NVFP4 checkpoint. The compression does not come from simple post-training quantization, but from quantization-aware distillation: the original BF16 model acted as teacher for 2,446 steps, while every linear layer across the transformer blocks was quantized to 4 bits for both weights and activations (W4A4).

Usually attention and GDN layers remain in FP8 or BF16 because quantizing them can degrade quality significantly. This checkpoint instead uses an aggressive configuration: all linear layers drop to NVFP4. The published numbers show a contained loss: GPQA-Diamond moves from 0.9141 for BF16 to 0.9091, while AIME26 holds at 1.0000. The vLLM serving command on NVIDIA Blackwell GPUs sets a context of 262,144 tokens and GPU memory utilization of 0.85.

The price of compactness

For teams evaluating on-premise deployment, the jump is not only technical. A 19.7 GB checkpoint lowers the VRAM threshold compared with the 55.6 GB BF16 model. That changes TCO calculations and makes self-hosting a 27-billion-parameter model more concrete, without relying on cloud APIs and without moving data outside. But compactness has a constraint: NVFP4 is tightly tied to the NVIDIA ecosystem and, for this checkpoint, to Blackwell GPUs. It is not a hardware-neutral precision reduction.

In the reported comparison, the QUASAR version is smaller than unsloth (23.4 GB) and Inferact (26.4 GB) yet scores higher: 0.9091 versus 0.8939 and 0.8763 on GPQA-Diamond, and 1.0000 versus 0.9778 and 0.9667 on AIME26. That is the most interesting detail: the more compact checkpoint beats heavier alternatives. It suggests that quantization-aware distillation, not just post-training compression, is becoming the real differentiator for teams running LLMs in local production.

The message for architects of on-premise infrastructure is not that W4A4 is free. It is that the cost of precision shifts from the individual operator to the training process: if the teacher model guides quantization well, inference can drop below 20 GB while keeping reasoning capability nearly unchanged. The gap between 19.7 and 26.4 GB is not just memory: it is a signal of how quantization-aware training can redefine the trade-off between hardware cost and quality.