Quantization has become the mandatory step to bring large language models out of cloud data centers and into enterprise infrastructure, shrinking memory footprint and speeding up inference. So far, however, those choosing lossy methods like GPTQ or AWQ had to accept a measurable degradation in quality, while lossless techniques preserved fidelity but offered no real performance gains. A new paper, published two months ago and recently reshared by Red Hat AI, proposes a third way: “statistically lossless” compression, which guarantees output quality in probabilistic terms without sacrificing speed.

The work introduces SLQ, a layer-wise non-uniform scalar quantization method with asymmetric quantization and a wide bitwidth search. The researchers define three tiers of losslessness. The first, “task-lossless,” preserves zero-shot benchmark accuracy within natural sampling variance and is achievable at very aggressive bitwidths: below 4 bits, down to 3.3 bits depending on the model. The second, stricter tier is “distribution-lossless”: the quantized model’s next-token distribution is practically indistinguishable from the original. To measure this, the paper formalizes the Expected Acceptance Rate (EAR), the maximum token-agreement probability under optimal coupling — for example, an EAR ≥ 0.99 means 99% agreement. This level requires an average of 5–6 bits per parameter.

The third contribution is a gamma-squared variance law showing that symmetric quantization inflates noise variance by a factor of gamma² relative to asymmetric quantization — a mathematical detail with immediate practical consequences. Asymmetry is essential for distributional fidelity but not required to preserve aggregate task performance. In other words, if the goal is a model that responds “like the original” — not merely with the same benchmark scores — you must abandon the common symmetric schemes.

The numbers matter. SLQ accelerates inference by a factor of 1.7 to 3.6× over FP16, using optimized kernels that run on both GPUs and CPUs (llama.cpp is explicitly cited, confirming compatibility with widely adopted local deployment stacks). This shifts the equation for anyone running models on-premise. For an organization that today spends tens of thousands of euros on GPUs to host a 70‑billion‑parameter model, dropping below 4 bits without quality loss means they can use less expensive hardware, or colocate multiple models on the same machine, or extend the context window without exhausting VRAM. The reduction in energy consumption, another hidden cost of local inference, follows the same trajectory.

Structurally, this kind of research signals that compression is no longer a “lossy” trade‑off. The fact that statistical indistinguishability can be maintained at 5–6 bits — and full accuracy preserved below 4 bits — raises the bar for on‑premise solution vendors. Those who were betting on dedicated inference hardware, perhaps targeting INT8 or INT4, are now faced with models that can run on non‑standard formats yet are supported by open‑source kernels. The room for specialized silicon narrows, while pressure mounts on deployment‑platform providers to optimize their runtimes for asymmetric, non‑uniform schemes.

For the AI‑RADAR reader, the question is no longer “how much can I compress without breaking the model” but “which statistical quality guarantee do I need for my use case.” A customer‑support application requiring high semantic coherence might demand the distribution‑lossless level (EAR ≥ 0.99) and accept 5–6 bits. An internal classification tool could instead operate at 3.3 bits task‑lossless, saving resources. The SLQ code is announced on GitHub as “coming soon,” and the reshare by Red Hat AI suggests that the enterprise world has already trained its radar on these techniques.