When a model runs locally, perceived quality does not depend only on the main weights. An experiment on the LocalLLaMA subreddit has highlighted an often overlooked weak link: the n-gram table, called PLE, that Qwen 3.8 Flash Next uses for inference and that common GGUF distributions quantize aggressively.

User sillynoobhorse, starting from an observation by MarkoMarjamaa and a suggestion by Eyelbee about using a bf16 sidecar, built a full FP8 sidecar using the ple_fp8_pack.py script. He kept the main model in IQ3_S but replaced the included PLE, quantized to IQ4_NL, with the full FP8 version. The result, according to his account, is a visible improvement: fewer errors in 3D models generated in HTML, fewer missing faces, and the ability to add features to a KDE app through a Hermes agent. No impact on inference speed, only more disk space used.

The point is not that the full PLE is always necessary. It is that quantization is not a single switch: different components of the same model tolerate different levels of compression. The PLE is an n-gram table residing on SSD and does not occupy VRAM. Compressing it aggressively reduces file size but can degrade accuracy without providing memory or speed benefits. In a self-hosted scenario, disk is often abundant compared with VRAM. The asymmetry — compressed main model, high-precision sidecar — therefore exploits the real constraint: VRAM, not storage.

This has implications for people who download pre-packaged GGUFs. Distributors tend to reduce file sizes because smaller downloads are convenient for distribution. Those who do not dig into technical details get a model that may make more mistakes, while those who rebuild the sidecar get better quality at the same VRAM footprint. The barrier is not compute but skill and bandwidth: you need the original safetensors files and about 47 GB for the full FP8 sidecar.

Structurally, the episode signals that distribution formats for local LLMs are becoming more complex. The model is no longer a single blob but an assembly of components with different precision profiles. Packing and serving tools must start treating these components separately, allowing differentiated quantization policies. For those evaluating on-premise deployment, the trade-off is no longer just between a large model and a small model, but between uniform quantization and selective quantization. AI-RADAR offers analytical frameworks at /llm-onpremise for those who want to evaluate these trade-offs without oversimplification.

The post does not provide rigorous benchmarks, but the visual comparisons between HTML scenes and the agent experience are enough to justify a test. In the end, the initial question — why not run the full n-gram table — has a practical answer: it can be done, and in many local contexts it is worth doing.