Anyone assuming a small LLM needs aggressive fine-tuning to improve has a concrete counterexample. Qwengram-0.8B starts from Qwen3.5-0.8B, freezes the backbone, and attaches a pretrained PLE n-gram memory of about 51 billion parameters. On the frozen validation set, perplexity drops from 18.2759 to 17.3534, a 5.05 percent reduction. The author is explicit: this is a language model validation result, not a benchmark accuracy gain.

The technical setup is deliberately sparse. An R=1 reader is trained at decoder layers 3 and 9, while a token-dependent linear gate controls the late injection. There is no backbone fine-tuning. The balanced checkpoint uses a reader trained on 15 million tokens and a separately calibrated dynamic gate. The inference path is implemented in llama.cpp.

The intermediate results explain the choices. The real PLE memory beats the random-memory and permuted-memory controls. Reader loss keeps improving beyond 5 million training tokens. The 20M reader improves aggregate LM loss further but regresses on math tasks; the 15M checkpoint remains the balance point. A warm-started R=4 reader, with the same memory budget, offers a small aggregate improvement but introduces code and math regressions. R=1 is a compromise architecture, not a naive simplification.

The most interesting part is the gate. Strong fixed late-layer injection hurts LAMBADA; dynamic token-level arbitration recovers much of that trade-off. The gate does not behave like a learned constant: memory strength varies substantially across tokens. Learned token placement beats shuffled placement at equal memory budget, and routing memory toward high-uncertainty positions recovers only part of the advantage. In short, the value is not in the memory itself, but in deciding when and where to use it.

On the deployment side, the package has a structural feature. The GGUF contains the Qwen3.5 backbone plus the trained reader and arbitration tensors. The large PLE memory is not packed into the GGUF: it remains an external quantized sidecar. In a separate WikiText-2 test, Q8_0 quantization retains 99.1 percent of the BF16 reader NLL gain. The author distinguishes this runtime measurement from the frozen Kaggle validation benchmark.

That separation changes the reasoning for local model operators. Instead of a single artifact to load and update, there is a lightweight backbone with an external memory managed separately. It is a pattern that could simplify knowledge updates without touching the main weights, but it requires the runtime to support token-dependent gates and quantized sidecars. Anyone evaluating self-hosted stacks must look not only at the GGUF checkpoint, but also at how the serving layer handles the sidecar and its quantization. For those evaluating on-premise deployment, AI-RADAR provides analytical frameworks at /llm-onpremise to assess these trade-offs.

The author points to larger Qwen backbones as the next step, especially the 35B-A3B MoE, but acknowledges that would require substantially more compute than free Kaggle notebooks. The direction, however, is already set: scale the reader and dynamic arbitration, not the backbone fine-tuning.