The test starts with a concrete prompt: a user asked which modern open weight models showed the least sycophancy, citing Kimi K2 and GPT OSS 120b as outdated. The model, in IQ3_S quantization, had to express an opinion on a post and the tester measured tokens per second on a long generation. In the thinking trace the system begins linearly, evaluates the current date (2026-06-22) and prepares to answer. Then, after 10,000 tokens, the generation gets stuck in a repetitive sequence: 'Need maybe maybe include...' The same exact build, run in llama.cpp, produces a coherent answer.

The difference is not in the weights nor in the quantization level, but in the runtime. IQ3_S is a low-bit quantization that reduces VRAM footprint and makes the model runnable even on non-extreme local hardware. But if the serving stack introduces instability on long generations, the memory savings do not translate into reliable output. The runtime handles sampling, token scheduling, cache and stop conditions: a different configuration can turn an answer into a loop.

Teams evaluating on-premise deployment can read this episode as a reminder about reproducibility. A model can pass benchmarks on one engine and fail on another with the same GGUF file. Test pipelines should include long and repeated outputs, not just synthetic benchmarks. Moreover, changing runtime is not free: integrations, parameters and quality metrics need to be reviewed. TCO is not only about GPUs and energy consumption, but about the stability of the entire stack.

The winners are teams that do not stop at the model's spec sheet. The losers are comparative evaluations based only on average scores, because a loop can escape a benchmark with short prompts. There is also a structural effect: runtime maintainers such as llama.cpp become first-level technical interlocutors, not mere executors.

The post title is an implicit instruction: 'Need maybe say Use llama.cpp'. It is not a universal recommendation, but a hint of how much the execution context matters. One open question remains: how many failures attributed to open weight models are actually failures of the serving stack?