It looked like the perfect compromise: a 27-billion-parameter model capable of fitting on a single RTX 6000, delivering long outputs and impressive HTML demos. But the moment it’s given an agentic task — the continuous stream of instructions, actions, and corrections that defines practical LLM use — Qwen 3.6 27B crumbles. “Every four turns it does something completely braindead,” writes u/TokenRingAI on Reddit, after testing it with a nightly build of llama.cpp compiled from Git.

The story is instructive for anyone evaluating on-premise deployments. The user has been running Qwen 3.5 122B at 4-bit for a while and recently moved to 5-bit, as llama.cpp now matches VLLM’s performance. Under the same conditions — an NVIDIA RTX 6000 and the same runtime — the smaller model, tested at 8 and 16 bits, produces longer, more creative single prompts but falls apart as the conversation lengthens. “I cannot get the model to not screw up,” they add.

The gap between monolithic performance and agentic capability is not new, but it takes on concrete form for self-hosted infrastructure. When an organization evaluates an LLM for tasks like autonomous API navigation, iterative code generation, or ongoing support to human operators, multi-turn coherence is the real litmus test. Standard benchmarks, often based on single prompts or isolated questions, risk masking such fragility.

Beyond the single report, the case signals a growing tension in the open-LLM landscape: distilled models or those with heavy emphasis on first-shot creativity can show discontinuities in prolonged inference. Qwen 3.6 27B is not the first to suffer from this, but its popularity in the on-prem community — drawn by the ratio of VRAM to single-output quality — amplifies the message. For those running everything locally, the choice is never purely technical; it intertwines with data sovereignty, TCO, and operational predictability.

The user’s response is to go back to Qwen 3.5 122B, a heavier model but more reliable in agentic flows. Even under aggressive quantization, its ability to follow extended instructions proved superior. It’s a shift in perspective: it’s not just about how well a model can dazzle with a demo, but how dependable it is when no one is watching the screen.

The Reddit discussion is now gathering other reports. If the problem is widespread, it would question Alibaba’s strategy of promoting the 27B as “better than 122B” in certain evaluations. For now, the warning bell sounds in a corner of the on-premise lab — but that’s precisely where fragility turns into cost.