The unwritten rule among people running LLMs locally was to avoid dropping below Q4 quantization. Moving to Q3 levels was considered too risky because models often appeared visibly degraded. A test on a 16GB RTX 4060 Ti tells a different story for Qwen 3.8 27B in Q3_xxs: fully loaded into VRAM it hit 30-35 tokens/s, dropping to 21-22 tokens/s only with long contexts. The speed was essentially identical to that of a 35B at higher quantization partially offloaded to RAM. In other words, keeping the entire model in VRAM offset the lower numerical precision, at least in terms of latency.

The most interesting aspect, however, is not fluidity. The user does not run the model in agentic workflows but uses it for occasional coding assistance. In this scenario Qwen 3.8 27B completed several serious programming tasks on the first attempt, producing fully working games and web apps. The Qwen 3.6 35B previously used, on the same tests, failed outright or required hours of prompting and corrections. Yet the same model in Q3_xxs also showed the other side: in normal conversations it misunderstood some requests and failed basic sorting or counting of a few values. The split between complex tasks solved immediately and simple mistakes is stark.

This asymmetry is the structural point. Aggregate metrics and generic benchmarks can be misleading: an LLM capable of handling advanced logic may fail on basic tasks that matter just as much for a conversational assistant. With quantization, degradation is not uniform; it hits some capabilities harder than others, and in a model heavily oriented toward code it may concentrate exactly on layers less exercised by programming tests. For anyone evaluating a self-hosted deployment, the consequence is that quantization level should be treated as a design variable tied to workload, not as a universal threshold.

There is also a signal for hardware. A consumer GPU with 16GB, often considered borderline for 27B-class models, proves sufficient for smooth inference as long as the model stays entirely in VRAM. This reduces pressure toward cards with more memory for some local coding workloads and shifts attention to model quality and stability at low precision. At the same time, the fact that Q3_xxs holds up on demanding tasks but slips on simple ones means acceptance testing for on-premise systems should include open conversations and basic operations, not just hard benchmarks. For those weighing precision, speed, and control in on-premise deployments, AI-RADAR offers analytical frameworks at /llm-onpremise to assess the trade-offs.

The case does not say Q3_xxs is the new normal. It says the old Q4 rule as an inviolable boundary needs to be revisited model by model, and that quantization degradation shows up in ways that only prolonged, task-specific use can reveal.