Qwen3.6 27B is the density king of the 24GB class: strong general quality, excellent code, first-tier multilingual output (including Italian, which many open models handle poorly), and a hybrid thinking mode you can toggle per request for harder reasoning. At Q4 it fits a single used RTX 3090 with room for real context, which makes it the default recommendation for a first serious local deployment.
VRAM by quantization level
| Quant | Weights | Fits on |
|---|---|---|
| Q4_K_M | ~16 GB | 24GB comfortably; 16GB tight, short context |
| Q5_K_M | ~19 GB | 24GB with moderate context |
| Q8_0 | ~29 GB | 32GB (RTX 5090) or 48GB; 100K context fits on 32GB with Q8 KV-cache |
Weights only — add KV-cache (grows with context and concurrency) and ~1–2GB runtime overhead. Formula and cache math in the VRAM guide.
Quick start
LM Studio: search “Qwen3.6 27B”, pick the quant with the green fits-your-VRAM badge (MLX build on Macs). Production: vLLM with the AWQ build for multi-user serving. Raise the context — Ollama defaults num_ctx low; set 32768+ in a Modelfile or the thinking mode will truncate its own reasoning.
Expected performance
| Hardware | Generation speed |
|---|---|
| RTX 3090 (Q4) | ~28–35 tok/s |
| RTX 4090 (Q4) | ~35–45 tok/s |
| RTX 5090 (Q8) | ~35–45 tok/s |
| Mac M-series Max/Ultra | ~12–20 tok/s |
Indicative single-user figures (llama.cpp/Ollama class runtimes); multi-user serving via vLLM multiplies total throughput 10–20× through batching.
Tips & gotchas
- Thinking mode multiplies output tokens — great for math/planning, wasteful for summaries. Toggle it per request, don't leave it always on.
- For agents and JSON output, use Q5+ — structured output degrades before prose under quantization.
- Enable Q8 KV-cache to double usable context on the same card.
- Fully permissive — commercial use, modification and redistribution allowed.
FAQ
Can I run Qwen3.6 27B on 16GB of VRAM?
Yes at Q4 with short context (4–8K), but 24GB is the comfortable tier: full quality quant plus working context.
Is it good in Italian?
Among the best open models for Italian output quality — a key reason it's our default recommendation for Italian teams.
Thinking mode on or off?
Off by default; on for math, code architecture and multi-step planning. It costs tokens and latency.