Qwen3-Coder 30B is the model to run if your goal is a local coding assistant rather than a general chatbot: a mixture-of-experts tuned specifically for agentic coding — multi-file edits, tool use, test-fix loops in harnesses like Cline and Continue — with a native 256K context that swallows whole repositories. With only 3.3B parameters active per token it generates at small-model speed, which matters enormously in agent loops where one task means dozens of generations. At Q4 it fits a 24GB card.
VRAM by quantization level
| Quant | Weights | Fits on |
|---|---|---|
| Q4_K_M | ~19 GB | 24GB card with working context |
| Q4 + 128K context | ~24+ GB | 32GB, or 24GB with Q8 KV-cache |
| Q8_0 | ~33 GB | 48GB card, full quality + long context |
Weights only — add KV-cache (grows with context and concurrency) and ~1–2GB runtime overhead. Formula and cache math in the VRAM guide.
Quick start
Raise num_ctx aggressively (32K minimum, 64K+ for agent work) — Ollama's default context wastes this model's headline feature. Wire it into VS Code via Continue or Cline pointing at Ollama's OpenAI-compatible endpoint; pair it with a tiny model (1–3B) for tab-autocomplete, keeping the 30B for chat, edits and agent tasks. It's a non-thinking model by design: answers come fast, without a reasoning preamble.
Expected performance
| Hardware | Generation speed |
|---|---|
| RTX 3090 (Q4) | ~60–90 tok/s |
| RTX 4090 (Q4) | ~90–130 tok/s |
| Mac M-series Max | ~40–60 tok/s |
Indicative single-user figures (llama.cpp/Ollama class runtimes); multi-user serving via vLLM multiplies total throughput 10–20× through batching.
Tips & gotchas
- In agent harnesses, speed compounds: at 3.3B active parameters, a 30-step Cline task finishes in minutes, not half-hours.
- Long context isn't free — 256K tokens of KV-cache dwarf the weights. See our KV-cache guide for the real memory math.
- For pure tab-autocomplete latency, a small dense model beats any 30B — use this one where intelligence matters: edits, refactors, agent loops.
- Fully permissive — commercial use, modification and redistribution allowed.
FAQ
How does it compare to the giant Qwen3-Coder 480B?
Same family and training recipe; the 480B-A35B flagship is cloud/API territory. The 30B captures most of the agentic behavior at 1/16th the size — it's the local sweet spot.
Qwen3-Coder 30B or Qwen3.6 27B for coding?
Coder for agent harnesses, repo-scale context and raw speed; the dense 27B for mixed chat+code use and when you want a thinking mode. Both fit 24GB.
Can it really use 256K context locally?
The model can; your memory usually can't. 256K of KV-cache adds tens of GB. Realistic local windows are 32–128K with a quantized cache on 24–48GB hardware.