gpt-oss-20b is the model that put 'runs ChatGPT-style reasoning at home' within reach of ordinary hardware: a 21B mixture-of-experts with only 3.6B parameters active per token, shipped natively in MXFP4 4-bit — so the whole model is ~13GB with no post-hoc quantization loss. It thinks before answering (with a reasoning effort you can set to low, medium or high) and is tuned for tool calling and agent loops. The MoE design makes it unusually fast for its quality tier, and unusually tolerant of CPU offload.
VRAM by quantization level
| Quant | Weights | Fits on |
|---|---|---|
| MXFP4 (native) | ~13 GB | 16GB card (tight context) or 16GB unified RAM; 24GB comfortable |
| MXFP4 + long context | ~16–18 GB | 24GB card with room for 64K+ context |
| BF16 (upcast) | ~44 GB | 48GB — rarely worth it: MXFP4 is the trained format |
Weights only — add KV-cache (grows with context and concurrency) and ~1–2GB runtime overhead. Formula and cache math in the VRAM guide.
Quick start
Ollama and LM Studio handle the harmony chat format automatically — use a current version, older builds garble the reasoning channel. Set reasoning effort in the system prompt ('Reasoning: high') for hard problems; keep it low for chat and summaries. In pipelines, show users only the final channel, not the analysis (thinking) channel.
Expected performance
| Hardware | Generation speed |
|---|---|
| RTX 4060 Ti 16GB | ~30–45 tok/s |
| RTX 3090 | ~70–100 tok/s |
| RTX 4090 | ~100–140 tok/s |
| Mac M-series Max | ~50–70 tok/s |
Indicative single-user figures (llama.cpp/Ollama class runtimes); multi-user serving via vLLM multiplies total throughput 10–20× through batching.
Tips & gotchas
- Reasoning effort is the main quality dial: 'high' rivals much bigger models on math and code, but multiplies latency — set it per task, not globally.
- Its factual recall is thinner than dense models of similar quality — pair it with RAG for knowledge work instead of trusting closed-book answers.
- Excellent tool-caller: it was trained for agent loops, making it a strong local backbone for function-calling apps.
- Fully permissive — OpenAI's first open-weight release since GPT-2, free for commercial use.
FAQ
Is gpt-oss-20b really free for commercial use?
Yes — Apache 2.0, the same fully permissive license as Qwen and Mistral open models. It's open-weight (weights published), not open-source (no training data/code).
gpt-oss-20b or Phi-4 on a small card?
gpt-oss-20b is stronger on reasoning and agent/tool work and much faster per token; Phi-4 fits smaller cards (8–12GB). If you have 16GB, gpt-oss is the more capable pick.
Why is it so fast for a 21B model?
Mixture-of-experts: only ~3.6B parameters compute per token, so it generates at small-model speed while drawing on 21B of capacity.