gpt-oss-120b is the strongest openly-licensed model you can serve on a single accelerator: 117B total parameters, 5.1B active per token, shipped in native MXFP4 so the weights land around 61GB. That means one H100/A100 80GB — or, remarkably, high-end unified-memory machines: a 96GB+ Mac Studio or a 128GB Ryzen AI Max mini-PC run it at usable speeds because the MoE only computes 5.1B parameters per token. Benchmark positioning is o4-mini-class reasoning: this is the open model you graduate to when the 20–70B tier isn't smart enough.
VRAM by quantization level
| Quant | Weights | Fits on |
|---|---|---|
| MXFP4 (native) | ~61 GB | 80GB GPU (H100/A100); Mac Studio 96GB+; 128GB unified-memory mini-PC |
| MXFP4 + 128K context | ~70+ GB | 80GB GPU with KV-cache quantization; 128GB unified comfortably |
| Multi-GPU split | 2×48GB / 4×24GB | workstation alternative — llama.cpp/vLLM layer or tensor split |
Weights only — add KV-cache (grows with context and concurrency) and ~1–2GB runtime overhead. Formula and cache math in the VRAM guide.
Quick start
On unified-memory machines, close everything else — the OS needs headroom beyond the ~65GB working set. For serving, vLLM on an 80GB card is the production path (the MoE batches beautifully). Same harmony-format and reasoning-effort notes as gpt-oss-20b apply. Renting an H100 by the hour is the cheapest way to evaluate it before committing to hardware.
Expected performance
| Hardware | Generation speed |
|---|---|
| H100 80GB | ~120–180 tok/s |
| A100 80GB | ~60–90 tok/s |
| Mac Studio M-Ultra 96GB+ | ~40–70 tok/s |
| Ryzen AI Max 128GB (unified) | ~25–40 tok/s |
Indicative single-user figures (llama.cpp/Ollama class runtimes); multi-user serving via vLLM multiplies total throughput 10–20× through batching.
Tips & gotchas
- The 5.1B active parameters mean generation speed is closer to a 7B than a 70B — the bottleneck on unified memory is bandwidth, not compute.
- Compare it against Llama 3.3 70B for your workload before buying anything: the 70B needs less memory but answers slower and reasons worse.
- Fully permissive — frontier-class open weights with no usage gates.
FAQ
Can any consumer setup really run a 117B model?
Yes — the native 4-bit MoE is the trick. 128GB unified-memory mini-PCs and 96GB+ Macs run it at 25–70 tok/s. A quad-3090 workstation also works via layer split.
gpt-oss-120b or Llama 3.3 70B?
gpt-oss-120b reasons better and generates faster, but needs ~61GB vs ~43GB. Llama remains stronger purely as a long-form writer. For agents, code and analysis: gpt-oss.