The largest of the R1 distills: DeepSeek's reasoning behavior transferred onto a Llama 3.3 70B base. You get the <think>-before-answering pattern of the full R1 with materially better raw capability than the 32B distill — at 70B-class hardware prices (~43GB of weights at Q4). It's the model for the hardest problems you can pose locally: competition math, gnarly debugging, multi-step planning. For everything else it's overkill — thinking tokens at 70B speeds make every answer expensive.
VRAM by quantization level
| Quant | Weights | Fits on |
|---|---|---|
| Q4_K_M | ~43 GB | 48GB card, or 2×24GB (layer split) |
| IQ2/IQ3 | ~26–32 GB | single 32GB — quality loss; the 32B distill at Q4 is usually the better trade |
| Q8_0 | ~75 GB | 80GB datacenter card |
Weights only — add KV-cache (grows with context and concurrency) and ~1–2GB runtime overhead. Formula and cache math in the VRAM guide.
Quick start
Same operational rules as the 32B distill, amplified: num_ctx 16K+ so thinking never truncates, temperature ~0.6, strip the <think> block in pipelines. Latency budgeting is the real design task — at ~14 tok/s on dual 3090s, a 3,000-token thinking chain adds ~3.5 minutes before the first visible word. Route only genuinely hard problems here.
Expected performance
| Hardware | Generation speed |
|---|---|
| 2× RTX 3090 (Q4, layer split) | ~12–16 tok/s |
| RTX A6000 48GB (Q4) | ~10–14 tok/s |
| A100/H100 80GB | ~25–35 tok/s |
| Mac Studio 96GB+ (Q4) | ~6–9 tok/s |
Indicative single-user figures (llama.cpp/Ollama class runtimes); multi-user serving via vLLM multiplies total throughput 10–20× through batching.
Tips & gotchas
- Total answer time = (thinking + answer) ÷ tok/s. At 70B speeds that's minutes, not seconds — build UX and timeouts around it.
- Three-tier routing works beautifully: small model → R1 32B → R1 70B, escalating only on failure. Reasoning-per-euro stays sane.
- MIT from DeepSeek; built on Llama 3.3, so Meta's community license also applies — fine for almost all business use.
FAQ
Is it worth 2× the VRAM of the 32B distill?
Only if you routinely hit the 32B's ceiling on math, hard code or planning. For most workloads the 32B answers the same questions in half the memory at twice the speed.
R1 70B or gpt-oss-120b?
Different economics: R1 70B fits 48GB but generates slowly; gpt-oss-120b needs ~61GB but generates several times faster. If your memory allows it, gpt-oss usually wins on throughput per answer.