Run LLMs locally: model-by-model guides
One page per model: exactly how much VRAM each quant needs, the honest minimum hardware, the Ollama command, expected tokens/sec, and the gotchas specific to that model. All figures follow our sizing method and the quantization guide.
Not sure which model? Start from your GPU: 12GB → Phi-4 · 16GB → Mistral Small or gpt-oss-20b · 24GB → Qwen3.6 27B (Gemma 3 for vision, DeepSeek-R1 for reasoning, Qwen3-Coder for coding) · 48GB+ → Llama 3.3 70B or DeepSeek-R1 70B · 80GB/unified → gpt-oss-120b.
Qwen3.6 27B
The 24GB-card sweet spot: coding, strong Italian, and a switchable thinking mode.
Read the guide →Llama 3.3 70B
The 70B-class reference: top open-weight general quality, if you have 48GB.
Read the guide →DeepSeek-R1 32B (distill)
Frontier-style reasoning on a 24GB card — it thinks before it answers.
Read the guide →Gemma 3 27B
Vision + strong multilingual on 24GB — with official QAT builds that shine at 4-bit.
Read the guide →Mistral Small 3.1
The efficiency champion: near-27B quality that fits a 16GB card — and it's European.
Read the guide →Phi-4 14B
The small giant: 14B punching far above its size — runs on a 12GB card.
Read the guide →gpt-oss-20b
OpenAI reasoning on a 16GB card: MoE speed, native 4-bit weights, adjustable thinking effort.
Read the guide →gpt-oss-120b
Frontier-class reasoning that fits one 80GB GPU — or a 128GB unified-memory mini-PC.
Read the guide →Qwen3-Coder 30B
The local coding agent: repo-scale 256K context, agentic tuning, small-model speed on a 24GB card.
Read the guide →DeepSeek-R1 70B (distill)
The strongest open reasoner below datacenter scale — R1 thinking grafted onto Llama 70B.
Read the guide →Two rules from our guides worth repeating: down to 4-bit, a bigger quantized model beats a smaller full-precision one at equal memory; and with good RAG, retrieval quality moves answer quality more than model size. Full context: GPU buyer guide · real TCO · On-Premise Observatory.