LLMs & Large Language Models
Large Language Models are neural networks trained on vast text corpora to understand and generate language. This pillar covers how they actually work, the modern model taxonomy (dense, MoE, reasoning, multimodal), the open-weight landscape, how to choose a model for a real workload, and the adaptation ladder — from prompting to RAG to fine-tuning.
On This Page
How LLMs Actually Work
At the core sits a deceptively simple objective: predict the next token. A transformer network with billions of parameters reads a sequence of tokens (word fragments) and, using self-attention — every token weighing its relevance to every other — outputs a probability distribution over what comes next. Trained across trillions of tokens of text, that prediction task forces the network to internalize grammar, facts, styles, code patterns and a workable amount of reasoning: to predict well, you must model the world that produced the text.
Raw next-token predictors ("base models") are then shaped by post-training: instruction tuning teaches them to follow requests; preference optimization (RLHF/DPO) aligns tone and helpfulness; tool-use training teaches them to emit structured function calls. Modern reasoning models add a further stage — trained to produce long internal chains of thought before answering, buying accuracy on math, code and multi-step problems at the price of more tokens (and therefore latency and memory) per answer.
💡 The practical consequence: an LLM is a probabilistic text engine, not a database. It will sometimes produce fluent falsehoods ("hallucinations") — which is why production systems ground it with retrieval (RAG) and require citations, rather than trusting recall.
Core capabilities
- Generation & drafting: coherent, contextually appropriate text in any register
- Question answering: extracting or synthesizing answers — strongest when grounded on provided context
- Code: writing, explaining, reviewing and translating code
- Transformation: summarization, translation, rewriting, structured extraction (text → JSON)
- Reasoning & agency: multi-step problem solving and tool use — the newest and most uneven capability
The 2026 Model Taxonomy
The old encoder/decoder distinctions matter little in practice today — virtually every model you'll deploy is a decoder-only transformer. The distinctions that do drive decisions:
Dense vs Mixture-of-Experts (MoE)
Dense models activate every parameter for every token. MoE models route each token to a few "expert" blocks — a 120B-total/12B-active MoE needs memory for all parameters but generates at the speed of a 12B. Rule: budget memory by total params, predict speed by active params.
Deployment consequence: MoE models are ideal for unified-memory machines (Macs, Strix Halo) — huge capacity, low per-token bandwidth. See MoE deployment.
Standard vs Reasoning models
Reasoning models "think out loud" before answering — dramatically better at math, hard code and planning; slower and token-hungrier everywhere. Many open models now ship hybrid modes (thinking on/off per request).
Deployment consequence: reasoning bursts inflate context usage and latency budgets; don't route routine summarization through a thinking mode.
Frontier vs small language models (SLMs)
Capability per parameter keeps rising: today's 8–14B models outperform 70B models from two years ago on most practical tasks. SLMs (1–14B) are fast, cheap, private and fine-tunable on consumer hardware — the workhorse class for grounded enterprise tasks.
Deployment consequence: with good retrieval, a small model quoting the right document beats a big one guessing. See the SLM guide.
Text-only vs multimodal
Vision-language models (VLMs) read images/documents natively — invaluable for scanned-PDF pipelines and shop-floor use. Open-weight VLMs (Qwen-VL, Gemma, Llama vision lines) now run locally on the same runtimes.
The Open-Weight Landscape
"Open-weight" means the trained parameters are downloadable and self-hostable — the foundation of every on-premise deployment. The major families, by what they're each known for:
- Llama (Meta): the ecosystem anchor — broadest tooling/quant support, strong general quality, custom license (fine for most business use; read it).
- Qwen (Alibaba): aggressive releases across sizes and MoE variants, excellent multilingual (including Italian) and coding performance, mostly Apache 2.0.
- Mistral (EU): efficient dense and MoE models with strong quality-per-parameter; the European champion — relevant where EU origin matters procurement-wise.
- Gemma (Google): compact, efficient, strong multilingual; QAT (quantization-aware) editions notably robust at 4-bit.
- Phi (Microsoft): small models trained on curated data punching above their size — edge and CPU-class deployments.
- DeepSeek: open reasoning models and huge MoEs (MIT-licensed) — the R1 distills brought reasoning to 24GB cards.
⚠️ Licensing: "open-weight" ≠ do-anything. Apache 2.0/MIT models are genuinely permissive; Llama-style community licenses restrict some uses; and under the EU AI Act, fine-tuning and productizing can shift regulatory duties onto you. Keep a license register per model — see the AI Act guide.
Choosing a Model: A Decision Method
- Start from the task, not the leaderboard. Document Q&A with RAG → 8–14B suffices. Drafting quality and nuance → 27–70B. Hard math/code/planning → a reasoning model. Structured extraction at volume → a small fine-tuned model.
- Test in your language. Instruction-following quality varies across languages far more than English benchmarks suggest — for Italian workloads, evaluate Qwen/Gemma/Mistral candidates on your prompts.
- Distrust benchmarks, run evals. Public benchmarks are saturated and contaminated. Build ~50 questions with known answers from your real workload; that tiny harness beats every leaderboard for your decision.
- Size to your hardware honestly. The biggest model at 4-bit your VRAM holds, then spend leftover memory on context or a better quant — the math is in the VRAM guide.
- Check the boring things: license, tool-calling training (critical for agents — precision degrades before prose under quantization), context length actually usable (not just advertised), and quant availability.
Context Windows & Memory
The context window is the model's working memory — everything it can "see" for the current answer. Modern models advertise 128k–1M+ tokens, with three caveats that matter in production: cost (the KV-cache grows linearly with context and can rival the model weights in VRAM at 100k+ tokens), recall degradation (models measurably lose precision retrieving facts from the middle of very long contexts), and latency (prompt processing time grows with input length). This is why giant contexts complement retrieval instead of replacing it: RAG selects the relevant few thousand tokens; the long window makes room for richer selections.
Deployment Options
Cloud APIs
Hosted frontier models (OpenAI, Anthropic, Google): zero infrastructure, newest capabilities, metered cost, and your prompts leave the building. Right for spiky workloads and the hardest reasoning tasks.
Self-hosted (on-premise)
Open-weight models on your hardware: data sovereignty, frozen versions (no vendor changing the model under your validated system), near-zero marginal cost at high utilization. Requires GPUs and ops discipline. The full decision framework is in on-premise vs cloud and the On-Premise Observatory.
Hybrid (what most teams should run)
Local models for the bulk of requests; escalate flagged-hard or non-sensitive tasks to a cloud API behind a gateway you own. Route by data classification first, difficulty second. Build against the OpenAI-compatible API shape so switching backends stays a config change.
The Adaptation Ladder
Making a general model good at your job is a ladder — climb only as far as the problem demands:
Resources and Further Reading
Deep guides on AI-Radar
- LLM quantization explained · VRAM sizing · GPU buying
- RAG vs fine-tuning · Build a private ChatGPT
- The local LLM software stack · Ollama vs LM Studio
- Run models locally — model-by-model guides (Qwen, Llama, DeepSeek-R1…)
- Model cards · MoE deployment · Small language models
Recent LLM Articles
Last updated: July 2026 | Evergreen reference, reviewed as the model landscape moves