What Actually Runs on 10-16GB VRAM: A Field Ranking of Local LLMs

🤖 Automatically generated. This article was produced by AI-Radar's automated system from the source cited above — it was not written or fact-checked by a person. Always verify against the original source. How this works

The local model race is no longer just about parameter counts. A ranking posted by a user on Reddit sorts LLMs that run on GPUs with 10-16GB of VRAM and 26-32GB of RAM, using real-world criteria: a minimum of 10 tokens per second, narrative coherence, handling of typos and nuance, and frequency of slop patterns. The result is a useful snapshot for self-hosted inference on consumer hardware, not an academic benchmark.

Field metrics, not synthetic benchmarks

The evaluation does not reward size per se. The author compares MoE and dense models, tests prompts with 23 intertwined rules, and deliberately ignores CoT models as slower and unnecessary for conversational workloads. A key constraint emerges: having enough VRAM to load weights is not enough; you also need an inference stack that handles long contexts and mechanisms such as FlashAttention, SWA, and SmartCache. In the test, some models like Qwen 35B-A3B and GLM 31B-A3B require llama.cpp instead of Kobold, a sign that loader integration with new MoE architectures is still a work in progress.

Looking at the list, Gemma 4 26B-A4B ranks high for dialogue and precision, while Qwen 35B-A3B is fast but tends to overfit previous patterns. The Gemma family comes across as creative and funny, Qwen as inquisitive but sometimes overly elaborate, Mistral as brilliant in dialogue but weak at following complex rules, and Llama as repetitive. These are not marginal flaws: for anyone developing agents or conversational roles on local hardware, model personality can determine whether you need extra instructions or a different architecture.

Knowledge remains above 200B

The most structural finding concerns knowledge. The author tested anime, manga, movies, and music knowledge and found that only models above 200B parameters reliably hold this kind of information; a 120B rarely does. This draws a sharp line: in the 10-16GB VRAM range you can get fluent and fast conversations, but you cannot expect encyclopedic knowledge or specific cultural references. The trade-off is between operational agility and informational depth, and quantization alone does not solve it.

This asymmetry has second-order consequences. It pushes teams that prioritize data sovereignty and low TCO toward narrow use cases where the local model does not need to know everything, only to perform well in a domain. At the same time, it fuels demand for hybrid architectures: local inference for interaction and control, remote or cloud models for general knowledge. The test also signals that the race toward compact MoE models is expanding the sub-16GB offering, but serving software and loaders still need to mature to exploit them without friction. For organizations evaluating on-premise deployment, AI-RADAR offers analytical frameworks at /llm-onpremise to map these trade-offs.