Twenty-five meal photos, a public dataset, and one practical question: which multimodal LLM can estimate calories while staying within 20% error? The micro-benchmark posted on Reddit by mr_tolkien doesn't aim to be a paper, but it says a lot about how we choose models when the goal is a concrete task rather than an abstract score.

The author used Nutrition5k images and a tool with access to calorie data from USDA FoodData Central and MEXT. Models were scored on the share of meals where the estimate landed within 20% error, across the same 25 randomly selected meals. Models too large for the author's machine were run through OpenCode Go/OpenRouter; the rest ran locally.

The results upend the assumption that more parameters automatically mean more reliability. Muse Spark 1.3, a model expected to be released with open weights, hits the target in 48% of cases, with a median error of 45 kcal and a slight average undercount (-24 kcal). At the opposite end, Qwen 3.8 27b stops at 16%, with a mean bias of +64 kcal and a median error of 148 kcal. In between, models such as GLM 5.3 Flash and DeepSeek v4 Flash Vision show intermediate, non-monotonic performance relative to size.

Model % within 20% Mean bias Median error
Qwen 3.8 27b 16% +64 kcal 148 kcal
GLM 5.3 Flash 28% +18 kcal 65 kcal
Qwen 3.8 Max 32% -11 kcal 48 kcal
Muse Glimmer 30b 32% +25 kcal 92 kcal
Qwen 3.8 Flash 36% +2 kcal 91 kcal
DeepSeek v4 Flash Vision 40% +52 kcal 65 kcal
Muse Spark 1.3 48% -24 kcal 45 kcal

The most interesting point isn't the overall winner, but the inversion between Muse Glimmer 30b and Qwen 3.8 27b: the former clearly beats the latter despite a modest parameter difference. For anyone working with consumer hardware around 32GB of VRAM, this carries weight: a 1.3-billion-parameter model can run locally with VRAM headroom, while a 27B demands aggressive quantization or a cloud API. If the task is calorie estimation from photos, the smaller model can be not only sufficient but better.

The author's setup is hybrid by necessity: some models run locally, others on OpenRouter. That's the picture for many self-hosted projects, where the VRAM threshold decides what stays on-premises and what goes to the cloud. An open 1.3B model reaching the top result tilts the balance toward leaner infrastructure, reducing dependence on external APIs and operational costs.

The benchmark isn't scientific: 25 meals is a small sample, there are no confidence intervals, and prompt or temperature choices can shift the results. But it's useful as a qualitative signal, especially because it mirrors how an individual developer picks a model: try, compare, then decide based on their own hardware and use case.

For those weighing local versus cloud execution, there are trade-offs to map: latency, image privacy, inference costs, and hardware maintenance. AI-RADAR offers analytical frameworks on /llm-onpremise to compare these scenarios without shortcuts. If a 1.3B model can beat a 27B on such a specific visual task, how many other application niches are paying for size without getting the value?