A thread on the LocalLLaMA subreddit broke an unwritten convention: the author, user ForsookComparison, lists positions that challenge the groupthink of the community focused on open-weight and self-hosted LLMs. These are not scattered complaints, but a targeted attack on several pillars of local evaluation: the Artificial Analysis indexes, the comparison between open and closed models, the interpretation of the Strata project, and the controversial ThreeJS benchmark.

The first point touches a raw nerve. According to the author, the Artificial Analysis V5 index contains questionable choices, such as the preference for gemini-flash over qwen3.8-2.4T, but in version V4 the open-weight models were ranked too highly. The LocalLLaMA community would therefore have welcomed some corrections, but continues to defend "darling" models like GLM 5.3, which in his view does not hold up against Grok. This is a typical dynamic of aggregated rankings: when a leaderboard becomes an identity benchmark, methodological corrections are read as betrayals.

The Haiku 5.5 case is equally instructive. The author argues that those who react vehemently to the idea that it could challenge deepseek-v4.1-flash-1017 should spend more time testing it. In an on-premise context, where VRAM, quantization, and inference speed determine the real TCO, a model that loses on some synthetic parameters but excels in concrete tasks may be more convenient than a leaderboard winner. The risk is that the preference for open-weight models becomes a bias that excludes valid options simply because they do not align with the narrative of total sovereignty.

The Strata issue is a case of projected expectations. The 0.1 release had answered an explicit request from many users, but the project has its own direction; those who follow it should evaluate its roadmap, not judge it for what the community hoped it would become. In an ecosystem of open-weight projects often sustained by volunteer contributors, confusing collective desire with the maintainer's goal is an analytical error that can lead to premature adoption choices.

The ThreeJS benchmark deserves a separate discussion. The author calls it something to ignore, unless it specifically serves to generate 3D assets. His observation is stark: for over a year he has seen no correlation between a model's ability to animate a pagoda and its effectiveness in general or coding work. If true, the ThreeJS benchmark has polluted many local evaluations. In a self-hosted environment, where every gigabyte of VRAM and every second of latency count, adopting models based on metrics disconnected from your real workload is a hidden cost: you risk optimizing for an irrelevant task and discarding alternatives better suited to your pipelines.

The question posed by the author remains open: which other community convictions deserve to be questioned? Dissent, when argued, is a tool of intellectual hygiene. For those evaluating on-premise deployments, the lesson is not to take sides, but to build your own tests on real cases, without delegating the choice to leaderboards or to a consensus that, as this thread shows, can be fragile.