The open-source leaderboard whatmodel.sh, just published, aggregates benchmark results with unusual transparency: 22,113 results, 987 benchmarks, 1,011 sources, each linked to the original publication. For text models, the picture is even more granular: 18,755 results for 1,211 models across 617 benchmarks. One specific fact emerges: China's open-weight models trail US closed models by about seven months. The source also separates 14,436 independent evaluations from 7,677 lab-reported scores.

The seven-month gap is notable, but the data architecture matters at least as much as the ranking. In a sector where proprietary benchmarks are often cherry-picked, distinguishing evaluations run by third parties from scores reported by labs helps filter out noise. Here independent evaluations are nearly double the lab-reported ones. That changes the reading: we are not looking at a race between vendors grading themselves, but at a corpus where academic researchers and independent organizations ran the models themselves. That methodological step is crucial for anyone deciding which LLMs to bring into production.

For a self-hosted deployment, the temporal gap between Chinese open-weight models and US closed models has concrete implications. An open-weight model can be run locally, without third-party APIs and without sharing sensitive data with external endpoints. That reduces a data sovereignty risk, but moves the cost elsewhere: you need infrastructure with adequate VRAM, quantization choices to reduce memory footprint, and an optimized inference pipeline. TCO is not just the price of GPUs, but also the staff managing updates, audits, and maintenance. Anyone evaluating this kind of deployment cannot stop at the leaderboard: they must check whether the open-weight model maintains quality after quantization, whether the serving framework handles the required context, and whether the licence allows the intended commercial use. AI-RADAR offers analytical frameworks at /llm-onpremise to compare these trade-offs.

There is a second-order effect: when open-weight models get closer to closed models, the incentive for cloud vendors to keep proprietary APIs strengthens, but so does the local alternative. Companies that have already invested in on-prem infrastructure can evaluate Chinese open-weight models not for ideological reasons but to reduce dependence on a single vendor. That does not remove constraints: transparency of published benchmarks is necessary but not sufficient, and open-weight models still require continuous updates, testing on internal data, and regression monitoring. An open dataset under CC BY-SA 4.0 with MIT code, however, makes independent verification of lab claims possible.

The leaderboard does not say when Chinese open-weight models will reach parity, and it does not claim to replace internal evaluations. But it shows that the race is no longer only between models: it is a race between measurement infrastructures. Those with independent, reproducible benchmarks have an informational advantage in deciding what to run on-prem. In that light, the seven-month lag is less a verdict than a starting point for a different question: in production, how much is the difference worth between a closed model via API and an open model that can be run and verified in-house? The answer is not in any single score, but in the quality of the data used to choose.