The analogy is tempting: if a problem requires more tokens, it should be harder. But for a Large Language Model the reasoning does not hold. A group of researchers has disproved the hypothesis that token consumption during inference grows monotonically with classical time complexity. The reason lies in how LLMs allocate computation during generation: a longer output does not always correspond to proportionally more computational work.

From this observation comes LLM Complexity, a metric that shifts attention from raw token count to the shape of the accuracy-token curve. The goal is no longer how many tokens are produced, but how much accuracy is obtained per token consumed. The area bounded above by the curve becomes a measure of complexity from the model's point of view. Paradoxically, problems with long outputs can turn out to be less complex than short ones if the additional tokens do not translate into real improvements.

Cross-model stability and the curriculum

The most relevant finding is the consistency of rankings. Different models, under the same evaluation, produce very similar difficulty orders under LLM Complexity. This suggests the metric captures stable task properties rather than the idiosyncrasies of a single model. The stability also holds in real-world tasks, where human-annotated difficulty labels show less consistent cross-model correlations.

The authors use this property to build an easy-to-hard curriculum for supervised fine-tuning. The results point to a double advantage: accuracy improves and token consumption falls. In a self-hosted scenario this has immediate operational weight. Fewer tokens to reach the same accuracy means less inference time, less pressure on VRAM and serving pipelines, and a lower TCO. The key is not necessarily changing hardware, but reordering the data presented to the model during training.

Winners, losers, and structural signals

For teams adapting models on their own infrastructure, the metric introduces a data-ordering criterion that can extract more value from the same compute budget. By contrast, monitoring systems built almost exclusively on token throughput risk looking at a partial indicator: the quantity of tokens generated says nothing about the marginal utility of each token for the task. Benchmark construction based on human difficulty labels also weakens when cross-model stability is the goal.

The structural signal is broader. The industry is moving from pure productive capacity to conditional efficiency: not just how many tokens per second, but how much accuracy per token. This affects serving pipeline design, on-premise workload planning, and contracts where the variable cost of inference is still tied to token count. The metric does not solve deployment dilemmas, but it offers a way to avoid reading them too simplistically.

One question remains open: how much does this model-calibrated complexity reflect the difficulty perceived by users? Cross-model stability is an advantage for automation and data reordering. But the risk is a self-referential optimization, measuring how hard a task is for LLMs without considering whether that task is actually useful for the people using them.