The problem with AI benchmarks is no longer only technical: it is conceptual. When a test is calibrated on the abilities of humans or current models, it becomes circular or saturates quickly. The new work states this clearly and proposes an escape route based on inverse problems, where difficulty depends on the complexity of the underlying mechanism given observations and prior knowledge. The chosen measure is Levin's Kt, a computable variant of Kolmogorov complexity that sets a theoretical upper bound on difficulty by balancing two elements: the simplest representation encoding for the mechanism and the logarithm of the time steps needed to run it.

The decisive move is using reasoning LLMs as reference machines. Previous tests, such as the original C-test, were based on minimalistic universal Turing machines, capable of very limited representations and no prior knowledge. Here, large language models are treated as universal Turing machines that capture a compendium of human culture and interpret prompts as natural language programs. This is not a cosmetic difference: the power of the reference machine defines the scale on which difficulty is measured. The study formalises the measure, builds a lightweight generation and certification pipeline, and analyses precisely the effect of the power gap between examiner and examinee.

The result is a first battery of certified items administered to twelve LLMs. The agent characteristic curves decline as difficulty increases, down to a high-difficulty region the authors call the supra-intelligence window: items that can be certified but remain beyond the abilities of current generations of frontier intelligence. The scale is a ratio scale measured in bits, and it allows fresh items to be produced at high difficulties before any human or frontier model can solve them.

For those evaluating models in self-hosted or air-gapped environments, this approach has structural implications. An absolute metric, not tied to percentage scores or public leaderboards, reduces the risk of benchmarks polluted by training data or widespread overfitting. But it introduces a new dependency: the generation and certification pipeline must be able to run locally, without relying on external APIs, and the choice of the reference model becomes an infrastructure decision as much as the metric itself. This is not only a matter of evaluation: it is how the measurement of capability intertwines with data sovereignty and control of compute resources. AI-RADAR offers analytical frameworks at /llm-onpremise to evaluate these trade-offs, but the underlying point is that AI certification is becoming a deployment problem, not only a theoretical one.