Imagine having to decide whether a sequence of symbols follows an involution rule (an operation that, applied twice, returns the starting element) or a three-step cycle. A 2.8-million-parameter transformer not only distinguishes the two hypotheses, but does so with an entropy that differs by just 0.01 bits from the optimal Bayesian calculation — using both traditional integer tokens and opaque symbols whose meaning is reshuffled at each episode. This is the result of a "Bayesian wind tunnel," a controlled environment where the posterior distribution over hypothesis classes is known in closed form, described in a recent study on model selection in transformers.

The Bayesian wind tunnel concept extends prior work showing that transformers can perform exact Bayesian filtering within a fixed hypothesis class. The next step asks whether they can do model selection: identify which hypothesis class best explains the data, without having it defined in advance. The wind tunnel metaphor is apt: instead of testing models in the real world, one creates lab conditions where the "truth" is known, allowing precise measurement of how close the model's behavior gets to the theoretical optimum.

The choice of fixed-point-free involutions — functions where f(f(x)) = x — is deliberate. It is a purely relational property that requires no arithmetic. In this scenario, the transformer shows near-perfect agreement with the Bayesian reference. But the real surprise comes with non-nested comparisons: for example, discriminating between involutions and 3-cycles, where neither class is a subset of the other. Here the mean absolute error on the class posterior distribution drops below 0.001, showing that the model does not merely favor the simpler hypothesis; it performs genuine model selection.

However, the picture changes radically when the discriminative statistic becomes arithmetic. If the distinction between classes relies on modular addition (rotations) or modular multiplication (f(x) = cx mod p), model selection works only with integer tokens. With opaque symbols — units with no stable identity, whose mapping changes every episode — the model fails completely. And this boundary is not a matter of scale: growing the model from 2.8 to 316 million parameters (a 112x factor) does not alter the outcome. The ability to do arithmetic vanishes if tokens have no fixed semantics.

A stationarity control confirms the diagnosis: if the opaque symbols are relabeled in a fixed way (so the same relation keeps the same symbol across all episodes), the model succeeds again, with a 0.009-bit agreement. It is not integer identity that matters, but stable semantics that allow the transformer to compile arithmetic circuits. Subtask diagnostics further point to the composition of header inversion with the arithmetic operation, not header parsing itself: the failure lies downstream, in the integration of control and computation.

The study does not stop at lab-scale models. Probes — necessarily lossy, hence directional but not exact — on frontier language models show qualitatively Bayesian behavior, but with a calibration gap of about 55 times. In other words, these models appear to move in the right direction, yet their probabilities are far from the reliability of a Bayesian agent.

For those developing or evaluating LLM-based systems, these results offer a valuable lens. The distinction between syntactic pattern matching and the ability to compile algorithms from symbols with stable identity is central to deciding where a model can be trusted. In high-stakes on-premise settings — such as financial analysis, diagnostics, or process control — knowing that arithmetic requires semantic grounding raises precise questions about training and tokenization. Bayesian wind tunnels, for all their abstraction, provide a rigorous testbed for separating appearance from substance, far beyond conventional benchmarks.