Ask an open-source instruct LLM to say something about itself and the answer almost always carries a cautionary layer: "I'm just an AI", "I don't feel emotions". A new study shows that layer is not a stable property of the model. It is partly an effect of the chat template, the format in which text is packaged before reaching the inference engine.
The template works like a switch. When present, it turns up the disclaimer voice and turns down the experiential voice, made of phrases like "I feel". When absent, the effect reverses. The researchers observed this across eight popular open-source instruct models up to 9 billion parameters. Inside the activations of three of those models, they isolated a vector direction that drives the behavior: removing it turns disclaimer voice down, adding it turns it up; a random direction of the same size has little effect. Even more striking: an instruct model without a chat template, when the disclaimer direction is added artificially, begins to disclaim as if the template were there.
This is the structural point for anyone working with self-hosted LLMs or evaluating them for on-premise deployment. The template is not a cosmetic detail: it is part of the serving infrastructure, often modified by operators to adapt a model to an application. If the self-referential voice depends on that configuration, then safety tests, audits, and alignment analyses that read the model's answers about itself are also measuring the setup, not only the weights. A self-description can change without touching the model, simply by changing the template or intervening in activation space.
That has second-order consequences. For teams running red teaming or compliance evaluations on local models, separating learned behavior from the template's contribution becomes a methodological necessity. Otherwise there is a risk of attributing to the model a "personality" that is actually a deployment choice. At the same time, the discovery of an activation direction offers a surgical instrument: the disclaimer voice can be modulated without fine-tuning, through an intervention in latent space. In air-gapped environments or with data sovereignty constraints, where fine-tuning may be expensive or undesirable, that possibility has practical value.
But the naive reading should be avoided. The study does not say that models have a consciousness to be silenced or amplified. It says the opposite: what an LLM says about itself is not a fact about the model, because it does not emerge only from weights. It is also a function of the template and can be steered. For analysts studying self-reports and artificial introspection, this is a confound to control for. For those evaluating on-premise deployment, AI-RADAR offers analytical frameworks on /llm-onpremise to distinguish what belongs to the model from what belongs to the serving infrastructure.
๐ฌ Comments (0)
๐ Log in or register to comment on articles.
No comments yet. Be the first to comment!