Knowing that an LLM has made a decision is no longer enough. In regulated settings, those deploying models on-premise need to understand exactly which signals — and especially which combinations of signals — drove a prediction. IMEX (Interaction-Based Model Explanation) was born precisely for this: it does not merely rank relevant features but maps the interactions that matter, even beyond pairwise combinations.

The underlying idea is that many predictive behaviors escape purely additive explanations. Two variables taken individually may appear negligible, while their joint action explains the target far better than a simple sum of contributions would. IMEX captures this complexity through two metrics: Static Correlation Power (PCS), which quantifies the contribution of a feature considered in isolation, and Interaction Correlation Power (PCI), which measures the non-additive part of the joint effect.

The validation published by the authors focuses on PCS, comparing it with the INVASE method on three synthetic datasets with known structures. The results show that IMEX recovers relevant structures even in the presence of non-linearity, conditional dependencies and multicollinearity. No hardware benchmarks, no throughput metrics: it is a methodological study. But that is the point, and for those dealing with on-premise deployments it deserves attention.

Why interactions matter in self-hosting

When an organization decides to keep its models in-house — for data sovereignty, compliance or simply TCO reasons — the explainability problem becomes concrete the moment something goes wrong. Fine-tuning on proprietary data can trigger unexpected behaviors, and without tools like IMEX it becomes difficult to isolate whether the culprit is a specific feature or a combination poorly digested by the model. In a local stack, the ability to profile inference is no longer a nice-to-have: it is an audit tool.

Those working with self-hosted LLMs know that quantization, for example, can alter the internal representation of tokens and thus modify interactions among latent features. An explainability framework that does not stop at the individual attribute but also explores second- and third-order effects helps validate that the compressions applied to run the model on more modest hardware are not distorting important causal chains.

The open workshop of higher-order interactions

There is another signal. The absence of limits on the order of interactions analyzed by IMEX suggests that the community is seeking methods capable of keeping pace with the growing complexity of models. This is no accident: the more a system becomes capable of compositional reasoning, the more one-dimensional explanations become misleading. On a practical level, for a team managing on-premise inference, this means that investment in explainability tooling cannot stop at libraries that return a heatmap. The ability to query the model on suspicious combinations is needed, especially when the deployment involves sensitive data and every automated decision must be justifiable.

Ultimately, IMEX does not solve the problem of model transparency, but it points in a clear direction: explainability cannot remain anchored to individual features. Those designing validation pipelines for local environments would do well to keep this in mind, because the real game is played on the interactions that remain invisible until someone goes looking for them.