When a system decodes language from the brain, the question is not only whether it works, but what it is actually reading. Large Language Models have accelerated progress in brain-language decoding, but they bring a fundamental ambiguity: does the output text reflect neural representations, or is it largely reconstructed by the language model itself? MD-SigLIP, a margin-regularized structured semantic alignment framework, tackles this problem at its root.

The research team proposes to align brain embeddings and text embeddings directly in a shared semantic space, enabling retrieval-based decoding. Instead of asking the LLM to generate a sentence from neural signals, the system selects the closest item from the semantic space. This makes the correspondence between neural representations and language semantics explicit, rather than leaving it implicit in the generative process.

The technical core is duplicate-aware sigmoid contrastive learning, extended with a listwise margin-regularized term. This term enforces structured ranking constraints between positive semantic clusters and negative samples. In practice, the model must respect a hierarchy of similarities, not just classify. The multi-positive structure and margin-based ordering are modeled together, capturing the manifold organization of language embeddings reflected in neural signals. Experiments show state-of-the-art retrieval performance on both full-vocabulary and subset evaluations.

The choice of retrieval is not a minor detail: it changes the role of the LLM. In a generative architecture, the model can fill gaps with its own linguistic statistics, producing plausible output that may not be tied to the neural signal. Here, the model's contribution remains anchored to a predefined semantic space. For neural data, which is among the most sensitive, this approach has a practical consequence: the pipeline can be run locally, with a self-hosted semantic index and without sending signals to a cloud API. This reduces the exposed surface and simplifies compliance with residency and consent constraints.

There is also a second-order effect: if retrieval-based decoding becomes reliable, the incentive for developers of neural interfaces shifts from the race for the largest generative model to the quality of embeddings and the local index. It is no longer only about how much VRAM is needed for inference, but how to manage and update a semantic archive under control. For those evaluating on-premise deployment, AI-RADAR offers analytical frameworks at /llm-onpremise to assess these trade-offs.

The critical point remains interpretability: accurate retrieval does not yet prove that the captured correspondence is causal. But the boundary between neural signal and linguistic reconstruction is now sharper.