Unmasking Experts: How MoE Models Expose and Mitigate Hallucinations Without Retraining
A new technique exploits the Mixture of Experts architecture to identify and correct hallucinations in LLMs at inference time, without altering the model. The study reveals that in higher MoE layers, different expert groups activate in opposite ways ...