## DeepSeek Engram: Dedicated Memory for LLMs DeepSeek AI has announced Engram, a new static memory architecture designed for large language models (LLMs). The key idea is to integrate native memory that allows retrieving static information – such as entities, facts, or patterns – without having to recalculate it each time through expensive Transformer layers. Engram introduces a concept of "conditional memory", complementary to the traditional MoE (Mixture of Experts) approach that focuses on conditional computation. This separation between remembering and reasoning allows LLMs to reason more deeply, handle larger contexts, and offload the computational load of the early layers from the GPUs. **Key highlights:** * Knowledge lookup in O(1) instead of recomputation. * Use of explicit parametric memory. * Improved performance in reasoning, math, and code. * Massive memory scaling without GPU limits. * Greater freedom for attention, which can focus on global reasoning rather than static knowledge. In summary, Engram represents a step forward in optimizing LLMs, allowing them to manage knowledge more efficiently and focus on more complex reasoning tasks.