## LLM: Large Context, Problems Ahead? The expansion of the context window in Large Language Models (LLMs) has become a priority, with the aim of improving complex reasoning and analysis of extended documents. However, this growth leads to a significant increase in computational load. A recent study examined the delicate balance between system performance and model quality when dense transformer architectures, such as Llama-3.1-70B and Qwen1.5-14B, are subjected to large amounts of irrelevant and distracting context. The research identified a non-linear performance degradation, directly linked to the increase in the Key-Value (KV) cache. Furthermore, the in-depth analysis of the Mixture-of-Experts (MoE) architecture revealed unique behavioral anomalies at different context scales. This suggests that architectural benefits may be masked by infrastructure bottlenecks when handling high volumes of tokens. In summary, increasing the context window is useful, but requires careful optimization to avoid penalties in terms of performance and accuracy.