The idea comes from South Korea and upends a logic that has held for decades. According to a Korean scholar cited by DIGITIMES, the memory industry must adopt the foundry model — the same that enabled custom chips in the smartphone and data center era — to break the bottleneck that is strangling AI inference today.

The memory market is dominated by a few vertically integrated giants. Samsung, SK hynix, and Micron design and manufacture DRAM and HBM in-house, with roadmaps driven more by volume logic than by the specific needs of AI workloads. The result is that anyone doing inference — especially on-premise, where every watt and nanosecond counts — ends up fighting a bottleneck familiar to every engineer: memory bandwidth cannot keep pace with GPU compute power.

It’s not a new problem. Large Language Models, ever larger and with extended context windows, require continuous movement of weights and activations between VRAM and compute units. Even with top-tier GPUs, effective inference speed is often limited by memory bandwidth, not by available TFLOPs. In an on-premise scenario, where total cost of ownership (TCO) optimization is crucial, this inefficiency translates into more hardware needed for the same throughput, or into latencies that degrade user experience.

Adopting a foundry model — where independent design houses create memory architectures optimized for specific AI access patterns, relying on third-party manufacturers for fabrication — would change the game. It would open the way to inference-specialized memory: chips with tighter interfaces, reconfigurable cache hierarchies, or memory stacks directly coupled to compute units in chiplet configurations, without waiting for the roadmaps of vertical giants.

Who would benefit? Organizations running self-hosted LLM deployments. They could choose memory modules tailored to the model and desired throughput, instead of settling for off-the-shelf components designed for gaming or generalist cloud workloads. Hardware startups already experimenting with ASIC inference accelerators could integrate more easily with a fragmented but specialized memory ecosystem, reducing time-to-market. Large cloud vendors would still enjoy high volumes, but differentiation would shift to the ability to compose heterogeneous systems.

There’s also a geopolitical angle. A more distributed memory supply chain, with fabrication separated from design, could lessen dependence on a few Asian suppliers — a hot topic for those in Europe who must meet data sovereignty requirements and fear supply chain disruptions. It’s no accident that the European Union is pushing for greater semiconductor autonomy; a memory foundry ecosystem would fit perfectly into that plan.

Of course, the transition is not painless. Memory is not logic processors: margins are thin and economies of scale dominate. But precisely the pressure from AI, with workloads so different from databases and web serving, could create the economic space for specialization. The signal coming from Korea is not an isolated one: it’s the symptom of an industry beginning to recognize that inference hardware is no longer a derivative of training hardware, but deserves dedicated design. And memory, for once, could be the starting point.