The price hike that redraws on-premise budgets

When a foundry like PSMC raises DRAM prices by 45%, the hit lands squarely in the budgets of organisations running AI infrastructure in-house. This is not an end‑of‑quarter tweak; it signals that the demand for Large Language Models is warping the semiconductor supply chain far beyond GPUs. System memory, a seemingly mundane component, becomes the first financial friction point for self-hosted clusters.

In a server dedicated to LLM inference or fine-tuning, DRAM is no accessory: it is the bridge between storage and VRAM, the buffer that sustains data pre‑processing and workload orchestration. Every on-premise node, especially in multi‑accelerator configurations, loads tens or hundreds of gigabytes of system memory. A blunt +45% turns into thousands of euros of extra cost per machine, forcing a revision of already approved budgets or the postponement of expansions.

Those who chose on-premise to keep the Total Cost of Ownership under control over the long term now face a variable that depends neither on compute power nor on energy efficiency. DRAM cost, in a cyclical market that alternates oversupply with speculative peaks, becomes hard to forecast. And this unpredictability is exactly what infrastructure managers fear most when they need to justify multi‑year investments.

PSMC’s announcement comes, moreover, at a moment when the AI rush has absorbed production capacity on all fronts, from GPUs to high‑bandwidth memory. Foundries are reallocating lines and resources, and a 45% hike can be read as an attempt to steer demand toward more advanced nodes where margins are higher. For the end buyer, though, it remains an immediate burden that risks shifting the break‑even point between cloud and on-premise.

Why DRAM is strategic for LLM workloads

When memory for artificial intelligence is discussed, attention almost always focuses on the HBM integrated into GPU packages. But system DRAM plays a role just as critical, especially in on-premise deployments that cannot rely on cloud‑optimised orchestration. During batch inference or distributed fine‑tuning, data must move rapidly from system memory to the compute units, and every bottleneck in this pipeline translates into latency and additional power draw.

For workloads involving large models, VRAM alone is not enough: techniques such as memory offloading shift some weights and intermediate states onto DRAM, lengthening execution times but allowing models that would otherwise not fit in the GPUs. Less DRAM available, or more expensive DRAM, means having to scale back these compromises, pushing toward even more aggressive quantization — with potential quality losses — or toward CPU‑centric architectures that, however, sacrifice latency.

In an on-premise cluster, the amount and speed of DRAM also influence the number of concurrent requests the system can handle. Inference queues grow longer if system memory cannot keep up with the GPUs’ demand. In practice, saving on DRAM to contain unit cost can backfire, because it reduces overall throughput and lowers the return on investment.

The 45% increase is therefore not just a pricing problem but an architectural one. Anyone designing a new cluster today must recalculate memory sizing upward, adding a cost buffer that was previously not considered. And the risk is that, to stay within budget, a key component ends up undersized, creating a bottleneck that nullifies the expenditure on GPUs.

The 3D AI Foundry: promise and stopwatch

On the same day it announced the price hike, PSMC also launched the “3D AI Foundry” division, aiming to bring it to 20% of revenue. This is no coincidence: the company is signalling that revenue from traditional DRAM will finance the leap toward advanced packaging, where logic and memory co‑exist on multiple layers through interposers and 3D stacks.

This direction answers a physical necessity before an economic one. With models exceeding a hundred billion parameters, the power consumed by chip‑to‑chip communication becomes the real limit. Integrating memory and processor in the same package reduces the power spent moving data, cuts latency, and frees thermal headroom to increase frequency or core count. For on-premise inference, this means being able to load larger models without multiplying cards or exceeding power budgets.

The 3D AI Foundry’s stopwatch, however, is not aligned with today’s urgency. Advanced packaging techniques require investment in dedicated fabs and a maturation of production processes that will take at least twelve to eighteen months before yielding significant volumes. Meanwhile, traditional DRAM costs will stay high, and vendors already offering integrated‑memory solutions (such as wafer‑on‑wafer) have limited capacity and premium prices.

For the IT manager, the message is twofold: on the one hand, the prospect of a more efficient architecture in the medium term may justify a pause in massive discrete‑GPU investments; on the other, in the short term there is no escape from higher costs, and every purchasing decision must be weighed with the thought that the hardware will age quickly. The transition window is treacherous, and bad timing can be expensive.

Data sovereignty and the regulatory paradox

PSMC’s move makes a deeper paradox more visible. Data protection regulations, from the European GDPR to sector‑specific rules, are pushing more and more enterprises to keep AI workloads within their own borders, often on self‑hosted hardware. The surge in memory prices, however, makes this choice economically harder exactly when compliance would make it mandatory.

The result is a wedge that risks widening: organisations with large budgets will be able to keep refreshing their on-premise clusters, while others will be forced to consider compromises such as sovereign cloud services or hybrid environments that, however, introduce management complexity and potential compliance risks. In sectors like healthcare or finance, where data sovereignty is non‑negotiable, the increase in hardware costs could translate into a slowdown of AI adoption, with competitive effects.

This scenario forces a rethink of deployment strategies. Self‑hosting is becoming a luxury not all enterprises can afford, at least in the form of “heavy GPUs and abundant DRAM”. The consequence is growing interest in more compact models and compute‑efficient architectures, but also in distributed inference solutions that exploit less powerful but cheaper nodes, such as high‑memory‑density CPU clusters.

The foundry industry, for its part, is riding this tension: on one side selling memory at high prices, on the other promising integrated packaging that will solve the problem. But the solution takes time, and the regulatory wedge does not wait. The risk is that regulation, born to protect citizens, may unintentionally slow innovation in less‑capitalised organisations.

Quantization and offloading: from experiment to necessity

Faced with more expensive DRAM, techniques that until yesterday were considered frontier optimisations become tools of economic survival. Low‑precision quantization (INT4, INT8) reduces the amount of memory needed to host a model, packing more parameters into the same VRAM and relieving pressure on system memory. If once it was a choice to improve latency, now it is a way to contain outlay on DRAM modules.

Similarly, memory‑offloading techniques, which move model layers onto system RAM or even NVMe storage, are once again being considered for production settings. The price of extra DRAM could make it worthwhile to invest in software solutions that intelligently manage weight transfers, even at the cost of lower throughput. Every on-premise team will need to evaluate this trade‑off: slowing inference by 10‑20% may be acceptable if it avoids a 45% memory cost hike.

These strategies are not without risk. Aggressive quantization can degrade answer quality, especially in domains where precision is critical, such as legal analysis or medical diagnostics. Offloading introduces variable latencies that sit poorly with real‑time applications. Yet the alternative — giving up on-premise altogether — could be worse for those with sovereignty constraints. The market for optimisation tools, from llm.c to llama.cpp, is therefore experiencing a second youth, no longer as a niche for tinkerers but as a corporate necessity.

The paradox is that these techniques, born to democratise access to LLMs on consumer hardware, are becoming the main route even for professional deployments. The memory price increase accelerates a process that was already underway, but transforms it from a trend into an obligation, redefining the criteria by which models and architectures are chosen.

What to watch in the coming months

The PSMC signal must be monitored alongside other indicators: the trend of DRAM spot prices, the investment decisions of other foundries (Samsung, SK hynix, Micron) on AI‑dedicated lines, and the evolution of long‑term contracts of the large hyperscalers. If the latter begin to renegotiate upward, it will mean the pressure is set to last.

Another front to observe is the emergence of alternative integrated‑memory solutions, such as chips with non‑HBM stacked‑package memory, which could offer a cost‑performance compromise for mid‑range servers. Some vendors are already proposing architectures that combine CPUs with DRAM soldered onto the motherboard, reducing the space for standard modules but lowering the overall cost. If these solutions gain traction, they could reshape the on-premise hardware landscape.

On the software side, the maturation of frameworks for distributed CPU inference — with the support of optimised libraries for new processors with integrated AI accelerators — could reduce reliance on fast DRAM. Already today it is possible to serve models of 7‑13 billion parameters on CPU‑only clusters, provided they have plenty of memory, and the DRAM price rise could make this path more economically attractive.

Finally, the question to ask is whether the memory price spike will mark a turning point in the cloud‑versus‑on‑premise balance. While the cloud promises to absorb price shocks better, it transfers the operational cost onto a monthly basis, making it less transparent. The PSMC increase may push companies to recalculate more carefully the real TCO of on-premise, including not only the initial hardware but also component volatility. In this equation, memory ceases to be a commodity and becomes a strategic factor.