The collateral effect of the H20: when the “compliant” chip is not enough

The H20 GPU was meant to be Nvidia’s equilibrium point: a card powerful enough to satisfy Chinese demand for training and inference, yet sufficiently throttled to respect Washington’s export restrictions. With its Hopper architecture and reduced bandwidth, along with constricted interconnects, the H20 is the technical compromise that now shows all its cracks. Deliveries lag, commercial priority goes to Western customers, and regulatory uncertainties pile up. The result is that many Chinese data centers and research labs are left without accelerators just when LLM adoption demands ever-growing compute capacity.

The news, reported by AFP in the tone of a trade war bulletin, signals far more than a temporary supply-demand mismatch. It exposes the structural bottleneck of a silicon supply chain subject to geopolitical swings. For those, like AI-RADAR readers, evaluating on-premise stacks for language models, the Chinese case is an accelerated test bench: when the reference hardware becomes inaccessible, technical choice yields to the need for operational continuity and control over data residency. It is no surprise that Chinese buyers are redirecting orders to domestic manufacturers – Biren Technology, Cambricon, Huawei’s HiSilicon division – even accepting a tangible performance gap to avoid freezing their projects.

The paradox is that the very chip conceived to keep the Chinese market open to Nvidia is now acting as an accelerator for China’s technological independence. The message from Beijing is unequivocal: a GPU is not just a component but a strategic asset. And the H20, in its “castrated” form, projects the image of a supplier that can guarantee neither lead times nor volumes, eroding the trust of the most demanding clients. Thus, the search for local alternatives becomes a rational, not ideological, choice, opening the door to a hardware ecosystem that until yesterday was considered niche.

The software knot: why CUDA remains the real barrier

If the shift to Chinese GPUs is driven by urgency, the real cost is paid on the software integration front. Nvidia’s supremacy is not only measured in TFLOPS: it rests on CUDA, a framework so pervasive that it has become the lingua franca of AI development. Migrating an LLM to non-Nvidia hardware means dealing with a development ecosystem that has not yet reached the maturity of tools, libraries and established practices. It is not simply a matter of recompiling a few modules; often entire pre-processing pipelines must be rewritten, compute kernels adapted, and optimized libraries for quantization or distributed fine-tuning – available on CUDA with a single line of code – must be abandoned.

For teams working in on-premise environments, the software gap translates into greater maintenance complexity and longer time-to-market. Chinese companies must account for weeks or months of extra work to bring a model trained on Biren or Cambricon cards to the same efficiency level they would have obtained on an A100 or H100 cluster. This effort directly impacts TCO: while the purchase price of local cards may be competitive, the cost of specialized personnel and accumulated integration delays risks eroding the initial economic advantage. In many cases, the choice to stick with CUDA has been dictated precisely by the desire to avoid these hidden costs.

Yet the Chinese situation shows that when physical silicon availability falls short, the equation changes radically. An LLM running on less performant but accessible hardware is preferable to a project frozen while waiting for uncertain deliveries. The real lesson, for anyone managing on-premise infrastructure, is that dependency on a single software vendor proves to be a rigidity factor just as critical as dependence on silicon. And while urgency in China is pushing manufacturers to invest in enhancing their own SDKs – aiming to narrow the gap with CUDA – in the rest of the world the discussion on hardware abstraction and on frameworks such as Triton or ONNX Runtime takes on unprecedented strategic relevance.

Data sovereignty and on-premise computing: the trade-off reshaping architectures

In China, the push toward domestic GPUs is not solely a child of the H20 shortage. Data residency regulations, such as the Personal Information Protection Law and the Data Security Law, impose stringent constraints on those handling sensitive information: data must remain within national borders and providers must pass severe security audits. In this context, adopting hardware from a foreign supplier subject to external controls represents a permanent compliance risk. Companies prefer to sacrifice a few percentage points of inference throughput to have certainty that the entire stack is managed by local actors and that data never traverses components exposed to sudden suspensions.

For those designing on-premise deployments in other geopolitical quadrants, the Chinese case is emblematic. Data sovereignty, traditionally framed as the choice not to entrust models to cloud services run by big tech, now extends to hardware selection. A growing number of organizations – government agencies, financial institutions, healthcare facilities – are beginning to evaluate not only where data resides but on which silicon it is processed. The chip’s provenance becomes an audit factor: a board whose manufacturer is subject to embargoes or to foreign government interference is perceived as a weak link in the trust chain.

The resulting trade-off is stark. On one side, Nvidia hardware offers an almost linear path to deploy quantized LLMs or to perform distributed fine-tuning, with a battle-tested tooling ecosystem and a global developer community. On the other, local chipmakers – in China as in other regions investing in manufacturing capacity – allow building self-hosted environments entirely under one’s control, but impose an integration burden that may require rare specialized skills and lengthens the return-on-investment horizon. The balance, in the Chinese case, has shifted drastically toward control because the alternative – standing still – is deemed unacceptable.

A market that splits: global repercussions for inference and training

The H20 short-circuit is not an isolated phenomenon: it is a symptom of a structural fragmentation of the AI accelerator market. Nvidia finds itself having to manage Western demand that absorbs every unit produced, while progressively losing traction in one of the most receptive markets on the planet. Chinese chipmakers, for their part, can exploit this window to fund the maturation of their own platforms, pushing innovation on fronts such as advanced packaging, systolic array architectures, and interconnects that do not rely on technologies subject to US control.

This bifurcation will have second-order effects far beyond China’s borders. A parallel hardware ecosystem, even if still inferior in absolute performance terms, will start developing toolchains, optimization frameworks, and libraries that, in the medium term, may compete not so much on peak performance but on the developer experience. It is a process already observed in other sectors: necessity sharpens ingenuity, and the availability of Chinese capital – combined with a first-class engineering workforce – can close the software gap faster than many analysts anticipate.

For AI professionals working on on-premise stacks, this polarization means preparing for a multi-vendor hardware scenario that until a few years ago seemed like science fiction. Compatibility with a single ecosystem will become a luxury; the ability to orchestrate inference and training on heterogeneous silicon – perhaps aided by abstraction layers such as OpenCL, Vulkan Compute or open compilers – will be a distinguishing competency. Already today, several software vendors for LLM deployment are investing in modular backends that allow changing accelerators without rewriting the entire application logic. The H20 crisis will give further impetus to these solutions.

Lessons for on-premise deployment: diversifying hardware as a resilience strategy

The Chinese experience with the H20 offers a clear lesson to anyone designing long-lived on-premise AI infrastructures. Relying on a single silicon vendor – however excellent – means tying one’s innovation cycle to decisions that may be dictated by geopolitical dynamics wholly unrelated to business needs. Hardware diversification is no longer a theoretical exercise: it is an insurance against exogenous bottlenecks. This applies to European banks wanting to do in-house fine-tuning, to telcos pushing AI to the edge, to healthcare institutions handling sensitive clinical data.

Diversifying, however, does not mean simply purchasing a few cards from an alternative vendor and waiting for the IT department to make everything work. It requires a conscious strategy starting with the selection of frameworks and model formats. The most forward-looking teams are already investing in intermediate representations like ONNX and in execution environments such as Apache TVM or IREE, which allow compiling a model once and deploying it to different hardware targets with modest adaptation overhead. The initial investment in abstraction pays dividends when, as happened in China, the reference chip disappears from the price lists.

At the same time, diversification forces an update of the metrics used to evaluate TCO. It is no longer enough to compare the cost per token produced in inference or the training time of a given model. One must include the cost of an eventual rewrite of the entire software stack should migration from one ecosystem to another become necessary, the cost of finding specialized skills on less common architectures, and the opportunity cost of months of additional development. For the AI-RADAR reader, accustomed to weighing trade-offs between self-hosted and cloud services, the lesson is consistent: control has a price, and that price must be well calculated before it becomes a necessity.

What to watch going forward: signs of an evolving ecosystem

The Chinese case will be a laboratory to observe closely. The coming months will tell whether local manufacturers manage to close the software gap with Nvidia, not only by publishing throughput benchmarks but by offering a development experience that does not force a complete codebase rewrite. Biren has already released its own CUDA-inspired compilation stack, while Cambricon is betting on a more vertical approach with libraries optimized for its own silicon. Huawei, with the Ascend line, is investing massively in the MindSpore framework. The question is not whether these tools work – they do – but how quickly they can reach the maturity needed to handle enterprise-scale LLMs without hiccups.

Globally, a signal to monitor is the evolution of hardware abstraction projects. Initiatives such as Triton, which allows writing optimized kernels once and compiling them for multiple backends, or OpenXLA, which aims to unify compilation for more accelerators, could become the true enablers of a fragmented market. If these intermediate layers reach critical mass, the CUDA lock-in will shrink, and hardware diversification will become not only possible but feasible with manageable effort. For infrastructure managers, keeping an eye on these developments is a low-cost, high-potential investment.

Finally, one must watch Nvidia’s response. The company will not stand by while the market fragments. It could push for software licenses that decouple CUDA from hardware – a move that today seems remote but in a fragmentation scenario could become defensible – or accelerate the development of solutions that facilitate coexistence with other ecosystems. Other Western players, from AMD to Intel, also have an opportunity to position themselves as credible alternatives in a scenario that rewards redundancy. The only certainty, at present, is that the H20 crisis has shaken a market once thought to be monolithic: anyone designing on-premise LLM environments today would do well to read this upheaval not as a temporary accident, but as the first chapter of a structural transformation.