When DIGITIMES headlines that the generative AI race in China is turning toward open-source ecosystems, it is not a mere statement of principle. It is a snapshot of a deep restructuring, forced by material constraints and geopolitical calculations that are redrawing not only the Chinese tech landscape but the entire map of global AI infrastructure. For AI-RADAR, whose focus is always on on-premise deployment, inference and training hardware, and data sovereignty, this shift acts as a truth accelerator: China is proving that under extreme restrictions, self-hosting LLMs is not a niche alternative but the main path to ensure operational continuity and strategic control. And the lessons are transferable to any organization placing digital sovereignty at the core of its strategy.

Hardware dead ends and the push to on-premise

US export restrictions on high-performance GPUs have created a bottleneck that no software shortcut can entirely bypass. Chinese companies can no longer rely on a guaranteed supply of NVIDIA or AMD accelerators to scale their LLM training. This constraint has triggered a dual movement: on one hand, the search for alternative computing architectures, from domestic chips like Huawei Ascend to solutions from Biren and Moore Threads; on the other, immense pressure on software efficiency. Here open source enters the picture: frameworks like PyTorch (already widely adopted) and open models such as the Llama family or Chinese-derived Qwen become the foundation for building entire stacks optimized for local hardware. It is not just about saving on license fees—access to code and weights allows engineers to dig into bottlenecks and adapt kernels, schedulers, and inference pipelines to silicon that would never have received attention from Western vendors. Quantization, in particular, has become the tactical weapon: bringing models of tens of billions of parameters to GPUs with limited VRAM at INT4 or FP8 is no longer an academic exercise but a production necessity. This optimization effort shifts the competitive center of gravity from “who has the largest model” to “who serves inferences at acceptable cost and latency on constrained hardware.”

The second consequence of the blockade is the sealing-off of the Chinese market from global AI cloud services. When every gigabyte of VRAM counts and network latency to overseas datacenters becomes an untenable luxury, on-premise turns into the only viable architecture for sensitive workloads. It is no accident that Chinese companies are investing heavily in orchestration software capable of aggregating heterogeneous nodes—clusters made up of GPUs from different vendors, even previous generations—to squeeze out every useful compute cycle. In this scenario, the serving framework becomes the real competitive differentiator, not the individual accelerator. And open source provides the common ground for this forced interoperability: without the ability to inspect and modify drivers and runtimes, integrating non-standard silicon would be prohibitive.

Open source as a sovereignty multiplier

China’s embrace of open source has never been a purely technical matter. For Beijing, code verifiability and modifiability are national security requirements. An LLM running on Western cloud infrastructure is an opaque asset: data flows through proprietary stacks, logging mechanisms and training checkpoints escape the end user’s control. Open source flips this dynamic: organizations can inspect every component of the pipeline—from data preprocessing to token sampling—and certify compliance with data residency regulations without delegating trust to third parties. Government-backed platforms like ModelScope act as an enabler: they offer repositories of pre-trained models and fine-tuning tools that foster a collaborative ecosystem, but crucially keep control within national borders. This approach has profound implications for total cost of ownership: the absence of recurring licenses and proprietary lock-in reduces TCO over the long term, especially when operating on hundreds or thousands of inference nodes.

Moreover, the ability to customize the entire stack frees enterprises from vendor-imposed upgrade cycles: they can maintain stable versions for years, integrate workload-specific optimizations, and even patch vulnerabilities without waiting for official updates. In a context of geopolitical tensions, this autonomy is an invaluable asset that turns software from a cost into a store of value. Not surprisingly, some Chinese banks and financial institutions are already migrating to on-premise stacks based on open LLMs, where granular control of the data pipeline is legally binding.

The new hardware ecosystem: silent winners and losers

Looking at the semiconductor supply chain, the open-source turn is redistributing opportunities asymmetrically. The major Western cloud vendors—AWS, Azure, Google Cloud—see their ability to gain share in the Chinese AI market progressively eroding, already hampered by regulatory barriers. US chip companies, already burdened by export controls, witness an acceleration of the decline of their direct presence: not only can they not sell the most powerful GPUs, but the entire software ecosystem is now adapting to expel them from the value chain. In their place, a constellation of local silicon producers—Huawei, Moore Threads, Biren, Enflame—is rising, finding in open frameworks and open-source models a market hungry for hardware compatible with inference workloads. Quantization plays a crucial role here: techniques like INT4 and FP8 enable running LLMs with tens of billions of parameters on domestic GPUs that would otherwise have been considered inadequate. The TCO of an on-premise cluster based on local chips drops, and with it the adoption barrier for mid-sized enterprises.

This is not just about silicon: the industry is learning that software efficiency can, within limits, compensate for hardware shortcomings. Chinese research teams are investing in pruning, distillation, and asynchronous scheduling techniques, turning resource scarcity into a forced-innovation lab. This could yield methodologies and tools that, once mature, will flow into the global ecosystem, lowering inference costs everywhere. Already, some companies are experimenting with models under 10 billion parameters that, thanks to targeted fine-tuning pipelines, achieve competitive performance on modest hardware.

The serving pipeline as the battleground

In the ecosystem taking shape, the race is not to train the single largest LLM, but to build the most efficient fine-tuning and serving pipeline on domestic hardware. This infrastructure layer—encompassing schedulers, load balancers, KV caching systems, speculative decoding techniques, and memory optimizations—is the real bottleneck for on-premise deployment. Frameworks like vLLM, SGLang, and Chinese-optimized variants for Ascend are capturing investor attention. Those who can orchestrate heterogeneous nodes (a mix of GPUs from different vendors, perhaps with varying VRAM capacities) and squeeze every available gigabyte will find themselves in a position of enormous advantage. This is not science fiction: in Chinese datacenters, clusters of older-generation NVIDIA A100s are already being aggregated with new Huawei cards, and the difference is made by the middleware capable of balancing the load without bottlenecks.

This shifts the competitive barometer from training to inference. For most enterprises, training an LLM from scratch is prohibitively expensive; value emerges from reusing open models and adapting them via fine-tuning. Consequently, the ability to serve millions of requests per day on limited hardware becomes the discriminating factor. And it is here that China, thanks to the pressure of restrictions, is accumulating know-how that is hard to replicate in less constrained environments. Western companies that neglect this layer risk finding themselves dependent on serving solutions designed for high-end, expensive, and hard-to-source GPUs.

Beyond China: a testbed for the world

The Chinese experiment has a reach that extends beyond national borders. European, Indian, or Middle Eastern organizations currently evaluating on-premise LLM deployment for data sovereignty or regulatory compliance reasons can observe China as an extreme case study. If self-hosting works under such severe hardware restrictions, it can work even better where access to cutting-edge GPUs is only limited, not blocked. This redraws the incentives for the entire industry: open-source AI software vendors—from Hugging Face to GitHub—see an expansion opportunity in regions that want to emulate China’s control model, but without ties to Beijing’s government ecosystems. At the same time, the rise of a Chinese AI ecosystem based on open standards but deeply intertwined with proprietary local hardware (Ascend chips, for example, use a heterogeneous memory architecture not always compatible with mainstream tools) risks creating a market bifurcation. On one side, we will have a Western stack based on CUDA and NVIDIA GPUs; on the other, a Chinese stack with frameworks adapted to domestic silicon. For global companies operating in both markets, this means maintaining dual skill sets and codebases, increasing complexity and overall TCO.

Signals to watch

To capture the evolution of this transition, several key indicators must be monitored. First, the maturation of orchestration software for heterogeneous clusters: if open-source serving frameworks succeed in fully abstracting hardware diversity, on-premise will become a commodity, further reducing dependence on single suppliers. Second, the TCO trend of deployments based on Chinese chips: if the cost per token inferred on Ascend or Moore Threads falls below a critical threshold, even emerging markets may prefer non-Western solutions for economic reasons. Third, the direction of US export policies: a further tightening could accelerate the flight from CUDA and solidify the parallel ecosystem, while a relaxation might slow but not reverse the trend, given that the accumulated know-how in optimizing for modest hardware is now a strategic asset. Finally, the behavior of large Chinese AI companies like Baidu, Alibaba, and Tencent must be watched. If these entities, while maintaining some cloud services, visibly shift their workloads to on-premise infrastructure based on local chips and open models, the signal will become undeniable: digital sovereignty is not a slogan, but a new operational paradigm that will reshape the geography of artificial intelligence for the next decade.