The latest move to erode Nvidia’s AI dominance doesn’t come from new silicon but from a software runtime. Paris startup ZML has released a free tool that aims to run language models and other inference workloads fast across silicon from different vendors: Nvidia, AMD, Google (TPU), Apple Silicon, and Intel. If the promise holds, teams deploying open-source models can finally mix hardware without rewriting code for each platform.

Nvidia’s grip isn’t just about GPUs. The real moat is CUDA, the software stack that ties developers, libraries, and optimizations to a single ecosystem. Breaking that bond has been attempted repeatedly with open standards like OpenCL or SYCL, but CUDA’s convenience and performance have always prevailed. ZML enters this fray with an approach that, by announcement, stresses ultimate simplicity: a single runtime, ready-made models, and cross-vendor acceleration without configuration compromises.

For anyone evaluating on-premise deployments or environments with strict data sovereignty requirements, such a tool reshapes the equation. Organizations can buy servers with GPUs from different suppliers – AMD for some workloads, Nvidia for others, Apple Silicon nodes for low-power inference – without juggling separate codebases. This cuts vendor lock-in risk and lowers total cost of ownership (TCO) by allowing hardware diversification based on availability and price. AI-RADAR has explored on-premise trade-offs in detail; you’ll find an analytical framework at /llm-onpremise.

Of course, the word ‘fast’ in these announcements warrants caution. Nvidia’s proprietary optimizations (TensorRT, cuDNN) are hard to match, especially for complex models and high throughput. But for a vast swath of use cases – medium-sized model inference, prototyping, edge computing – a runtime that delivers ‘good enough’ performance on any hardware could be a game-changer. The tool’s free price tag further lowers adoption barriers.

ZML’s launch signals something deeper: AI hardware market fragmentation is creating demand for software abstraction reminiscent of the Java Virtual Machine era. Just as the JVM made code portable across operating systems, portable runtimes could make AI models independent of the underlying hardware. If the industry moves in this direction, value would shift from platforms to models and data, forcing Nvidia to compete not on ecosystem control but on raw silicon performance. ZML is still a startup, and the road to challenging a trillion-dollar giant is long. But every crack in the software monopoly is one more crack for those who need flexible and truly sovereign AI infrastructure.