A single H100 GPU that yesterday could handle only one request with a million-token context now sustains eight simultaneously. That's not a marketing trick, but the raw gain delivered by the Nemotron-Labs-3-Puzzle-75B-A9B model, released by NVIDIA on Hugging Face and ready for commercial use.

The Nemotron Labs team took the hulking Nemotron-3-Super (120.7B total parameters, 12.8B active) and compressed it using a post-training framework called Iterative Puzzle, shrinking it to 75.3B total and 9.3B active parameters. This is no simple pruning: the model uses a hybrid MoE architecture with interleaved Mamba, Mixture of Experts, and Attention layers, and supports Multi-Token Prediction to speed up text generation. The result is roughly doubled server throughput on an 8×B200 node at matched user-throughput constraints, and a capacity to handle long-context workloads that changes the calculus for reasoning-heavy, agentic, and multilingual tasks.

Beneath the numbers lies a sharp strategic posture. NVIDIA is not just shipping a lighter model; it's proving that carefully orchestrated post-training compression can become an efficiency multiplier for on-premise deployment. Fewer active parameters mean less VRAM per request, more room for the KV cache, and the ability to pack more users onto the same metal, cutting per-query costs without sacrificing quality. Internal benchmarks, detailed in the tech report, confirm solid accuracy remains on reasoning, coding, long-context, and agentic scenarios.

The bottom line for self-hosted setups

For enterprises evaluating self-hosting, the TCO of an LLM largely depends on the number of GPUs needed to absorb peak load. A leap from one to eight concurrent million-token requests on a single H100 can halve or more the inference cost, making economical what previously required a cluster. Instead of adding nodes, you exploit VRAM you already have, lowering CapEx, energy consumption, and operational complexity. Moreover, support for European languages – Italian included – and immediate commercial use remove barriers for companies operating in regulated markets or with sensitive data, where control stays in-house.

Another underrated aspect: the Iterative Puzzle framework is not public nor described as replicable by other vendors. NVIDIA trains and compresses for its own hardware ecosystem, creating a virtuous (or vicious, depending on your perspective) cycle that ties model choice to GPU selection. Buyers of B200 or H100 who run these optimized models see throughput competitors struggle to match with generic solutions. That's a powerful incentive for those already invested in NVIDIA infrastructure, and a signal to anyone evaluating alternatives: the difference won't come just from silicon, but from the symbiosis between chips and purpose-compressed models.

The million-token context, finally, is not a stunt. Real workloads – analyzing entire codebases, reviewing legal documentation, agents combing through enterprise logs – demand ever-larger attention windows. That Puzzle-75B-A9B can handle eight such streams in parallel on hardware already common in datacenters pushes the frontier of what's feasible without reaching for next-gen specialized GPUs. For the AI infrastructure lead, the question becomes: how many workloads can I consolidate on one node before scaling out? With this model, the answer is significantly higher than yesterday.