Compacting a 74-billion-parameter vision-language model into a memory budget that leaves room for other applications: that is the technical move behind NVIDIA's positioning of Nemotron 3.5 Bolt 74B-A9B 24GB. The Hugging Face listing presents it not as an isolated research experiment, but as a component for everyday agentic tasks executed locally, under the OpenMDW-1.1 license and available for commercial or non-commercial use.

The model combines a hybrid Mamba-2-Transformer architecture, a LatentMoE module, and a C-RADIO v4-H vision encoder. Of the 74B total parameters, 9B are active: a typical configuration for MoE approaches to limit per-token computation, while memory remains tied to the full set of experts. The most relevant choice is IQ2 precision, a native 2-bit quantization obtained with training that accounts for compression from the start. That is where the footprint drops to 24GB, allowing the model to run alongside other applications.

This is not compression for its own sake. NVIDIA lists support for text, image, and video inputs, with text output and a declared context window of up to 256K tokens. The intended deployment runs through vLLM on NVIDIA Ampere (A100), Hopper (H100-80GB), or Blackwell GPUs. Speculative decoding with a LiLiCorr drafter via DFlash and 7 speculative tokens, available in the Bolt recipes, is meant to offset latency that such a compressed model might introduce in reasoning and generation. Reasoning modes are controlled through the chat template with enable_thinking True or False, while recommended sampling keeps temperature 1.0 and top_p 0.95.

The structural point for anyone tracking the evolution of on-premise inference is the shift in priorities: rather than starting from the GPU and looking for a model that fits, the 24GB constraint is set first and the model is built to respect it. This has consequences for TCO and resource allocation: a 74B VLM can remain active next to other workloads without exhausting available VRAM. At the same time, a context of up to 256K tokens is not free during serving: the key-value cache grows with sequence length and needs to be included in capacity planning, despite the 24GB figure declared for the model package. AI-RADAR sees this kind of release as a way to measure the minimum hardware and pipeline threshold for bringing vision-language models inside enterprise perimeters.

The availability of weights, training data, and recipes from the Nemotron family matters just as much. Teams that already run a vLLM pipeline on NVIDIA hardware can evaluate Bolt as a component to integrate into existing systems, without depending on a single inference provider. The message is clear: the threshold for bringing a 74B vision-language model inside a local perimeter has dropped, and competition shifts toward orchestration quality, pipelines, and data control. The listing also reports a post-training data cutoff of May 2026.