A single model to speak, listen, and reason. NVIDIA has released Audex-30B-A3B, a mixture-of-experts (MoE) LLM with 30 billion total parameters that integrates audio capabilities—understanding, speech recognition, translation, text-to-speech, and sound generation—without sacrificing the text performance of its backbone, Nemotron-Cascade-2. The message for those looking at on-premise deployment is clear: multimodal convergence doesn't force you to give up control or data sovereignty, because the entire stack can run on your own hardware.

The choice of MoE architecture is no minor detail. Of the 30 billion parameters, only 3 billion are activated per token. This reduces compute load during inference—an immediate advantage for continuous workloads such as real-time audio processing—but it does not relieve memory pressure: all experts must reside in VRAM to be called upon. At 16-bit weights, that's roughly 60 GB for parameters alone, not counting the context. And here the real self-hosting challenge begins.

Audex-30B-A3B claims a maximum context of 1 million tokens, a window that brings a substantial KV cache. Anyone designing on-premise or air-gapped deployments knows memory cost is non-linear: extending the context multiplies VRAM requirements and can turn a setup designed for fast inference into a machine hungry for bandwidth and capacity. GPUs like the NVIDIA A100 (80 GB) or H100 (80 GB) can certainly host the base model, but as you push the context toward a million tokens, you hit limits that force compromises—aggressive quantization, CPU offloading, or a hit on latency.

Yet the game may be worth the candle. Having one model capable of handling the entire audio pipeline—from speech recognition to semantic analysis, from text-to-speech to sound generation—eliminates the need to orchestrate separate services for each modality. In regulated scenarios, where voice data cannot leave the corporate perimeter, Audex-30B-A3B allows end-to-end processing on-premise, zeroing out the risk of third-party exposure. You no longer need a cloud ASR plus a text LLM plus an external TTS: everything runs on a single track, with a single inference engine and a single maintenance contract.

The ability to operate in two modes—thinking (explicit reasoning enclosed in <think> tags) and instruct (non-thinking)—adds further granularity. In contexts where reasoning traceability is an audit requirement, thinking mode can be selectively enabled; otherwise, a <think></think> token prepended to the assistant's response triggers direct answers, saving tokens and speeding up the flow. It's a control that feels familiar to those already working with text-only models, here naturally extended to audio.

The vocabulary extension with discrete audio tokens and the addition of a dedicated encoder mark a structural evolution: this is not a text-only model hooked to an external converter, but a natively multimodal LLM. This deep integration reduces the chances of degradation when the model switches from purely textual to audio tasks, a common risk in juxtaposed hybrid architectures. NVIDIA mentions marginal or no regression compared to the text backbone, but verification rests with users who will test it against their own real-world workloads.

For those already eyeing local infrastructure as an alternative to cloud services, Audex-30B-A3B is a signal: the market is producing models that no longer force a choice between multimodal power and infrastructure control. The next step, all to be measured, will be real efficiency on consumer and prosumer hardware, because the line between feasible and cost-effective is often drawn by a few gigabytes of VRAM and a few milliseconds of latency.