When hardware outpaces software

Those who bought a laptop or mini PC with an AMD Ryzen AI Max+ 395 – part of the Strix Halo series – a year ago knew they had cutting-edge silicon: an APU with integrated graphics and a dedicated NPU for AI. Too bad the NPU was more of an ornament than a usable asset until recently. Support through ROCm, AMD’s accelerated computing framework, was virtually non-existent for LLM workloads. Now the tune has changed: developer community efforts and tools like Lemonade have opened the door to hybrid inference, finally leveraging iGPU and NPU together.

The news, surfacing in a Reddit post, is more than a tinkerer’s delight: it signals the maturation of a hardware platform that promised to lower the barrier to running large language models locally. For those eyeing on-premise deployment with a focus on data sovereignty and total cost of ownership, this is a strong hint.

Hybrid mode: two engines for inference

Technically, the Strix Halo NPU is designed to be extremely fast at prompt processing – the initial model input phase. Meanwhile, the integrated GPU can handle subsequent token generation. Hybrid mode, orchestrated by Lemonade – a bare-bones but effective GUI developed in coordination with AMD – splits the workload between the two accelerators. In practice, the NPU chews through the initial processing while the iGPU runs decoding in parallel, reducing perceived latency.

Models optimized for NPU alone already exist, like FastFlowLM NPU, but the real breakthrough is the combination: not an alternative, but a mutual boost. According to AMD documentation, building hybrid models requires specific conversions, and the path isn’t yet smooth for all formats. GGUF models, for instance, can’t simply be turned into ONNX; a dedicated adaptation process – detailed in AMD guides – is needed.

Implications for on-premise deployment: control, cost, sovereignty

For environments evaluating on-premise solutions, a working NPU on x86 platforms is a significant piece of the puzzle. It’s not just about performance: running LLMs on consumer or prosumer hardware, without depending on remote GPU servers, means keeping full control over data. In regulated sectors or where privacy is paramount, that’s a non-negligible competitive advantage.

Moreover, the TCO of a Strix Halo APU, compared to a high-end discrete GPU, is substantially lower, and the NPU’s energy efficiency in prompt processing can reduce overall consumption. True, the software stack is still maturing – ROCm on consumer hardware has had a bumpy road – but the improvement seen in just a few months bodes well. For those already running models via GGUF and Vulkan, switching to hybrid mode could multiply performance without additional investment.

The future: multi-token models and the road ahead

The original post’s enthusiasm goes further: the author explicitly asks for support for Multi-Token Prediction (MTP) models like Qwen 3.6, which promise a further leap thanks to Unsloth-introduced techniques. AMD has already published guidelines for adapting these “new processor shapes” to ONNX conversion, but the climb remains steep.

One fact stands: a computer bought a year ago now runs modes that were only theoretical back then. It’s proof that a hardware platform’s value doesn’t stop at purchase day but grows with software evolution. For AI-RADAR, which closely follows on-premise deployment choices, this confirms that investing in forward-looking silicon can pay off – provided you have the patience to wait for tools to catch up. The next challenge will be standardizing hybrid models and sharing them on platforms like HuggingFace, turning a hacker success into an industrial lever.