AMD is enhancing its Ryzen AI NPUs for running large language models (LLMs) on Linux, thanks to the Lemonade 10.0 server and FastFlowLM 0.9.35. A key component of this strategy is MLIR-AIE, a compiler toolchain developed internally.

MLIR-AIE v1.3: What's New

The v1.3 release of MLIR-AIE introduces new features to improve performance on AMD AI Engines. MLIR-AIE leverages Multi-Level Intermediate Representation (MLIR) and LLVM-based code generation to optimize the performance of artificial intelligence workloads.

For those evaluating on-premise deployments, there are trade-offs to consider. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these aspects.