AI inference is shifting from cloud to local machines, driven by Nvidia personal AI computers and AMD platforms with high memory bandwidth. Mini PCs and workstations now run large MoE models entirely on-device, reducing cloud dependence.
The Kyojin project runs two roughly 300-billion-parameter MoE models on a single 128 GB mini PC with AMD Strix Halo. Measured on a Ryzen AI Max+ 395, GLM-5.3-Flash and MiMo-V2.6-Flash reach up to 650 tokens/s prefill and 26 to 44 tokens/s decode. EXL...
The Ryzen AI Max 400 'Gorgon Halo' series is nearing market, but Linux support for its NPU has lagged. A simple enablement patch is expected to land in kernel 7.4. For those evaluating local and self-hosted inference, the delay signals a structural i...
AMD introduces PerfOpt in Linux 7.4 to improve AI/LLM performance on integrated Radeon graphics. Benchmarks show 18-23% gains on Ryzen hardware, especially entry-level. A signal for local inference: iGPUs become a more credible option for lightweight...
AMD's new 256-core EPYC delivers memory bandwidth at 91% of an RTX 5090 through a 16-channel DDR5-12800 controller. The LocalLLaMA community responds with sarcasm and realism about the cost of high-density RDIMMs and the question of how many tokens p...
Nvidia's push on personal AI computers signals a shift in the center of gravity: LLM inference moves closer to the local device. It is not just consumer hardware, but a miniature on-premise node touching VRAM, quantization, TCO and data sovereignty. ...
Nvidia's push for personal AI computers, reported by DIGITIMES, signals a structural shift: inference is moving from cloud to local machines. For enterprises, this opens scenarios around data sovereignty and TCO, but also VRAM and quantization constr...
A user fed Opus 5.5 motion graphics examples to Qwen 27B and produced a local version on a single RTX 4090. The project was built with the Accuretta framework and shows that self-hosted creative pipelines are starting to cover tasks once reserved for...
