New silicon from Nvidia, Qualcomm, Apple and AMD aims to move LLM inference from the cloud to local devices and on-prem servers, with growing attention to memory bandwidth, integrated GPUs and personal AI computers.
AMD introduces PerfOpt in Linux 7.4 to improve AI/LLM performance on integrated Radeon graphics. Benchmarks show 18-23% gains on Ryzen hardware, especially entry-level. A signal for local inference: iGPUs become a more credible option for lightweight...
AMD's new 256-core EPYC delivers memory bandwidth at 91% of an RTX 5090 through a 16-channel DDR5-12800 controller. The LocalLLaMA community responds with sarcasm and realism about the cost of high-density RDIMMs and the question of how many tokens p...
Nvidia's push on personal AI computers signals a shift in the center of gravity: LLM inference moves closer to the local device. It is not just consumer hardware, but a miniature on-premise node touching VRAM, quantization, TCO and data sovereignty. ...
Nvidia's push for personal AI computers, reported by DIGITIMES, signals a structural shift: inference is moving from cloud to local machines. For enterprises, this opens scenarios around data sovereignty and TCO, but also VRAM and quantization constr...
Qualcomm has unveiled two system-on-chips with GPU matrix cores and LPDDR6 support. This is not about benchmark scores: it signals that data sovereignty is moving onto the device. AI-Radar's analysis of memory constraints, quantization, and TCO for l...
Qualcomm has introduced the Snapdragon 8 Elite Gen 6 and Elite Extreme Gen 6 with Oryon CPU cores, GPU matrix cores and LPDDR6 support. The signal is stronger local inference: AI workloads move to the device. Memory constraints and quantization remai...
Nvidia unveils RTX Mega Geometry 2.0, which streams ray-tracing geometry into VRAM only when needed. The approach reduces memory pressure and points to a broader shift in how local workloads use GPU memory, with implications for teams evaluating on-p...
A user tested a Mac Studio with M5 Ultra, 256 GB of RAM and an 80-core GPU for agentic inference with GLM-5.3-Flash. They are satisfied overall, but report the GPU feels underpowered relative to memory, raising doubts about a 512 GB model. The case h...
The Mac M5 Ultra with 256GB is on the market but serious benchmarks are missing: only influencers testing 8b models. This information gap hurts those evaluating Apple hardware for self-hosted LLMs, effectively favoring the Nvidia ecosystem. Without r...
Intel shifts focus to agentic AI: CPU-GPU coordination becomes the centerpiece. The move signals a rethinking of hardware for workloads that do not end with a single inference but chain reasoning, tool calling, and memory. Direct implications for tho...
The new Apple A20 Pro SoC doubles the Neural Engine from 16 to 32 cores and, thanks to a 96-bit LPDDR5X bus on a 2nm process, reaches about 115 GB/s of memory bandwidth. A costly investment that signals Apple's strategic priority for local model exec...
System76 has introduced the Thelio Mira AI, a desktop designed for multi-GPU AI training, fine-tuning, inference, computer vision, and image generation. The system pairs AMD Ryzen 9000 CPUs with NVIDIA or AMD GPUs and can also handle simulations and ...
Huawei has put LogicFolding to the test on the Kirin 9050 Pro: according to the source, the density gain reaches 55% and NPU power consumption drops by 66%. The data points to a clear direction for local inference: denser chips and more efficient NPU...
A builder spent €18,000 on an EPYC server with 768 GB of VRAM and 256 GB of RAM to run frontier open source LLMs locally. But upcoming models such as GLM 6 could be two to three times larger, making the machine unusable at 4-bit quantization. The cas...
Acer's decision to bet on RTX and GoogleBook signals a hardware shift toward local inference. On-site AI demand grows for sovereignty, latency, and cost control reasons, pushing vendors to invest in GPUs and devices for self-hosted workloads.
A new TTS stack with 294k parameters occupies 337 KB in INT8 and runs on a $3 microcontroller with 512 KB of SRAM and no NPU. It supports 11 voices and 6 languages; the 1.5M-parameter model beats larger networks on SCOREQ. A signal for local voice AI...
