📁 Hardware

The ORL (On-Policy Residual Learning) project aims to speed up diffusion LLMs without sacrificing quality. On a single RTX 5090 the team reports 2,788 tokens/s with LLaDA-2.1-mini in NVFP4; the open-source dlmrt inference engine uses CUDA graphs and supports Blackwell GPUs. Peak single-request numbers, but the shift matters for self-hosted deployment evaluations.

At Open Source Summit Europe, the focus was on GPUs and AI, but the most concrete part came from CRA compliance plans, OSPO metrics, and migration targets such as Berlin's 70 percent open source by 2032. Bureaucracy is becoming infrastructure.

Bolt Graphics is seeking $100 million to develop Zeus, a GPU aimed at competing with Nvidia and AMD beyond AI workloads. The source includes no technical specifications, but the move signals an attempt to build a third hardware pole. Success will depend less on silicon than on software ecosystem, TCO, and the ability to persuade self-hosted pipeline operators.

A real-world test shows affinity handles four Radeon AI PRO R9700 cards without degenerate output. The 2.875 bpw quant keeps 11,008 experts in VRAM, reaching 79 tok/s greedy decoding with no host pool allocated. The earlier bug was an artifact of simulating four ranks on two cards, not a flaw in multi-rank execution.

A change in the GNOME 52 compositor reduces unnecessary buffer copies between GPUs on hybrid laptops. Less wasted bandwidth and battery, with an indirect benefit for users running local inference on the discrete GPU.

A 24-billion-parameter LLM tested on an AMD W7900 with a 128,000-token context and sustained speed around 95 tokens per second signals that on-premise inference is maturing beyond NVIDIA's ecosystem. AI-Radar explores trade-offs, data sovereignty, and TCO.

Counterfeit RTX 4090s that are almost indistinguishable from genuine parts during visual inspection and initial software tests are turning consumer GPU purchases for on-premise LLM inference into a structural risk. The damage goes beyond replacement costs: it includes instability under sustained load, unreliable benchmarks and a higher TCO. For self-hosted and data-sovereign deployments, hardware provenance becomes an infrastructure skill, not just a logistics detail.

Qualcomm has unveiled two system-on-chips with GPU matrix cores and LPDDR6 support. This is not about benchmark scores: it signals that data sovereignty is moving onto the device. AI-Radar's analysis of memory constraints, quantization, and TCO for local LLM deployments.

AMD's initial work on GDDR7 support in Linux GPU drivers signals a shift for teams running on-premise AI infrastructure. The seemingly technical news highlights the importance of memory bandwidth for LLM inference and lowers integration friction for self-hosted environments. It's a signal to read through the lens of TCO and data sovereignty, rather than a product announcement.

Enflame's listing closes the IPO cycle of China's four main GPU producers. The financial signal is clear: Beijing treats AI chips as a strategic priority. But capital alone cannot close the gap in software ecosystem, VRAM, bandwidth, and compatibility with fine-tuning and inference pipelines. For self-hosted and on-premise infrastructure, the real contest is about drivers, tools, and operational maturity.

Production of HyperAccel on Samsung's 4nm node signals dedicated LLM accelerators moving from experimentation to industrial supply logic. Tight capacity adds pressure on deliveries and wafer allocation. For self-hosted deployments, ASICs promise lower TCO on stable workloads but constrain hardware to less flexible models and quantization schemes. The trade-off must be assessed beyond purchase cost.

Cambricon joining the PyTorch Foundation as a Platinum member signals a shift: controlling the open source software stack matters as much as chip performance. For teams evaluating self-hosted LLMs, native support for diverse backends reduces CUDA lock-in, maintenance burden, and TCO. The real test remains the continuity and maturity of its upstream contributions.

A Reddit setup shows a 27B dense model at 47-50 tokens/s with a 100,000-token context on a 16 GB RTX 4070 Ti SUPER. The signal is not the model but fine-grained runtime control: asymmetric kvarn5/kvarn4 KV cache, a 1,024-token full-precision tail, and speculative decoding. It shifts the constraint from VRAM to serving software, with implications for TCO and data sovereignty.

The HP Z4 G6i workstation combines Xeon 600 series, RTX graphics and declared Linux support, but the top configuration exceeds $41,000. The signal for AI-Radar is not raw power: it is the legitimization of workstations as on-premise nodes for LLMs, local inference and fine-tuning on sensitive data. GPU, VRAM and operational management still need verification, variables that are decisive for assessing TCO and real-world use.

An unverified report describes a nearly twenty-hour agentic coding session with Qwen3.8-27B Q6 on an RTX 3090 and an RTX 3060, sustaining 60–63 token/s. If replicated, the data shifts the cloud versus self-hosted boundary. We analyze trade-offs, TCO and data sovereignty in the local pipeline.

The doubling of the RTX PRO 6000 Blackwell list price, from under $8,000 to $16,000, signals inelastic enterprise demand and pricing power that reshapes TCO calculations for on-premise. The 96GB VRAM card becomes a filter: cloud, data sovereignty, and developer access all shift. Teams need scenarios, not budgets built on old price lists.

A handful of code lines in llama.cpp multiplies Intel Battlemage GPU performance up to 2.7 times on 118K-token contexts with quantized KV cache. The gain is so sharp that it turns a gaming card into a tangible alternative for local inference of 35B models, slashing total cost of ownership and opening unprecedented scenarios for those handling sensitive data on-premise.

PSMC’s 45% DRAM price hike is not a one-off event but a structural signal of AI-driven memory pressure. For on-premise LLM deployments, it directly hits TCO, while the chipmaker’s new 3D AI Foundry points to integrated solutions further down the road. A look at immediate fallout and strategies to absorb the shock.

A three-line patch for llama.cpp fixes a precision error that silently degraded Tesla P100 GPUs for years, improving calculation fidelity by over 2000x with no performance hit. This discovery reshapes the cost-effectiveness of these $80 cards for on-premise inference, raises questions about low-level arithmetic verification in open-source frameworks, and reinforces the link between stack control and model quality.

**The Pocket-Sized Behemoth: A Review of the GMKtec Evo X2**