Topic / Trend Rising

Self-Hosted and Edge AI Acceleration

Consumer GPUs, NPUs and Linux drivers are making local and homelab inference increasingly viable. AMD, Intel and the llama.cpp ecosystem are lowering the barriers for on-premise model serving.

Detected: 2026-11-22 ยท Updated: 2026-11-22

Related Coverage

2026-11-21 โ€ข Phoronix

Intel Linux NPU Driver 1.39.1 Released with Nova Lake Support

Intel has released version 1.39.1 of its Linux NPU driver, adding official Nova Lake support. The user-space driver completes work already merged into the kernel through the Intel IVPU driver, which has included patches for the sixth-generation NPU f...

#Hardware #LLM On-Premise #DevOps
2026-11-21 โ€ข Tom's Hardware

Two RTX 3090s and second-gen Optane run a 2.8-trillion-parameter Kimi AI model

A homelabber paired two GeForce RTX 3090s with second-generation Intel Optane storage to run a 2.8-trillion-parameter Kimi AI model at 3.11 tokens per second. The experiment highlights how memory-tiered inference is reshaping self-hosted LLM deployme...

#Hardware #LLM On-Premise #Fine-Tuning
2026-11-19 โ€ข Phoronix

Unsloth Desktop now uses AMD Ryzen AI NPUs, including on Linux

AMD and Unsloth have announced local Inference support for Ryzen AI NPUs in Unsloth Desktop, starting with XDNA 2 and including Linux compatibility. The move reduces dependence on discrete GPUs and cloud services for lighter workloads, with potential...

#Hardware #LLM On-Premise #Fine-Tuning
2026-11-17 โ€ข LocalLLaMA

Intel D270: 5.9x FP64 throughput and lower AI inference latency

Intel has shared the first benchmarks for the next-generation Crescent Island D270 GPU. It claims 5.9x FP64 throughput over the RTX PRO 6000, and in AI benchmarks reports 29% lower latency and 1.12x throughput against the same NVIDIA part. Preliminar...

#Hardware #LLM On-Premise #DevOps
2026-11-17 โ€ข Phoronix

Intel D270 and open source: a signal for self-hosted AI

At Supercomputing 2026, Intel's D270 GPU announcement offered no specifications but emphasized open source software. AI-Radar analyzes implications for self-hosted deployments, TCO, and data sovereignty: performance to verify, drivers and runtimes to...

2026-11-16 โ€ข Phoronix

Intel pushes the D270 GPU with performance and open-source software support

During Supercomputing 2026 week, Intel spotlighted its D Series 'Crescent Island' GPUs based on Xe3P, emphasizing performance and open-source software support. A signal for teams evaluating self-hosted hardware, though technical details remain to be ...

#Hardware #LLM On-Premise #Fine-Tuning
2026-11-15 โ€ข LocalLLaMA

Llama.cpp catches up: local benchmarks on RTX 5080 show a turning point

A hobbyist compared llama.cpp over time with his own inference engine on an RTX 5080 16GB. Within a few months llama.cpp went from being slower to surpassing the custom alternative, while keeping efficient VRAM usage. The story signals a maturation o...

#Hardware #LLM On-Premise #DevOps
โ† Back to All Topics