Compact models, on-device agents, and community GPU unlocks are accelerating local AI adoption, while VRAM, RAM, and supply constraints reshape hardware choices.
An open-source USB tool removes the software limit that cuts AI performance of Nvidia GPUs on Windows to half speed. The news reopens the debate on artificial segmentation of consumer hardware and on the choices of those evaluating on-prem deployment...
A Mac M4 Max test with a Qwen3.8 27B model and 164k tokens of context shows cache and KV can cost as much as weights. The old 64GB rule no longer holds.
Microsoft shows that a 4-billion-parameter vision-language model can hit 66% on OSWorld-Verified, using 13% fewer model calls and 25.4% fewer output tokens after online RL. The secret is a five-stage training pipeline and workflow-first synthetic env...
A detail in a pull request reignites the debate on a class of 120-billion-parameter LLMs. For teams managing self-hosted deployments, the size reshapes calculations around VRAM, quantization, and TCO, without official announcements. The signal points...
llama.cpp v0.8.0 adds compatibility with DFlash2 for Qwen3.8-Flash-Next. Tests on Apple Silicon M5 Max with quantized GGUFs show an average 17% decoding throughput gain over MTP, with peaks of 39% on list-like prompts and a minor prose regression. Th...
A pull request in the QwenLM/qwen-code repository has drawn attention to a possible 120-billion-parameter model. It is not an official announcement, but the 120B class touches a central issue for on-premise deployments: aggregated VRAM, quantization,...
The pull request in qwen-code signals that Qwen4 targets a one-million-token window. For self-hosted inference the cost is not theoretical: the key-value cache grows with tokens and fills VRAM along with weights. Qwen4-27B makes multi-GPU setups or a...
Alibaba brings Qwen4 to a one-million-token context window. The news emerges from a pull request in qwen-code: Qwen4-27B is expected tomorrow at the Apsara event in the Netherlands, while Qwen4-max is dated November 1st with October 1st as a likely p...
An IT vendor has listed a used AI supercomputer on eBay for $60 million: 9,600 Nvidia A100 GPUs, 4,200 AMD EPYC CPUs and 300 decommissioned racks, with free shipping. The listing highlights how AI hardware life cycles are feeding a secondary market r...
Liquid AI has introduced LFM3, a family of compact models from 1.7–2.2 billion parameters with a tri-hybrid architecture and llama.cpp support. The decision models skip token generation and return calibrated answers in a single forward pass. For on-p...
An open System One model runs locally on a laptop and reaches results close to GPT-4 on medical benchmarks, with well-controlled calibration on knowledge tests but less reliable patient-level predictions. A two-parameter recalibration method using 50...
Demand for local AI infrastructure is growing as cloud costs weigh on budgets. Supply for Asus RTX Spark is tightening, signaling a market shifting resources toward on-premises computing. The trend highlights pressure on hardware, lead times, and dep...
Liquid AI has released a family of models ranging from 1.7 to 2.2 billion parameters, designed to run on phones, laptops, and edge devices. Beyond text, there are multimodal versions and decision models that return answers in a single pass. The commo...
According to reports, Nvidia is no longer accepting RTX 5090 orders from board partners and has also blocked backorders. If confirmed, the move signals demand exceeding supply and reignites the hardware question for on-premise inference: availability...
A modification to Nvidia RTX 20, 30 and 40 series cards removes a limit that halves performance in AI workloads. The case touches the core of local inference and TCO for those choosing consumer hardware.
