llama.cpp adopts semantic versioning, new GGUF quantization paths improve accuracy and speed, and techniques like FPO reduce memory for local fine-tuning. The growing number of models on Hugging Face also makes curation a first-class problem.
The new llama.cpp release, with source and pre-built binaries, marks a step for self-hosted inference. Less friction for developers and companies running LLMs locally, more pressure on cloud-only services.
DeepSeek has updated its harness to version 0.1.1, adding the multimodal model DeepSeek-V4-Flash-Vision-Exp and support for native image requests. Commands like /goal and /plan accept text and images; MCP/ACP keep persistent attachments. For self-hos...
ECASQ jointly optimizes adaptive quantization and lossless compression by minimizing MSE under an entropy budget and an unbiasedness constraint. The optimal dynamic program runs in O(sd^2) time and O(d^2) space. A GPU-friendly approximate version red...
Three new Ornith 1.5 models—9B, 35B-A3B, and 397B—have appeared on Hugging Face, each with GGUF versions. The immediate availability of quantized formats signals a direct focus on local and self-hosted deployment, prompting reflection on the trade-of...
The second version of DFlash did not arrive with an announcement but through PR 27342 on llama.cpp and ready-made GGUF quantized files for Qwen 3.8 27B and Muse Glimmer. AI-Radar analyzes the shift: the on-premise bottleneck is not the model but the ...
The original authors of DFlash GGUF quants have published a second version alongside a llama.cpp pull request. The package covers Qwen 3.8 27B and Muse Glimmer, pointing to tight integration between model optimization and the local runtime. For self-...
Hugging Face has passed three million models published on the Hub. The number includes quantized versions, fine-tunes and conversions, rather than distinct base models. For teams managing local stacks, the milestone shifts the bottleneck from model a...
FPO proposes fine-tuning LLMs without propagating errors between layers or building autograd graphs. The method reduces peak training memory and increases throughput, but concentrates adaptation in the final layers. For self-hosted deployments, the o...
FPO adapts LLMs without a backward pass through the model body, reaching 2.7–3.2x the throughput of standard fine-tuning and about 40% less peak training memory. On OLMo-2-7B, Qwen3-8B, and Falcon3-7B, it improves in-domain perplexity while leaving M...
A new paper on LLVM offloading to GPUs with Rust discusses the prospects of leveraging memory safety in GPU kernels. Compared with C++/CUDA/HIP, Rust's model can reduce entire classes of critical bugs. For self-hosted deployments this has implication...
llama.cpp drops sequential build numbers and adopts semantic versioning with v0.1.0. For self-hosted and on-premise deployments, the move gives operators clearer signals about breaking changes, dependency pinning, and upgrade planning, even though 0....
KTransformers, a framework for heterogeneous LLMs, releases version 0.7 with expanded AVX-512 support, a targeted change for AMD EPYC servers. For self-hosted teams, the message is structural: the CPU is no longer a fallback, but an active component ...
A short thank-you post brings attention back to Georgi Gerganov, creator of llama.cpp. The open source project changed how Large Language Models run on common hardware, lowering barriers for self-hosted deployment and data sovereignty. Behind the gra...
