New GGUF and DFlash2 formats deliver double-digit throughput and accuracy gains, but also reignite debates on reproducibility. Community calls for private quality benchmarks and disputes over token budgets show the evaluation gap.
Unsloth has released new Unsloth Dynamic v3.1 GGUFs for Qwen3.8-Flash-Next, with 10-15% accuracy gains on agentic benchmarks such as DeepSWE and Terminal Bench. It includes a switching process between BF16, Q8_0, Q5_1, and IQ4_NL precisions, with dec...
A Reddit post turns a llama.cpp fork into a formula with square brackets: ultra-specific quantization and hardware produce incredible but irreproducible results. The case exposes the fragile side of local benchmarks and what it means for those evalua...
ISTA DASLab has released a Coder build of DeepSeek-V4.1-Flash-1017 with half its routed experts removed and transformer weights compressed to 1 bit per parameter. The GGUF takes 72.1 GB of transformer weights, down from 138.2 GB unpruned, and retains...
A LocalLLaMA user proposes moving beyond token/s chasing and building private quality benchmarks, especially across quantization levels. The idea: keep test data private and share only comparative charts. It is a reaction to benchmark noise and score...
A Reddit thread challenges the convictions of the self-hosted community: inflated open-weight rankings, skepticism toward Haiku 5.5, the misunderstood Strata project, and the uselessness of the ThreeJS benchmark. For those evaluating on-premise deplo...
llama.cpp v0.8.0 adds compatibility with DFlash2 for Qwen3.8-Flash-Next. Tests on Apple Silicon M5 Max with quantized GGUFs show an average 17% decoding throughput gain over MTP, with peaks of 39% on list-like prompts and a minor prose regression. Th...
ISTA attributes the benchmark gap to Unsloth's 32K token reasoning budget, versus the 128K output tokens allowed in its own setup. The case of the 763B model with a 1M context window shows why inference cost, more than raw parameters, decides the res...
