Executable benchmarks, multi-turn decision audits, and bias-variance formalization reveal reliability gaps that static leaderboards miss.
A shared learning rate can manufacture false advantages when comparing fine-tuning recipes with different parameter counts. In SVD-based KV-cache compression, healing only the encoder appeared superior, but with per-arm learning rates it reaches pari...
Four open decision models tested on a rented RTX 4090: Laya, d1, Clef-Flash and Lev read 9,534 words and flagged centipede names. Laya is fastest at 3.9 ms per word on llama.cpp; Lev is most accurate on the target but about 13x slower. The test shows...
GA-GRPO formalizes external guidance in LLM reasoning fine-tuning as an operator introducing measurable bias based on divergence between guidance and policy. The theory shows bias is inevitable and grows over time, but optimal guidance weight decreas...
A new framework, GA-GRPO, unifies external guidance methods for LLM reasoning, bounds their bias, and shows that optimal weighting cuts GPU-hours by 31 percent. The bias grows with the divergence between guidance and policy, and the ideal weight decr...
At the Heidelberg Laureate Forum, OpenAI’s Navier–Stokes claim ignited debate. AI companies are turning math into a benchmark, while academics report pressure, costs, and verification shifting outside peer control. The split is not only about theorem...
A community benchmark shows local RTX 5090 setups hitting 96% on coding tasks, close to Claude Code Opus 5.5's 100%. The key variable isn't just the model: with the same GPU and quantization, harness choice swings results from 22% to 96%. MTP variant...
An executable benchmark for Chinese chess shows that LLM agents can point to the right move and still lose. Across 119 tactical endgames, 8,568 multi-turn trajectories and 12 frontier LLMs, closed-loop success collapses compared with static evaluatio...
A paired evaluation across 11 decision points for LLM agents finds hosted Jev outperforms open-weight Laya on 9 tasks, but neither beats chance on zero-shot routing. The pipeline audit turns a claimed 23.9% saving into an actual 4.3%.
