Executable benchmarks, decision audits, and frameworks like GA-GRPO are exposing how static scores can hide reasoning failures, external-guidance bias, and hallucination risks. RAG and agentic comparisons show that looking right in a benchmark does not guarantee robust real-world behavior.
GA-GRPO formalizes external guidance in LLM reasoning fine-tuning as an operator introducing measurable bias based on divergence between guidance and policy. The theory shows bias is inevitable and grows over time, but optimal guidance weight decreas...
A new framework, GA-GRPO, unifies external guidance methods for LLM reasoning, bounds their bias, and shows that optimal weighting cuts GPU-hours by 31 percent. The bias grows with the divergence between guidance and policy, and the ideal weight decr...
At the Heidelberg Laureate Forum, OpenAI’s Navier–Stokes claim ignited debate. AI companies are turning math into a benchmark, while academics report pressure, costs, and verification shifting outside peer control. The split is not only about theorem...
An executable benchmark for Chinese chess shows that LLM agents can point to the right move and still lose. Across 119 tactical endgames, 8,568 multi-turn trajectories and 12 frontier LLMs, closed-loop success collapses compared with static evaluatio...
A paired evaluation across 11 decision points for LLM agents finds hosted Jev outperforms open-weight Laya on 9 tasks, but neither beats chance on zero-shot routing. The pipeline audit turns a claimed 23.9% saving into an actual 4.3%.
A test on Ollama with models from 1.5B to 120B shows CPU-based exact-match entropy can detect hallucinations without using VRAM. On Qwen 27B AUROC reaches 0.89; on Mistral 7B it matches DeBERTa. But the collapse to 0.09 on 120B signals Confident Mode...
Observational world models are not enough for modular LLM agents: a sequence does not say whether payment authorizes shipment or a hidden factor explains both. The FedCausalCompose framework shows that interventional error decreases with intervention...
An audit shows that LLM agents concentrate retrieval on a small core and leave rare states in a long tail, where prediction errors build up. The CTWM controller allocates prompt budget with a single exponent and reduces tokens by 24.48% on LongMemEva...
A benchmark on 824 multi-hop questions from FRAMES compares 18 RAG pipelines with an agent loop. The best pipeline reaches 78.9%, the agent 92.7%. The most awkward result concerns reranking: a small model drops accuracy by 9 points, a large one barel...
