As models approach near-perfect scores, saturated benchmarks lose discriminative power, while multilingual, forgetting/relearning, and generative studies expose blind spots in evaluation. New approaches use full transcripts, Bengali tests, and information-theoretic C-tests to recover signal.
A Qwen2.5-1.5B study shows that a previously learned fact is reacquired faster after partial forgetting, even when current accuracy fails to detect it. The advantage is binding-specific, not just familiarity with entities or format. In a secondary sc...
When several models reach near-perfect scores, benchmarks lose their ability to differentiate them. A study shows that full task transcripts from saturated benchmarks retain predictive information. Pairwise judging reduced prediction error by 5.3 tim...
A zero-shot benchmark of eight LLMs shows that summarization quality does not transfer from English to Bengali. GPT-5.6 Luna leads several English lexical metrics, but LLaMA 3.3 70B achieves the best Bengali ROUGE and BLEU scores; Gemini 2.5 Flash po...
A new test uses Levin's Kt and language models as representational machines to generate items whose difficulty is measurable in bits, before any human or frontier model can solve them. Twelve LLMs were evaluated on a first certified battery, showing ...
An open-source leaderboard aggregates 22,113 benchmark results and shows China's open-weight models about seven months behind US closed models. The methodology separates 14,436 independent evaluations from 7,677 lab-reported scores. For on-prem deplo...
