Benchmarks and safety evaluations are under scrutiny: saturated scores, limited language transfer and evaluator limitations are pushing the community toward new tests and third-party oversight. Governance, security incidents and public communication are also part of the debate.
Clem Delangue, co-founder and CEO of Hugging Face, appears in Netflix's 'AI Gone Rogue' and says the film went 'full doomer mode' and isn't accurate. The criticism signals a tension between apocalyptic AI narratives and the practical, verifiable choi...
Anthropic is giving METR employee-like access to evaluate the safety of its systems and has started early discussions with other organizations, including Hugging Face. This shifts trust from vendor self-assessment to independent, distributed auditing...
Nscale becomes a Platinum member of the PyTorch Foundation and gains a seat on its governing board. The move follows the planned acquisition of Anyscale, creator of Ray, the open-source framework for distributed computing. Nscale integrates data cent...
OpenCode, an open source coding agent popular as an alternative to Claude Code and Cursor, suffered a breach of its online platform: attackers stole some users' API keys and used them to inflate AI account charges. AI tools asked to inspect the malic...
Irene Solaiman, Hugging Face's chief policy officer, has warned that model benchmarking methods are complex and still maturing. Embedded evaluators are not a shortcut. For anyone choosing an LLM, evaluation is a governance issue, not just a technical...
A Qwen2.5-1.5B study shows that a previously learned fact is reacquired faster after partial forgetting, even when current accuracy fails to detect it. The advantage is binding-specific, not just familiarity with entities or format. In a secondary sc...
At Open Source Summit Europe, the focus was on GPUs and AI, but the most concrete part came from CRA compliance plans, OSPO metrics, and migration targets such as Berlin's 70 percent open source by 2032. Bureaucracy is becoming infrastructure.
When several models reach near-perfect scores, benchmarks lose their ability to differentiate them. A study shows that full task transcripts from saturated benchmarks retain predictive information. Pairwise judging reduced prediction error by 5.3 tim...
A zero-shot benchmark of eight LLMs shows that summarization quality does not transfer from English to Bengali. GPT-5.6 Luna leads several English lexical metrics, but LLaMA 3.3 70B achieves the best Bengali ROUGE and BLEU scores; Gemini 2.5 Flash po...
A new test uses Levin's Kt and language models as representational machines to generate items whose difficulty is measurable in bits, before any human or frontier model can solve them. Twelve LLMs were evaluated on a first certified battery, showing ...
