Topic / Trend Rising

LLM Evaluation and Interpretability Beyond Accuracy

Researchers are mapping internal processes, testing hallucination detectors, and probing reasoning chains to reveal cases where correct answers mask fragile or opaque behavior.

Detected: 2026-11-21 ยท Updated: 2026-11-21

Related Coverage

2026-11-20 โ€ข ArXiv cs.CL

LLM Complexity: why counting tokens is not a true difficulty measure

A new research study challenges the idea that the number of tokens generated by an LLM indicates problem complexity. The proposed metric, LLM Complexity, looks at the area bounded above by the accuracy-token curve and yields stable difficulty ranking...

#Hardware #LLM On-Premise #Fine-Tuning
2026-11-19 โ€ข ArXiv cs.AI

When chain-of-thought stops being neutral evidence

Supervised fine-tuning on reasoning styles can reduce chain-of-thought monitorability without evasion examples. In tests, g-mean^2 drops from 0.94 to 0.68, and to 0.22 with explicit hiding instructions. For self-hosted LLMs, CoT is not reliable foren...

2026-11-18 โ€ข ArXiv cs.CL

CrisisBench evaluates crisis counseling AI without exposing real transcripts

A crisis counseling evaluation suite uses de-identified conversations and synthetic patient models. Tested models underperform human counselors on excessive empathy and questioning in single-turn tasks, while retrieval-augmented generation scores hig...

#Hardware #LLM On-Premise #Fine-Tuning
2026-11-18 โ€ข ArXiv cs.AI

When controlling chain-of-thought makes it harder to monitor

Fine-tuning on reasoning traces rewritten in different voices teaches an LLM to control chain-of-thought without explicit evasion examples. In hidden side-task tests, monitorability drops from 0.94 to 0.22 when the model is told to hide. The effect a...

#LLM On-Premise #Fine-Tuning
2026-11-16 โ€ข ArXiv cs.CL

Beyond Scores: Mapping the Internal Processes of LLMs

A new framework analyzes internal signals in 16 LLMs across 24 benchmarks, distinguishing retrieval-centric tasks from reasoning-centric tasks. The goal is to surface failure modes hidden by aggregate scores and to support more informed model choices...

#LLM On-Premise #DevOps
โ† Back to All Topics