Topic / Trend Rising

Chain-of-Thought Transparency and LLM Evaluation

Fine-tuning on reasoning styles can make chain-of-thought less monitorable, while new audits and reasoning benchmarks show that accuracy alone does not guarantee valid reasoning or reliable hallucination detection.

Detected: 2026-11-19 ยท Updated: 2026-11-19

Related Coverage

2026-11-19 โ€ข ArXiv cs.AI

When chain-of-thought stops being neutral evidence

Supervised fine-tuning on reasoning styles can reduce chain-of-thought monitorability without evasion examples. In tests, g-mean^2 drops from 0.94 to 0.68, and to 0.22 with explicit hiding instructions. For self-hosted LLMs, CoT is not reliable foren...

2026-11-18 โ€ข ArXiv cs.CL

The hidden cost of refusal: measuring user disengagement in real-world LLMs

RAW uses LLM stochasticity as a natural experiment to measure the causal effect of refusals on user re-engagement without A/B tests. Applied to 15 deployed models in LMSYS-Chat-1M and WildChat, it finds that for about half of the precisely measurable...

#Hardware #LLM On-Premise #Fine-Tuning
2026-11-18 โ€ข ArXiv cs.AI

When controlling chain-of-thought makes it harder to monitor

Fine-tuning on reasoning traces rewritten in different voices teaches an LLM to control chain-of-thought without explicit evasion examples. In hidden side-task tests, monitorability drops from 0.94 to 0.22 when the model is told to hide. The effect a...

#LLM On-Premise #Fine-Tuning
2026-11-16 โ€ข ArXiv cs.CL

Beyond Scores: Mapping the Internal Processes of LLMs

A new framework analyzes internal signals in 16 LLMs across 24 benchmarks, distinguishing retrieval-centric tasks from reasoning-centric tasks. The goal is to surface failure modes hidden by aggregate scores and to support more informed model choices...

#LLM On-Premise #DevOps
โ† Back to All Topics