Research is moving beyond final-answer scoring toward step-level rewards, evidence boundaries, and explicit harnesses to catch silent failures in AI agents and RAG systems. New frameworks reward per-step correctness and uncertainty calibration rather than only terminal outcomes.
A training framework teaches grounded QA models to answer only when evidence becomes sufficient, locating the exact point where abstention gives way to a response. On HotpotQA, 2WikiMultiHopQA and MuSiQue, with Qwen2.5-3B-Instruct and LoRA, the metho...
PRO-Step introduces step-level supervision for Retrieval-Augmented Generation. Instead of evaluating only the final answer, a generative process reward model checks logical validity and evidential grounding at every step of multi-hop reasoning. The m...
A study across 400 trajectories shows outcome-only LLM judges catch 84% of loud faults but only 45% of silent ones while flagging 33% of correct trajectories. A step-rubric judge reaches 77% silent recall with zero false alarms at 3x cost. No judge r...
ERR+ introduces a two-phase RLVR framework that rewards token-level entropy drops and relative length efficiency. Correct traces show more frequent and larger entropy drops during thinking. Tests on five datasets report improved accuracy and concisen...
DS-Lighting is an open-source toolkit that makes the agent harness explicit for data-science automation. It decomposes the harness into four reusable layers—data, workflow, execution, and evaluation—and represents agents as executable operator progra...
