New techniques in hidden reasoning, edit-based decoding, compression, and diffusion LLMs aim to reduce tokens, latency, and computational cost.
The ORL (On-Policy Residual Learning) project aims to speed up diffusion LLMs without sacrificing quality. On a single RTX 5090 the team reports 2,788 tokens/s with LLaDA-2.1-mini in NVFP4; the open-source dlmrt inference engine uses CUDA graphs and ...
A preprint from Stanford, Together AI, UIUC, and Meta introduces Hybrid Reasoning Agents: models that reason internally and emit text only for tool use and final answers. In reported tests they gain about 10%, use half the tokens, run twice as fast, ...
PRISM recovers per-query signal from execution-verified graph corpora that lack rankings. A teacher reads a sibling graph via a zero-initialized low-rank branch; the student aligns per-decision targets on self-sampled trajectories. Across six benchma...
When several models reach near-perfect scores, benchmarks lose their ability to differentiate them. A study shows that full task transcripts from saturated benchmarks retain predictive information. Pairwise judging reduced prediction error by 5.3 tim...
Researchers use an inverse LLM to reconstruct reasoning traces from answers alone. Trained once, the model generates intermediate steps offline for prompt-answer data, avoiding the multiple rollouts of reinforcement learning. On Qwen3-0.6B, a single ...
AutoIndexer introduces a decoding mechanism based on a chain of edits: the model can insert, substitute, or delete tokens after generation, moving a cursor like in a text editor. Trained on perturbed sequences, it learns to correct erroneous contexts...
A new compression format for LLMs uses algebraic integers instead of the usual floating-point or fixed-point formats. The research shows competitive accuracy with the MX format at equal bit-per-parameter budgets. For self-hosted inference, the potent...
