New work explores latent self-explaining reasoning, topology of token representations, recurrent latent thinking, block causal memory, and extraction of hidden reasoning traces. These techniques aim to make Transformers more efficient or transparent while revealing overthinking and memorization behaviors.
BCMT separates local token interaction from global context propagation. In tests up to 1024 tokens, it achieves validation performance comparable to Dense Transformers, with higher training throughput and lower memory consumption. The exponential cau...
SELR introduces a single model that reasons in latent space and decodes its own reasoning into human-readable steps. A multi-task objective combines Answer Loss and CoT Loss, removing external decoders and keeping the explanation tied to the actual r...
A recurrent retrofit on Qwen2.5-0.5B-Instruct enables iterative latent reasoning at small parameter budgets. It matches the full block, extrapolates supervised depth, and answers 7.6 times faster than a scratchpad model. The inverse task reveals cata...
A paper shows how to extract all reasoning tokens from Claude and GPT models. It reveals widespread overthinking, benchmarks tainted by memorization, and China’s exploitation of the gap to distill frontier models. Closing this leak redefines the real...
A new framework uses persistent homology to track the transformation of token representations layer by layer. Global topological analysis could indicate where and how models develop relevant features, with implications for optimization, pruning, and ...
