Methods like FPO reduce fine-tuning memory and skip backpropagation between layers, while block causal memory, self-explainable latent reasoning, and recurrent retrofits improve LLM efficiency. Compact models such as Luth-2 and Nemotron MoE show strong local performance.
FPO proposes fine-tuning LLMs without propagating errors between layers or building autograd graphs. The method reduces peak training memory and increases throughput, but concentrates adaptation in the final layers. For self-hosted deployments, the o...
FPO adapts LLMs without a backward pass through the model body, reaching 2.7–3.2x the throughput of standard fine-tuning and about 40% less peak training memory. On OLMo-2-7B, Qwen3-8B, and Falcon3-7B, it improves in-domain perplexity while leaving M...
BCMT separates local token interaction from global context propagation. In tests up to 1024 tokens, it achieves validation performance comparable to Dense Transformers, with higher training throughput and lower memory consumption. The exponential cau...
SELR introduces a single model that reasons in latent space and decodes its own reasoning into human-readable steps. A multi-task objective combines Answer Loss and CoT Loss, removing external decoders and keeping the explanation tied to the actual r...
A study on Qwen3.6-35B-A3B shows that sensitivity to expert masking in MoE models depends heavily on depth. Early and middle layers are fragile, while late layers tolerate aggressive cuts. On H100 servers, targeted late-layer policies preserve far mo...
Writer has introduced a new model built as a post-training variation on Z.ai's open source GLM-5.2, alongside an upgraded harness aimed at containing token costs. The move shifts attention from raw performance to operational sustainability: for self-...
A recurrent retrofit on Qwen2.5-0.5B-Instruct enables iterative latent reasoning at small parameter budgets. It matches the full block, extrapolates supervised depth, and answers 7.6 times faster than a scratchpad model. The inverse task reveals cata...
The new Nemotron-3.5 Lightning 30B-A3B in BF16 arrives on Hugging Face. This move shifts the focus toward ultra-efficient MoE architectures, designed for those who run LLMs on their own hardware, cutting cloud dependency without sacrificing performan...
The new Luth-2-0.8B and 2B models set a new state of the art for French, outperforming models up to three times their size across language and math benchmarks. Light enough to run locally, they enable practical on-premise deployment and data sovereig...
