Efforts to decentralize AI training are gaining traction, with projects exploring methods to train LLMs on distributed consumer PCs. This approach aims to reduce reliance on large data centers and optimize training loads for memory-constrained nodes, democratizing access to AI development.
CrowdGPT releases version 2.6 and shows a concrete path to train a one-billion-parameter LLM on distributed PCs, outside datacenters. Six days for 4.5 billion tokens, with three-client Byzantine checks, cryptographic signing for updates, and a collab...
A Mixture-of-Experts training system cuts cumulative MoE-layer training time by 21.2 to 44.8 percent against SYMI by reducing memory demand in backpropagation and rebalancing retained work. LRMoE plans expert placement and retention quotas, selects t...
At EMNLP 2026, T-Tech researchers presented LIMe, an attention extension that reuses key-value buffers from earlier layers. In tests with models around one billion parameters, training reached the same loss as the LLaMA baseline using 15% less comput...
