Research projects reduce compute and memory through MoE optimizations, 1-bit weights, MTP decoding, dynamic quantization and distributed training on consumer hardware.
The November 10 deadline for PyTorch Day China 2026 reveals an agenda focused on distributed training, efficient inference, hardware acceleration, and physical AI. The analysis: the chosen tracks show the framework increasingly looking at production ...
The PyTorch Day China 2026 Call for Proposals closes November 10. The December 12 event in Beijing highlights five topic areas, with a notable emphasis on efficient inference, hardware acceleration, and physical AI. For teams evaluating local stacks ...
Unsloth has released new Unsloth Dynamic v3.1 GGUFs for Qwen3.8-Flash-Next, with 10-15% accuracy gains on agentic benchmarks such as DeepSWE and Terminal Bench. It includes a switching process between BF16, Q8_0, Q5_1, and IQ4_NL precisions, with dec...
ISTA DASLab has released a Coder build of DeepSeek-V4.1-Flash-1017 with half its routed experts removed and transformer weights compressed to 1 bit per parameter. The GGUF takes 72.1 GB of transformer weights, down from 138.2 GB unpruned, and retains...
The JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP repository on Hugging Face introduces an integrated multi-token prediction head that accelerates decoding by 1.4–1.8x while accepting about 90% of proposed tokens. vLLM 0.33, transformers 5.20 and llama....
A new study compares GRPO with the lightweight LiSAO and finds an inversion: GRPO leads on reasoning tasks, LiSAO on agentic ones. LiSAO's early lag comes from a value model that is not yet well trained. Li²SAO addresses this with group-relative acto...
CrowdGPT releases version 2.6 and shows a concrete path to train a one-billion-parameter LLM on distributed PCs, outside datacenters. Six days for 4.5 billion tokens, with three-client Byzantine checks, cryptographic signing for updates, and a collab...
A Mixture-of-Experts training system cuts cumulative MoE-layer training time by 21.2 to 44.8 percent against SYMI by reducing memory demand in backpropagation and rebalancing retained work. LRMoE plans expert placement and retention quotas, selects t...
llama.cpp v0.8.0 adds compatibility with DFlash2 for Qwen3.8-Flash-Next. Tests on Apple Silicon M5 Max with quantized GGUFs show an average 17% decoding throughput gain over MTP, with peaks of 39% on list-like prompts and a minor prose regression. Th...
ISTA attributes the benchmark gap to Unsloth's 32K token reasoning budget, versus the 128K output tokens allowed in its own setup. The case of the 763B model with a 1M context window shows why inference cost, more than raw parameters, decides the res...
SenseNova brings to Hugging Face a multimodal LLM with reused decoder blocks and hybrid attention. With 24 layers run twice and 56 executions from 32 distinct layers, it targets agentic workloads and documents up to 256K tokens. The approach reduces ...
At EMNLP 2026, T-Tech researchers presented LIMe, an attention extension that reuses key-value buffers from earlier layers. In tests with models around one billion parameters, training reached the same loss as the LLaMA baseline using 15% less comput...
