A wave of open-weight releases such as Ling, LiquidAI, Meta Muse Spark, NVIDIA Nemotron Bolt, Tencent Hy-Memory and Reflection Beam uses mixture-of-experts architectures with large total parameters but small active counts to balance capability and inference cost. Many explicitly target local or agentic deployment.
Reflection has released Beam, an open LLM with a 501-billion total and 23-billion active parameter MoE architecture. It supports a 1 million token context and BF16, FP8, and NVFP4 precision. The focus is token-efficient reasoning: about one third of ...
The model card for Tencent Hy-Memory-30B-A3B sparks debate over a slippery claim: an architecture with 3 billion active parameters out of 30 billion total would beat Opus 5 on an unspecified memory test. The issue is not the benchmark but the structu...
Meta has opened the weights of Muse Spark Open, but the point is not just scale: Helion, a PyTorch-native DSL for portable kernels, and TorchTitan support for fine-tuning shift the discussion to hardware control, TCO, and self-hosting. A second- and ...
Meta releases the weights for Muse Spark Open, a model with 816B total and 42B active parameters and a 1M-token context window. The announcement combines Helion and TorchTitan: portable kernels across NVIDIA, AMD, and Intel GPUs, plus PyTorch-native ...
NVIDIA positions Nemotron 3.5 Bolt as a vision-language model for everyday agentic tasks running locally: 74B total parameters, 9B active, native 2-bit quantization, and a 24GB footprint. It supports text, images, video, and up to 256K tokens of cont...
Microsoft AI has developed a local version of MAI Code 1.1 Flash, a coding-optimized mixture-of-experts model with 137 billion total and 6.8 billion active parameters. The news points to a push toward self-hosted development tools on Windows, with im...
inclusionAI's Ling-3.1-flash appears on Hugging Face with 560 billion total parameters, 25 billion active parameters per token, and a 1-million-token context window. The mixture-of-experts configuration reduces per-token compute, but shifts the focus...
NVIDIA's Nemotron Bolt 3.5 74B A9B arrives Oct 15 as a 74B-parameter, 9B-active MoE compressed to 24GB using 2-bit quantization with quantization-aware training. It targets speed and intelligence balance; a 4GB E4B model follows Oct 23.
LiquidAI has introduced LFM2.5-24B-A2B, a reasoning model with 24 billion total parameters and 2.45 billion active. Built for private on-device agents, it claims to compete with Qwen3.6-35B-A3B and run up to 1.7 times faster than comparable MoEs, wit...
OpenBMB has released MiniCPM-V 4.7, a 35B-A3B multimodal model and the first MoE in the series to run smoothly on-device. Combining Lookahead Expert Scheduling, layer-wise quantization, and speculative sampling, it claims 13+ tokens/s on iPhone 17 Pr...
