A wave of open-weight MoE models with massive total parameters but few active ones, combined with aggressive quantization, is making high-performance LLMs runnable on consumer hardware.
Maple-Preview applies ternary weights to a 20B-parameter model, shrinking memory to ~5 GB while activating only 1B parameters per token via an MoE-like design. This preview hints at local inference on consumer GPUs, but framework support remains imma...
Maple-Preview is a new open-weight model with 20 billion total parameters, only 1 billion active per token, and ternary weight quantization. The combination slashes VRAM requirements for inference, bringing on-premise deployment within reach of consu...
InclusionAI has released Ling-3.0-flash weights on Hugging Face. The model packs 127.5 billion parameters with 512 experts, 8 active per token. The real story is the official FP8 version: 128GB, enough for a single unified-memory system or a multi-GP...
InclusionAI released Ling-3.0-flash, an open-weight MoE with 124 billion total parameters but only 5 billion active per token. Announced before the Kimi K3 and DeepSeek-V4-Flash wave, its sizing could carve a niche in on-premise deployment, where eff...
thinkingmachines has released Inkling-Small, a mixture-of-experts LLM with 276 billion parameters, only 12 billion active, a 1 million token context window, and NVFP4 and GGUF quantizations. AI-RADAR analysis focuses on the hidden challenge of the KV...
Thinkingmachines released Inkling-Small, an LLM with 276B total parameters but only 12B active and a 1 million token context window. NVFP4 quantization and Unsloth's GGUF files make local inference viable on modern NVIDIA hardware. The MoE architectu...
Inference provider Baseten has publicly released GLM-5.2-Vision, a version of GLM 5.2 augmented with a vision encoder from Kimi k2.6 and quantized in NVFP4 format. The move addresses a community-flagged gap and lowers the hardware barrier for on-prem...
A user experiment runs the Kimi K3 model locally with Q2_K quantization, 768GB of system RAM and two NVIDIA 5090 GPUs. Prefill at 50-70 tokens/s and decoding around 4 tokens/s, but the generation speed increases over time, hinting at cache or swap dy...
