A new generation of open-weight models—like Ling-3.0-flash, Maple-Preview, and on-device LLMs—employs Mixture of Experts with very few active parameters or ternary quantization. This enables high-quality inference on consumer GPUs and even smartphones, lowering the barrier for local AI deployment.
A developer ran LFM2.5-2.6B, a 2.69-billion-parameter LLM with a 128K context window, on a OnePlus 13 smartphone using CPU alone and Q4_K_M quantization. The custom inference engine, just 450 KB, hits 17 tok/s and supports other model architectures. ...
Maple-Preview is a new open-weight model with 20 billion total parameters, only 1 billion active per token, and ternary weight quantization. The combination slashes VRAM requirements for inference, bringing on-premise deployment within reach of consu...
InclusionAI released Ling-3.0-flash, an open-weight MoE with 124 billion total parameters but only 5 billion active per token. Announced before the Kimi K3 and DeepSeek-V4-Flash wave, its sizing could carve a niche in on-premise deployment, where eff...
Three open-weight LLMs match human agreement on IMO-GradingBench at costs up to 100X lower than frontier models. The unanimous voting rule proves most stable, opening paths for local, sovereign, low-TCO evaluations.
