New open-weight models combine large total parameter counts with low active parameters, long contexts, and permissive licenses, targeting constrained local hardware. Examples include Kolibri-1, Ling-3.1-flash, and other MoE architectures with small active parameter counts.
Blockway releases a 32B preview model with a hybrid architecture. It combines 54 linear-attention layers, 17 compressed-sparse attention layers, one full-attention layer, and hashed n-gram memory in host RAM. Only 18 of 72 layers keep a KV cache, ena...
A researcher trained Apex-2, a decoder-only MoE with 3.87 billion total parameters and 1.45 billion active per token, from scratch on 86.5 billion tokens while scaling from one to two GH200 with DiLoCo. The base checkpoint reaches HumanEval+ parity w...
Aleph Alpha publishes the Kolibri-1 card: 78 billion total parameters, 3.46 billion active, and a 1 million token context under Apache 2.0. AI-RADAR analysis shows the constraint shifting from compute to memory and KV cache, with direct implications ...
The Kyojin project runs two roughly 300-billion-parameter MoE models on a single 128 GB mini PC with AMD Strix Halo. Measured on a Ryzen AI Max+ 395, GLM-5.3-Flash and MiMo-V2.6-Flash reach up to 650 tokens/s prefill and 26 to 44 tokens/s decode. EXL...
Aleph Alpha releases Kolibri-1, an LLM with 78 billion parameters, 3.46 billion active per token, and up to 1 million tokens of context under Apache 2.0. The combination touches three constraints of self-hosted deployments: memory, compute, and licen...
Microsoft has released a 4-billion-parameter model derived from Qwen3.5-4B, trained with reinforcement learning on synthetic SWE environments for agentic software engineering. The approach without behavioral distillation aims to reduce dependence on ...
As reported, Ling-3.1-flash combines 560 billion total parameters, 25 billion active per token, and a context window up to one million tokens, with a two-week free trial and then an open-source release. The profile points to a sparse architecture: lo...
