New compact architectures, tiny parameter counts, and ultra-low-bit quantization are pushing language and decision models onto CPUs, microcontrollers, and memory-constrained devices. Releases like Kolibri-1, TinyDecide, LittleBit, and small MoE variants reduce compute requirements while keeping useful capabilities.
The JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF collection highlights a compact MoE design with roughly 2.5 billion active parameters and GGUF distribution. For on-prem teams, this mix lowers compute per token, supports quantization on consumer hardw...
An arXiv preprint describes LittleBit, an ultra-low-bit quantization approach based on latent factorization and quantization-aware training for very small models. The news matters for on-premise LLM operators: it is not just about saving space, but a...
The LiquidAI/d1-3B page on Hugging Face, shared without technical details, places the project in the compact LLM segment. The 3B label suggests a model aimed at local inference, self-hosted setups, and data sovereignty constraints. The article analys...
A 19,969-parameter, 81 KB FP32 model generates short stories with goal, problem, and resolution while running on CPU. MacroStories lowers the bound for coherent narrative generation and questions assumptions about local inference.
TinyDecide is a 10-million-parameter model with a roughly 6 MB footprint, smaller than every entry on the Decision Index leaderboard. It runs in the browser, Node.js, Python, Rust, and on an ESP32. For edge and on-premise deployments, the signal is n...
Blockway releases a 32B preview model with a hybrid architecture. It combines 54 linear-attention layers, 17 compressed-sparse attention layers, one full-attention layer, and hashed n-gram memory in host RAM. Only 18 of 72 layers keep a KV cache, ena...
A researcher trained Apex-2, a decoder-only MoE with 3.87 billion total parameters and 1.45 billion active per token, from scratch on 86.5 billion tokens while scaling from one to two GH200 with DiLoCo. The base checkpoint reaches HumanEval+ parity w...
Aleph Alpha publishes the Kolibri-1 card: 78 billion total parameters, 3.46 billion active, and a 1 million token context under Apache 2.0. AI-RADAR analysis shows the constraint shifting from compute to memory and KV cache, with direct implications ...
Aleph Alpha releases Kolibri-1, an LLM with 78 billion parameters, 3.46 billion active per token, and up to 1 million tokens of context under Apache 2.0. The combination touches three constraints of self-hosted deployments: memory, compute, and licen...
Microsoft has released a 4-billion-parameter model derived from Qwen3.5-4B, trained with reinforcement learning on synthetic SWE environments for agentic software engineering. The approach without behavioral distillation aims to reduce dependence on ...
