Compact open-weight LLMs under 4B-10B parameters and BabyLM results show gains from data organization, architecture and single-GPU training rather than parameter scale alone.
An independent researcher has prepared a dense 9.4-billion-parameter model for training on a single card, with distillation from Llama 3 and architectural simplifications. No benchmarks are public, but the signal for on-premise AI is clear: the bottl...
An independent researcher has opened a roughly 9.4B parameter dense model designed for single-card training. The project uses a Llama 3 tokenizer, distilled logit data, and architecture cues from Moonshot and Qwen. The real test is not technical; it ...
In the BabyLM 2026 Strict-Small, Qiushi Engine gained 0.23 Overall points across nine metrics without increasing scale, but by reorganizing experience around contextual dependencies. For teams evaluating self-hosted LLMs, the relevant signal is the d...
Qiushi Engine ran an end-to-end autonomous research program on BabyLM 2026 Strict-Small, using 10 million corpus words and 100 million cumulative word presentations. Three stages linked frontier advancement, principle discovery, and principle-guided ...
OpenBMB has released MiniCPM5-2B, a 2-billion-parameter open weights model scoring 15 on the Artificial Analysis Intelligence Index v4.2, the highest among open models up to 4B. For self-hosted deployments, the small size and open weights lower the p...
A llama.cpp pull request adds support for Spark-X2.5, two compact 4B and 1.7B parameter LLMs with a native context window of up to 1M tokens and a hybrid sliding-window attention architecture. The move lowers the barrier for local and self-hosted inf...
