Experiments show that capable small LLMs can be trained for a few hundred dollars, and new architectures such as BDH, Luth-2, TEXAS and BitNet reduce compute barriers. This is redefining cost, data sovereignty and language coverage for self-hosted AI.
Unsloth Desktop is the first open-source app for running and training LLMs locally. It spans Windows, macOS, Linux and supports NVIDIA, AMD, Intel, and Mac hardware. It claims 2× faster training, 70% less VRAM, private search, RAG, MCP, and an OpenAI...
The new Luth-2-0.8B and 2B models set a new state of the art for French, outperforming models up to three times their size across language and math benchmarks. Light enough to run locally, they enable practical on-premise deployment and data sovereig...
An experiment shows that for $200 on rented GPUs you can train a 1.1B parameter LLM and deploy it on CPU or even a smartwatch. The negligible cost rewrites the TCO calculus for organizations evaluating self-hosted solutions: data sovereignty becomes ...
A personal project demonstrates how to train a 1.1-billion-parameter LLM from scratch on 20 billion tokens for around $200. The model, fine-tuned with LoRA and quantized, runs on CPU and even on a WearOS watch. Concrete evidence that data sovereignty...
A new approach called TEXAS exploits how mixture-of-experts models activate their sub-models to steer fine-tuning, focusing supervision only on relevant tokens. For those running LLMs on-premises, this means more efficient adaptation on proprietary d...
Pathway introduces BDH, a post-transformer architecture matching GPT-2 scaling from 10 million to 1 billion parameters while training on ordinary GPUs. A signal for those seeking self-hosted LLMs without extreme hardware constraints.
A C99 inference engine with zero dependencies drives the 2-billion-parameter ternary BitNet model to 36.25 tokens per second on an Intel Xeon CPU, using custom SIMD and VNNI instructions. But the real ceiling isn't compute: at batch size 1, memory ba...
