A wave of compact, quantized and CPU-friendly models is lowering the barrier for local inference, while new compression and lightweight correction techniques reduce VRAM and training costs.
PRQuant combines channel reorganization and static residual compensation to reduce inference overhead in low-bit quantization. The method is training-free, builds contiguous tail blocks offline, and avoids dynamic online gathering by turning scattere...
SupraLabs has released Supra2-IMG, a 100M parameter DiT text-to-image model trained from scratch in under 10 hours on a single H100. It generates 256×256 images with competitive quality and runs locally: about 20 seconds on CPU and 2 on GPU. A concre...
A post on r/ROCm highlights the problem of FP4-optimized inference engines: fast in benchmarks, but often unreliable on small dense models. Compression can work on Large Language Models with many parameters, but on small models logical errors become ...
A layer-wise curriculum learning method for LLM compression starts with easy optimization tasks and moves to harder ones, stabilizing knowledge transfer. It reduces GPU memory usage and training hours by more than 50% on BERT and GPT-2, and outperfor...
From the AI lab comes a signal: a small LLM could shift attention from brute-force scale to efficiency. Without public technical data, the news matters as a directional indicator: compact models suit local deployment, data control, and cost-conscious...
Ternary Bonsai 2 is available on Hugging Face: it derives from Qwen3.8-27B, keeps the hybrid-attention architecture, and uses ternary weights to shrink below 6GB. According to the model card, it is 9x smaller than FP16 while retaining 98.2% of the in...
A Hugging Face page and online demo present a 230-million-parameter model fine-tuned for chess. The news is small, but it opens a practical discussion on specialized compact models for local deployments.
A 34-million-parameter module corrects 53.3% of base-model errors in Gemma 4 E2B without degrading tested benchmarks. A LoRA baseline reaches 83.3% correction but loses up to 75% capability. The KL anchoring term and logit-level correction are centra...
