Research on efficient LLM adaptation and compression is accelerating beyond classic quantization. Methods such as FPO, ECASQ, and depth pruning reduce memory and training cost while often trading accuracy or robustness.
A single developer reduced Qwen3.8-27B to 22.7 billion parameters with depth pruning, no fine-tuning. Distributed only as MLX for Apple Silicon, the model shows trade-offs: lower memory and compute pressure, but losses on edge cases. For on-premise d...
ECASQ jointly optimizes adaptive quantization and lossless compression by minimizing MSE under an entropy budget and an unbiasedness constraint. The optimal dynamic program runs in O(sd^2) time and O(d^2) space. A GPU-friendly approximate version red...
A developer applied depth pruning to Qwen3.8-27B, bringing it to roughly 22.7 billion parameters without fine-tuning. The model, available in bf16, q8, and q4 on MLX, handles coding, agentic use, and multi-turn conversations with limited degradation,...
FPO proposes fine-tuning LLMs without propagating errors between layers or building autograd graphs. The method reduces peak training memory and increases throughput, but concentrates adaptation in the final layers. For self-hosted deployments, the o...
FPO adapts LLMs without a backward pass through the model body, reaching 2.7–3.2x the throughput of standard fine-tuning and about 40% less peak training memory. On OLMo-2-7B, Qwen3-8B, and Falcon3-7B, it improves in-domain perplexity while leaving M...
