The open-source community accelerates development of lightweight, quantized, and fine-tuned LLMs, making powerful AI accessible for local inference and reducing dependency on proprietary cloud APIs.
The 2-bit quantized GLM-5.2 shrinks from 1.51TB to 238GB while retaining ~82% accuracy. It can now run locally on a 256GB Mac or systems with enough RAM/VRAM via llama.cpp and Unsloth Studio, opening new possibilities for on-premise AI deployment.
North Mini Code team drops a 4-bit quantized version on Hugging Face, requiring around 20 GB of memory. The model now runs on local hardware via Ollama and llama.cpp-based runtimes, and is also available through the OpenRouter API – a move that boost...
Z.ai has open-sourced its GLM 5.2 model, generating significant community excitement. Developers and enterprises are now eagerly anticipating a "Flash" series successor, ideally within the 27 to 120 billion parameter range, to optimize on-premise and...
A recent update to llama.cpp introduces comprehensive model management through its API, enabling the loading, unloading, and downloading of LLMs on demand directly from a programmatic interface. This enhancement simplifies on-premise deployment, offe...
The release of GLM 5.2, a 744-billion-parameter Large Language Model under an MIT license, marks a significant development for on-premise AI. While the full model necessitates enterprise-grade clusters, its potential for distillation and fine-tuning ...
The Large Language Model (LLM) landscape expands with the arrival of GLM-5.2, a new model released by zai-org. This development occurs as companies carefully evaluate deployment options, balancing performance, costs, and data sovereignty. For CTOs an...
A new open-weights LLM, Qwable-v1, has been released, derived from Anthropic's controversial Claude Fable-5. Distilled on a single H200 GPU, it offers agentic coding and tool-use capabilities, with GGUFs available for on-premise deployment, raising q...
The `llama.cpp` framework recently integrated support for the Command A Plus and North Mini Code Large Language Models. Thanks to community contributions, GGUF files for Command A Plus have been made available, facilitating efficient execution of the...
The integration of EAGLE support into the open-source `llama.cpp` project marks a significant evolution for the efficient execution of Large Language Models in local environments. This move strengthens the Framework's ability to offer high-performanc...
A recent poll on X, conducted by z.ai, reveals declining support for Large Language Models with open weights distributed under an MIT license. With 1,800 votes cast and only a few hours remaining, the preliminary result suggests a potential shift in ...
The assertion that open source AI must win reflects a growing need for companies to maintain control, data sovereignty, and transparency over their artificial intelligence workloads. This approach is crucial for those evaluating on-premise deployment...
The soaring costs associated with artificial intelligence are prompting companies to reconsider their deployment strategies. As cloud-based LLM subscription services hit a "pricing wall," an increasing number of enterprises are exploring open-source ...
