Qwen3.8, DeepSeek V4 and GLM models are driving a wave of self-hosted inference through aggressive quantization, llama.cpp support and community hardware tests. The shift reframes total cost of ownership and pushes closed vendors to respond.
Z.ai uses the same base model as GLM-5.2 for GLM-5.3, but all gains come from post-training. Coding improves by 50% on the internal Code Bench and reaches state of the art on Terminal Bench 3.0 and Agents' Last Exam. The real shift is the emergent cy...
llama.cpp has merged support for Qwen3.8-Flash-Next and a community report shows 55 tokens/s with a Q4 GGUF on four RTX 3090s. An informal test that shifts attention from software to hardware: for local inference, multi-GPU cost and complexity remain...
A Mac Studio M4 Max with 128GB of unified memory hosted the test of Qwen3.8-Flash-Next, the first model this year to break 94% on tolitius's personal benchmark. The 27B model excels at coding but loses in general knowledge to Gemma 31B and Qwen 3.6 o...
The Hugging Face page for zai-org/GLM-5.3-Flash signals a new model, but no technical details. For self-hosted deployments, the Flash label suggests inference efficiency while missing VRAM, quantization, and token details complicate planning. We anal...
The Qwen team has released Qwen3.8-Flash-Next, a 125B open-weight model with 6B active parameters and a native 262,144-token context. The architecture combines Qwen Sparse Attention, Gated Residual, and a 51B n-gram embedding to reduce long-context l...
A Qwen3.8-27B checkpoint in NVFP4 shows that W4A4 quantization guided by distillation can bring a 27-billion-parameter LLM below 20 GB without significant benchmark loss. AI-Radar reads the result as a shift in VRAM and TCO thresholds, but also as a ...
A fully quantized NVFP4 Qwen3.8-27B checkpoint drops to 19.7 GB from 55.6 GB in BF16 while keeping near-identical scores on GPQA-Diamond and AIME26. The team used quantization-aware distillation with the QUASAR algorithm and supports vLLM on Blackwel...
In less than a month, Qwen3.8-27B collected more likes than the next four models combined, surpassing QwQ-32B. The HuggingFace community favours general-purpose LLMs around 27 and 9 billion parameters, sizes that fit self-hosted setups and consumer G...
An Epyc 7663, 256 GB ECC RAM and one RTX 5090 run a 151 GB MoE LLM at 24 tokens/s with Q8_K_XL quantization. The case challenges the assumption that aggregate VRAM must match model weights: balancing CPU, memory bandwidth and PCIe becomes the real co...
A user tested DeepSeek V4 Flash locally with an Epyc 7663, 256 GB of ECC RAM and an RTX 5090 32 GB. Using Q8_K_XL quantization and 100-128k token contexts, the system reached 23.8-24.6 tokens/s. The result shows an LLM with roughly 151 GB of weights ...
A Reddit user describes how community suggestions turned a frustrating setup into a working local LLM with Home Assistant and visual input. The self-hosted barrier is not hardware but configuration, and the community reduces the cognitive cost. We an...
llama.cpp enables MTP for GLM-4.5-Air, a 106-billion-parameter MoE with only 12 billion active parameters. The change speeds up inference on memory-rich but compute-limited machines like Strix Halo, DGX Spark, and RTX 3090, and strengthens the fine-t...
A team created GGUF quantizations of Qwen 3.8 27B and compared them on an RTX PRO 6000 using a voxel island creation task. Q4_K_M occupies 17.1 GB and decodes at 67 tokens/s, with 95.6% top-1 agreement versus BF16. Qualitative differences are limited...
A one-shot test asks qwen3.8:27b in FP8 on vLLM and an RTX 6000 Pro 96GB to convert 2.1 MB of C into HTML/three.js. With a 262,144-token context window, the file does not fit so the agent must navigate it. Cloud Opus 5 finishes in 21 minutes with an ...
A user with an M1 Max is fed up: Qwen 3.8 27B in xhigh mode takes all night for a single task. They ask for a 35B A3B variant, less brilliant but faster. The request exposes the conflict between long-thinking and consumer hardware in self-hosted scen...
Closed vendor silence after Qwen 3.8 27B signals a shift in competitive pressure: safety rhetoric fades when a 27B LLM can run locally with 16 GB of VRAM. The hardware barrier drops, TCO moves from per-token fees to management costs, and on-premise b...
An industry post notes the silence of closed-model vendors after Qwen 3.8 27B arrived. Earlier, with GLM 5.2 and Kimi K3, the narrative about open-source danger was used to protect the value of paid models. Now a 27-billion-parameter LLM runs locally...
