A wave of open-weight models, led by Qwen3.8-Flash-Next and 27B-class releases, is being compressed and served on consumer GPUs and local workstations. The focus is on long contexts, high token rates, and smaller VRAM footprints.
The Reddit post is thin, but the message is clear: MTP support for Qwen3.8-Flash-Next in GGUF format promises to raise tokens per second in llama.cpp runs. More than a single optimization, it signals the direction of the open-source stack for local i...
A Hugging Face repository named DeepSeek-V4-Flash-Vision-Exp is circulating on Reddit without technical details. The name suggests an efficient, experimental vision-language model, but without VRAM, quantization, or license. For those evaluating self...
LLMFan46 has published GGUF variants of LongCat-Flash-Lite-Sparse, Qwen3.8-27B, and Qwen3.5-122B-A10B, plus Qwen3-Coder-Next and Laguna-S2.1 with optional vision. The need for a personal llama.cpp fork to run sparse attention and a 1M token context h...
Support for RadixArk/Qwen3.8-Flash-Next-NVFP4 in SGLang-V100 shows four V100 32GB running full 256k-token context with over 50GB offloaded to system RAM. Prefill around 4,000 tokens/s and decoding around 60 tokens/s. MTP data signal throughput gains ...
A Reddit setup shows a 27B dense model at 47-50 tokens/s with a 100,000-token context on a 16 GB RTX 4070 Ti SUPER. The signal is not the model but fine-grained runtime control: asymmetric kvarn5/kvarn4 KV cache, a 1,024-token full-precision tail, an...
A three-and-a-half-hour test on a 128GB MacBook Pro M5 Max ran a 79GB 2-bit Qwen3.8-Flash-Next with a 358,400-token context slot. Prefill fell from 1,561 to 318 tokens/s; generation from 30–35 to 11.5 tokens/s at 169K. After 100K tokens the model mix...
A documented Reddit setup runs a Qwen3.8-27B on an RTX 4070 Ti SUPER at 47-50 tokens/s with a 100,000-token context window. The credit goes to the beellama.cpp runtime and the asymmetric kvarn5/kvarn4 KV cache, which freed about 6% VRAM. A precision ...
Z.ai uses the same base model as GLM-5.2 for GLM-5.3, but all gains come from post-training. Coding improves by 50% on the internal Code Bench and reaches state of the art on Terminal Bench 3.0 and Agents' Last Exam. The real shift is the emergent cy...
llama.cpp has merged support for Qwen3.8-Flash-Next and a community report shows 55 tokens/s with a Q4 GGUF on four RTX 3090s. An informal test that shifts attention from software to hardware: for local inference, multi-GPU cost and complexity remain...
The Apodex team's AMA on r/LocalLLaMA introduces Apodex 1.1, an open model family for agentic work, plus an open-source harness and two papers. Quantized variants in FP8, GPTQ-Int4, and NVFP4 point to local, self-hosted inference, but no benchmark fi...
The Apodex team has presented Apodex 1.1, an open model family designed for complex work involving reasoning, search, code execution, and multi-agent coordination. Alongside the models, the team released FrontierAgent and two papers. The NVFP4, GPTQ-...
A Mac Studio M4 Max with 128GB of unified memory hosted the test of Qwen3.8-Flash-Next, the first model this year to break 94% on tolitius's personal benchmark. The 27B model excels at coding but loses in general knowledge to Gemma 31B and Qwen 3.6 o...
The Hugging Face page for zai-org/GLM-5.3-Flash signals a new model, but no technical details. For self-hosted deployments, the Flash label suggests inference efficiency while missing VRAM, quantization, and token details complicate planning. We anal...
The Qwen team has released Qwen3.8-Flash-Next, a 125B open-weight model with 6B active parameters and a native 262,144-token context. The architecture combines Qwen Sparse Attention, Gated Residual, and a 51B n-gram embedding to reduce long-context l...
IBM has released Granite 4.2, three open-weight LLMs with 3, 8 and 30 billion parameters designed for download and self-hosting. All provide a native 128,000-token context window. The 8B and 30B variants add an agentic reinforcement learning block fo...
A Qwen3.8-27B checkpoint in NVFP4 shows that W4A4 quantization guided by distillation can bring a 27-billion-parameter LLM below 20 GB without significant benchmark loss. AI-Radar reads the result as a shift in VRAM and TCO thresholds, but also as a ...
A fully quantized NVFP4 Qwen3.8-27B checkpoint drops to 19.7 GB from 55.6 GB in BF16 while keeping near-identical scores on GPQA-Diamond and AIME26. The team used quantization-aware distillation with the QUASAR algorithm and supports vLLM on Blackwel...
