Qwen-family and other Chinese open-weight models are driving local self-hosting, with users reporting API-free setups and quantization-aware tuning. Download and deployment data show these models are becoming the default choice for many local LLM workloads.
A consultant built ImaJev-4b, a 4-billion-parameter model based on Qwen3.5-4B with LoRA and a small decision head. After an initial fine-tuning that degraded reasoning, careful data curation led to first place on JevBench among 91 models and third on...
A test on 50 MATH-500 benchmark questions shows that penalizing hesitation tokens via logit bias improves accuracy for Qwen3.5-4B in GGUF format across all quantization levels tested. BF16 goes from 74% to 84%, while Q2_K doubles from 12% to 24%. The...
A user fed Opus 5.5 motion graphics examples to Qwen 27B and produced a local version on a single RTX 4090. The project was built with the Accuretta framework and shows that self-hosted creative pipelines are starting to cover tasks once reserved for...
A user reports that a Qwen 3.8 Flash Next model on dual RTX 3090s outperformed GPT-5.6 Luna on a real project after being connected to Codex CLI. The result reignites the debate about which layer matters most in local deployment: not just the model, ...
A llama.cpp Q8_0 benchmark shows both fine-tunes cut median completion tokens by about 40% compared with Qwen3.8-27B. Pass rates remain similar, but Swift delivers a more uniform reduction, while ThinkingCap has a longer tail and different language-l...
A 12GB RTX 5070 runs Qwen3.8-Flash-Next at 65.1 tokens/s with a custom inference engine. The Strata project shows how a specialized runtime can beat llama.cpp on the same numerical precision, lifting prompt processing to 543 tokens/s on a 128K contex...
A user documents running Qwen-3.8-27B locally on a Raspberry Pi with Q4_K_S quantization and Q8_0 context. Costs: 2.4 cents per 1M input tokens and 70 cents per 1M output tokens, comparable to the cheapest providers. The experience highlights the pot...
Nathan Lambert's written Congressional testimony captures a structural shift: since August 2025, Chinese open-weight models have recorded 3.2 billion Hugging Face downloads versus 1.6 billion American ones. On OpenRouter, weekly open model traffic gr...
