A Reddit thread recently brought the on-premise coding model debate into focus. User Jorlen described a stable, self-hosted setup built around 64 GB of total video memory, using an Unsloth version of the Qwen 3.5 122b-a10b model quantized to UD-IQ4_NL, with only a few layers offloaded to system RAM.

According to the post, this configuration sustains a 100,000-token context window in bf16 and delivers around 30 tokens per second. While far from cloud-level throughput, the numbers are more than adequate for an interactive coding assistant that runs entirely on local hardware, with full data sovereignty and no round-trip latency to a remote API.

Why a MoE model changes the game

The model belongs to the mixture-of-experts family: 122 billion total parameters, but only 10 billion are active per token. That sparse activation drastically reduces memory pressure, making it possible to fit a model of this scale within a consumer-grade VRAM envelope when paired with aggressive quantization. The trade-off is some performance penalty when the GPU has to fetch expert weights from CPU memory, but the user reports being “deeply impressed” and plans to make the model their daily driver, occasionally alongside Qwen 3.6 variants.

On-premise coding assistants: from experiment to daily practice

The experiment underscores a broader shift in the on-premise LLM landscape. As MoE models mature, and quantization toolchains like Unsloth simplify deployment, the barrier to running competent coding assistants locally is lower than ever. Developers and small teams can consider ditching cloud subscriptions in favor of a one-time hardware investment, provided they can manage the electricity, cooling, and software stack overhead.

AI-RADAR regularly examines these infrastructure trade-offs, offering analytical frameworks on /llm-onpremise for those evaluating the balance between local control and cloud convenience. In regulated environments, or any setting where source code privacy matters, the ability to keep everything in-house remains a compelling advantage.

A practical sweet spot for 64 GB VRAM

The Qwen 3.5 122b-a10b case suggests that 64 GB of VRAM is not just a threshold for running large models, but a sweet spot for getting real coding work done without compromising on privacy. As the community continues to share their recipes, the on-premise coding assistant is quietly becoming a practical reality.