Local LLMs Challenge Cloud Giants in Code Generation

Interest in locally deployed Large Language Models (LLMs) continues to grow, driven by the need for data control, reduced operational costs, and lower latencies. While cloud-based "frontier" models often dominate headlines for their general capabilities, the question of how smaller, optimized models perform on local hardware for specific tasks remains crucial for technical decision-makers. A recent experiment conducted by a community user precisely tested this scenario, comparing the performance of various Qwen 3.6 variants running locally with some of the most advanced LLMs available via the web.

The objective was to evaluate these models' ability to generate complex HTML code for a specific animation: a realistic simulation of a moving car with a parallax scrolling background. The results offered unexpected insights, suggesting that local models can not only compete but, in some cases, exceed expectations even against more renowned solutions for dense, targeted coding tasks.

Technical Details of the Comparison and Methodology

The experiment utilized a detailed prompt, requiring the models to produce a single HTML file with a full-page canvas, without the aid of external libraries. The request included simulating a side-view moving car, with a continuously scrolling background landscape to create a sense of depth through layers moving at different speeds (nearby ground, roadside elements, trees, poles, distant hills or mountains). Realistic spinning wheels, subtle car body motion to simulate connection to the road, a varied and smoothly repeating environment, cinematic lighting (sunset, dusk, or daylight), and an overall calm, immersive, realistic animation with a seamless loop were also required.

The tested models fell into two categories. "Frontier" models included Claude Sonnet 4.6, Gemini 3.1 Pro, GPT 5.4, and Kimi k2.6, all accessed via the web through a Perplexity subscription. For local models, the hardware used was a modest configuration: a Ryzen 5 5600 processor, 24 GB of DDR4-3200 RAM, and an RX 5700 XT GPU with 8GB of VRAM. On this platform, Qwen3.5 9B Q4_K_M (~50 tok/s), Qwen3.6-27B (Claude-opus-reasoning-distilled) Q4_K_M (2.65 tok/s), Qwen3.6-27B Q4_K_M (2.70 tok/s), Qwen3.6-31B A3B Q4_K_M (12.13 tok/s), Gemma-4-31b-it (1.91 tok/s), and two variants of Qwen3.5 4B (Q8 at 60 tok/s and Q4_K_M at 80 tok/s) were run, some of which used the internet for reasoning.

Unexpected Results and Implications for On-Premise Deployment

The evaluation of the results was subjective, focusing on the visual quality of the generated animation: realism of the side-view, layered parallax effect, wheel and chassis motion, cohesive sky and lighting, and animation loop fluidity, all implemented in pure JavaScript/canvas. The Kimi k2.6 model achieved the best overall result, producing the visually cleanest animation. However, the real surprise came from the second-place finisher: the local Qwen3.6-27B Q4_K_M model. This model demonstrated unexpected strength, generating an excellent parallax effect and a realistic road feel, surpassing some of the "frontier" models' outputs in motion quality and layering. Another local variant, Qwen3.6-27B Claude-opus-reasoning-distilled, ranked third, confirming the strong performance of local Qwen models.

This outcome is particularly relevant for organizations considering on-premise LLM deployment. It demonstrates that for specific and complex coding tasks, smaller, quantized models running on accessible hardware can offer competitive performance. This paves the way for solutions that ensure greater data sovereignty, control over Total Cost of Ownership (TCO), and the ability to operate in air-gapped environments without relying on external cloud services. For organizations evaluating on-premise LLM deployment, these results underscore the importance of testing specific models and hardware configurations for their workloads. AI-RADAR offers analytical frameworks on /llm-onpremise to support the evaluation of trade-offs between self-hosted and cloud solutions.

Future Prospects and Concluding Remarks

While this evaluation is subjective and focuses on a very specific coding "primitive," the results suggest a significant evolution in the capabilities of local LLMs. The ability of a 27-billion-parameter model, quantized and running on an 8GB consumer GPU, to compete with or even outperform some frontier models for a complex visual task, is an important indicator. It highlights how model optimization and hardware efficiency are making self-hosted deployments increasingly viable for a wide range of enterprise applications.

The community is encouraged to replicate these tests on different hardware configurations and with other model variants, including those based on MoE (Mixture of Experts) architectures or further distillations. The continuous evolution of LLMs and optimization techniques promises to further expand the spectrum of applications where on-premise solutions can offer an optimal balance of performance, cost, and control.