The On-Premise LLM Challenge: MiniMax-M2.7 vs Qwen3.5-122B-A10B

The Large Language Model (LLM) landscape continues to evolve rapidly, with growing interest in on-premise deployment solutions. This trend is driven by the need to ensure data sovereignty, optimize Total Cost of Ownership (TCO), and maintain full control over infrastructure. For CTOs, DevOps leads, and infrastructure architects, selecting the right model and hardware for local AI workloads presents a complex challenge, where technical specifications and real-world performance are crucial.

In this context, a comparative analysis evaluated two prominent LLMs, MiniMax-M2.7 and Qwen3.5-122B-A10B, assessing their performance on hardware configurations featuring 96GB of VRAM. The objective was to determine which model offered the best balance of inference speed, output quality, and additional features for a fully local deployment, with a specific focus on code generation.

Technical Details and Testing Methodology

The testing platform used for this evaluation consisted of a system equipped with two NVIDIA A6000 GPUs, each with 48GB of VRAM, totaling 96GB of available video memory. This setup allowed for a "full offload" of the models, loading them entirely onto VRAM to maximize inference performance. The specific models tested were ubergarm/MiniMax-M2.7-GGUF in the IQ2_KS version, with a size of 69.800 GiB (2.622 BPW), and ubergarm/Qwen3.5-122B-A10B-GGUF in the IQ5_KS version, with a size of 77.341 GiB (5.441 BPW).

Evaluations were conducted using a Python client based on EvalPlus for the humaneval benchmark, which includes 164 code generation problems. Both models were run via ik_llama.cpp llama-server. The results showed that Qwen3.5-122B-A10B achieved a pass@1 (base) score of 0.494 and pass@1 (base+extra) of 0.482, completing the evaluation in 31 minutes and 20 seconds. MiniMax-M2.7 IQ2_KS recorded a pass@1 (base) and pass@1 (base+extra) of 0.220, with an evaluation time of 32 minutes and 48 seconds. In terms of general inference speed, measured with llama-sweep-bench, Qwen3.5-122B-A10B demonstrated superior performance.

Features and Deployment Implications

Beyond quantitative benchmarks, the analysis also considered aspects related to "quality of life" and specific model features. MiniMax-M2.7 supports a form of self-speculative-decoding, a technique that can improve generation speed, although it requires a heavily quantized kv-cache to accommodate extended contexts (up to 160k tokens). This need for kv-cache quantization can introduce trade-offs in precision or management complexity.

In contrast, Qwen3.5-122B-A10B stands out for its mmproj support for image processing, making it a multimodal model. Furthermore, it offers the ability to use a full unquantized kv-cache up to 256k tokens, a significant advantage for applications requiring very large contexts without sacrificing precision. These characteristics make Qwen3.5 a more versatile choice for scenarios beyond simple text generation, including visual data analysis in an on-premise environment.

Outlook for Local AI Infrastructure

The results of this comparison suggest that, for on-premise configurations with 96GB of VRAM, Qwen3.5-122B-A10B positions itself as a more performant and versatile choice compared to MiniMax-M2.7. Its superiority in code generation benchmarks, coupled with support for a larger, unquantized kv-cache and multimodal capabilities, makes it particularly attractive for enterprises seeking robust and flexible solutions for their local AI workloads.

The final decision on which LLM to adopt will always depend on specific use case requirements, budget constraints, and priorities regarding data sovereignty and compliance. However, this analysis provides concrete data for decision-makers evaluating on-premise deployment options, emphasizing the importance of testing models on real hardware and considering the full spectrum of features and trade-offs. AI-RADAR continues to monitor the evolution of these Frameworks and models, providing detailed analyses to support strategic choices in AI infrastructure.