Performance Analysis of Qwen3.5-27b with Quantization

A Reddit user shared the results of a benchmark conducted on the Qwen3.5-27b model, comparing different combinations of model weights (bf16, fp8) and KV cache (bf16, fp8). The Aider benchmark was run 10 times for each configuration on a workstation equipped with an Nvidia RTX 6000 Pro GPU.

The main goal was to evaluate the impact of quantization on model performance, particularly for agentic coding applications. The results indicate that the observed variance between the different configurations is not statistically significant. This suggests that, at least in the context of the Aider benchmark, using a specific quantization may not result in a significant degradation of performance.

Test Setup Details

  • Model: Qwen3.5-27b
  • Quantization: bf16, fp8
  • Benchmark: Aider (224 tasks, approximately 13300 tokens per task)
  • Hardware: Nvidia RTX 6000 Pro (600W)
  • Software: vLLM in Podman container (Linux)

The user specified that they used vLLM within a Podman container on Linux, with a 600W Nvidia RTX 6000 Pro GPU. The Aider benchmark was run in a separate Podman container.