Fitting a 27-billion-parameter model on a GPU with 16GB of VRAM forces choices that are not neutral. A user's comparison of 21 Qwen3.8 27B variants on an RTX 5080, using C code as the test workload, reveals a far wider gap than the quantization label alone suggests: mean divergence ranges from 0.893 for the 8.2GiB QAT Q2_0 to 0.056 for bartowski's 14.5GiB IQ4_XS.

This is not the usual model ranking: it confirms that, when VRAM is scarce, the choice of a single GGUF file can matter more than the base model choice. The table sorts variants by Mean KLD, a measure of divergence between token distributions where lower is better. Alongside it, Same top p indicates how often the quantized variant selects the same most probable token as the reference. In this case, the two metrics converge: the more aggressive variants pay a clear price in probabilistic coherence.

The best overall result comes from bartowski/Qwen3.8-27B-IQ4_XS, with Mean KLD 0.056482 and 95.835 percent agreement. It is not the lightest file in the batch, but it is where 4-bit compression seems to preserve much of the original behavior without exceeding 16GB. For those seeking a variant without filters, huihui-ai/Huihui-Qwen3.8-27B-abliterated-UD-IQ4_XS follows closely with 0.082871 and 94.981 percent, confirming that abliteration does not necessarily cost quality.

On the opposite end, sdkyuan's QAT Q2_0 records a mean divergence of 0.893 and 85.727 percent agreement on the most probable token: numbers that explain the author's 'underwhelming' assessment. Its 8.2GiB file is also the most compact in the list, but the memory saving translates into a significant loss. Other variants between 7.8 and 9.9GiB show Mean KLD between 0.77 and 0.35: they may remain theoretically useful for non-critical tasks, but the comparison with IQ4_XS is stark.

The two unsloth Q4_K_XL variants are even more instructive. With Mean KLD of 0.029844 and 0.028026 they would top the table, but the 16.4 and 16.7GiB files do not fit on the GPU, as the author notes with 'can't fit'. This is the practical limit of self-hosted deployment: a quantization can be superior on paper and still unusable if the file exceeds available VRAM after accounting for context, system and processes. On a 16GB card, the real operating margin is lower than the nominal size.

The structural lesson for on-premise evaluators is not 'take the largest model that fits', but that the quantization market is now a jungle of labels: QAT, IQ2, IQ3, Q3_K_XL, Q4_K_XL, NVFP4, Ridge, Cold Fusion. Benchmarks on real tasks, such as the C code used here, become the only way to navigate it, because the GGUF name does not say enough about perceived quality. It is no surprise that the author addresses people who are 'VRAM starved': the consumer 16GB GPU tier has become the local AI laboratory, where every gigabyte counts. Analytical frameworks on /llm-onpremise help reason through these trade-offs: there is no absolute best variant, only one that accounts for available VRAM, the type of workload and governance constraints.