Two variants of the same model make the same promise: cut the reasoning loops that make Qwen3.8-27B notorious for consuming tokens. ThinkingCap-Qwen3.8-27B and Swift-Qwen3.8-27B state it in their model cards with nearly identical numbers: about 40% fewer reasoning tokens and minimal quality degradation. The benchmark posted on Reddit using the Aider suite and llama.cpp 0.5.0 with Q8_0 quantization, with two runs per model, confirms the claim on the median: both models finish with roughly 5,000 fewer completion tokens than the original Qwen3.8-27B. But the headline number tells only the first part of the story.
The most striking detail is that ThinkingCap and the original Qwen3.8-27B achieve exactly the same first-try and retry pass rates: 27.1% and 77.6%. The author notes that he has never seen two runs of the same model close with identical scores and suggests treating the result as a coincidence. Still, it is the clearest confirmation that the fine-tuning did not sacrifice problem-solving ability: it worked on the cost of reasoning, not on the absolute level of the answers.
Swift sits within the noise margin: it loses two percentage points on the retry pass but records the highest first-try pass rate (30.8%). Its advantage shows up in the token distribution. On median values it uses about 7,300 completion tokens versus 7,436 for ThinkingCap, but above all it does not have the same long tail. In mean values, ThinkingCap uses 8.5% more tokens because in several cases it continues to overthink significantly. Swift achieves a more uniform reduction. The time per case follows the same pattern: Swift closes in 750 seconds, ThinkingCap in 777, while the original model needs 1,481 seconds.
Another element that separates the two models is behavior on failed cases. Both spend more tokens when they fail than when they succeed, but the gap is much wider for Swift: 13.2K tokens on failures versus 5.9K on successes. ThinkingCap stops at 9.4K versus 6.8K. In a self-hosted context, where every extra token translates into GPU time and energy consumption, this difference is far from academic: Swift can burn more compute on difficult cases before giving up, while ThinkingCap tends to stop earlier, which can be read either as an early surrender or as better awareness of its limits.
The real discriminator comes from programming languages. On C++, ThinkingCap loses eight percentage points on the retry pass compared with Swift (61.5% versus 69.2%), while it recovers on JavaScript (85.4% versus 81.2%) and Python (78.8% versus 72.7%). The differences on the other languages included in the Aider suite are not statistically significant. That means teams working mainly with a specific ecosystem may see results that differ from the aggregate number: general-purpose benchmarks risk flattening a profile that is highly context-dependent.
ThinkingCap also has a rare distinction: it is the only one of the three models to produce 100% well-formed diffs, with no malformed replies or output errors. For development pipelines that integrate the model into automated flows, this matters almost as much as accuracy: a malformed diff can break an automatic patching process even when the underlying solution was conceptually correct. The other two models produced a few invalid outputs.
This test signals a broader shift in how local models are evaluated. Pass rates alone are no longer enough: two models with the same capability can have very different TCO depending on median tokens, long tails, and output stability. The 40% reduction stated in the model cards is not a marketing detail but a concrete operational variable for anyone running the model on their own hardware. The llama.cpp benchmark with Q8_0 makes the comparison even more relevant for the self-hosted galaxy, where memory and latency constraints cannot be absorbed by elastic cloud infrastructure.
For teams evaluating which variant to adopt in an on-premise context, this story shows that benchmarks should be sliced by language, token distribution, and diff quality. On /llm-onpremise AI-RADAR offers analytical frameworks to compare these trade-offs. Meanwhile, the author also mentions the release of Swift Flash Next, suggesting that the comparison among Qwen3.8-27B-derived variants is set to continue.
๐ฌ Comments (0)
๐ Log in or register to comment on articles.
No comments yet. Be the first to comment!