When a Reddit user upgraded their local stack from GLM 5.1 to 5.2, they hit a wall: responding to a math problem took over 12 hours on an old Xeon server, forcing them to shut the model down. The cause? Reasoning tokens more than doubled, from 16.7k to 36.7k. On dated, GPU-less hardware, that leap turned inference into an interminable wait.

The surprise came when the same user studied the GLM development team's technical report. A graph showed that setting the model to "high level" – instead of maximum effort – reduces token usage to less than half, while coding task performance stays at 98% of the maximum. In other words, the extra 2% of intelligence costs more than twice the tokens, a luxury many on-premise environments cannot afford.

Reasoning tokens: the hidden fuel

Reasoning tokens are those the model generates internally, invisible to the end user, to structure a solution. Models like GLM produce them in growing quantities to boost accuracy, but each token requires compute, time, and energy. On old CPUs without massive GPU parallelism, the cost becomes prohibitive.

The graph in the report reveals a sweet spot: the performance curve flattens quickly, showing that a high token count adds diminishing returns. This is a familiar dynamic for anyone running on-premise inference: beyond a threshold, the marginal improvement doesn't justify the computational explosion.

The self-hosting lesson

The case of the user on an old Xeon is emblematic: a model's default setting can determine whether local deployment is even feasible. If defaults are tuned for industrial servers bristling with GPUs, smaller operators risk being locked out without knowing alternatives exist.

For those evaluating on-premise deployments, trade-offs among precision, latency, and total cost of ownership (TCO) aren't always spelled out in official docs. The ability to adjust reasoning effort – from "high level" to maximum – should be part of every self-hosted toolkit. It's not just about lowering quality; it's about finding the point where invested resources yield the best operational return.

AI-RADAR provides analytical frameworks to navigate these choices, linking hardware specs, workload types, and latency requirements. Often, a "downgraded" configuration delivers acceptable response times without sacrificing practical usefulness.

Beyond the default

The Reddit user's experience suggests model developers should better communicate the impact of different reasoning levels and, ideally, tailor defaults to detected hardware. Until then, finding the optimal setting remains an empirical exercise, typical of an on-premise ecosystem that is young but evolving fast.

The next time inference grinds to a halt on a local server, the solution may not be a more powerful GPU, but a simple change to the reasoning parameter. In a landscape where AI is often equated with million-dollar investments, the pragmatism of a "high level" that delivers almost everything with far less is a breath of optimism for those who choose to keep control.