The Reddit post starts with an unusual admission: the author has stopped using APIs to rely on Qwen-3.8-27B running locally. It is not an ideological choice, but the result of a bumpy path. At first the model 'thought too much', making it frustrating to watch while it worked. The solution was to stop watching: let it operate unsupervised. And, surprisingly, the model proved capable of completing complex refactors autonomously, making sensible decisions. Not perfect, but — as the author points out — neither are APIs.

The technical detail is essential: the model runs on a Raspberry Pi, in a Docker container, with Q4_K_S quantization for the weights and context quantized to Q8_0. The agent used is Pi, without MCP and with a minimal set of tools: Bash, read, write and edit. The edit tool is the weak point: the model often has to retry edits because of indentation errors. The author hopes for a more fault-tolerant version, or perhaps will build one himself.

The economic revelation comes from the declared costs: 1 million input tokens cost 2.4 cents, while 1 million output tokens cost 70. This is not a typo: generation is drastically more expensive than ingestion. This fact overturns many common assumptions. In cloud providers, input is often cheaper than output, but not with a 30-fold gap. Here output is 29 times more expensive. The explanation likely lies in energy consumption and hardware wear during generation, which on a low-power device like the Raspberry Pi has a disproportionate impact. Yet the author considers it comparable to the cheapest providers on nano-gpt.com. The point is not whether it is convenient in absolute terms, but for which type of workload it is.

For input-heavy workloads — analysis, classification, information extraction — costs are negligible. But for intensive generative tasks, such as extended code writing or report production, the cost per generated token rises quickly. Those evaluating on-premise deployment should therefore calibrate their needs: it is not enough to compare average cost per token; one must weigh the input/output distribution of their use case. And this changes incentives: if output is expensive, you will tend to design shorter prompts, use the model for brief iterations, or offload final generation to a cloud service when volume is needed.

There is also a second order of implications. The experience with Swift-Qwen — a community variant that is faster but prone to loops — shows that the local ecosystem is fragmented and that speed optimization can compromise reliability. In a context of autonomous agents, an infinite loop is a concrete risk: it can consume resources and produce useless output. The official version, slower, proved more stable. This trade-off between speed and robustness is a central theme for those building local automation pipelines.

Finally, the bottleneck is not only the model, but the tools surrounding it. Pi's edit tool failing on indentation is a trivial but significant example: self-hosting is not just a matter of quantized weights, but of an entire operational stack that must mature. The promise of data sovereignty and independence from providers clashes with the reality of still rough tooling. For those who accept these compromises, however, the advantage is tangible: no rate limits, no egress costs, total control over the workflow. And the ability to leave the model working alone, without watching, is perhaps the most important signal: autonomy is no longer a mirage.