A Reddit post crystallized in a few lines the frustration that hits anyone trying to set up a local LLM stack for coding: the user described configuring OpenCode, an AI coding assistant, with Ollama as the backend, but the model — a Gemma variant — would only respond with single words (“Hello”, “I”, “4”), likely just one token per reply. In some cases, lowering the context limit triggered an infinite compaction loop. The culprit, as the user himself suspects, is the context window.

The dynamic is instructive. When Ollama runs a model via ollama run, it adopts a default context of 4,096 tokens. OpenCode, on the other hand, allows a much larger limit in its configuration file — in this case 32,732 tokens for context and 8,024 for output. The problem arises because that value does not automatically propagate to the Ollama server: the OpenAI‑compatible API exposes a max_tokens or context parameter, but unless the Ollama runtime has been instructed with an environment variable (OLLAMA_CONTEXT_LENGTH) or by editing the systemd unit file, the model clips everything beyond 4k. And when a client like OpenCode expects to converse in a 32k window but the server serves only an eighth of that, the mismatch produces bizarre symptoms: generation truncated to a single token (as if the buffer were empty) or the compaction loop the system tries to use to compensate for the insufficient context.

The user tried setting the parameter through the canonical methods — environment variables, overriding the systemd service — without success. This touches a raw nerve of self-hosting. There is no single configuration file governing the entire stack, but a layering of mechanisms: parameters passed at model launch, the Ollama server configuration, the operating system unit file, and finally the client-side settings of OpenCode. The context thus becomes fragmented data, whose coherence is left to the user and their debugging skills.

This story is much more than a bug report. It is a structural signal about the maturity of local inference tools. While cloud providers hide these complexities behind consistent APIs, those who choose the on-premise route for data sovereignty find themselves having to wear the system administrator’s hat, solving mismatches that can halt a workflow for hours. The promise of a “private code assistant, under your control” collides with an adoption curve that penalizes those without an infrastructure background. And so, paradoxically, the quest for independence from the cloud may push developers back to external APIs just to regain productivity.

It is not merely a tooling issue. The incident reveals a gap in the local AI ecosystem: the lack of standardized negotiation of the context window between client and inference server. If every front-end (OpenCode, Continue, Aider) must adapt to the specific mechanisms of each runtime (Ollama, llama.cpp, vLLM), the market for on-premise coding assistants will remain fragmented and reserved for a technical niche. Conversely, robust interoperability on this mundane yet critical parameter could unlock much wider adoption. For those evaluating a self-hosted deployment today, analytical frameworks — such as those offered by AI-RADAR at /llm-onpremise — can map these trade-offs and anticipate friction points. The alternative is to remain trapped in a configuration loop that, just like the misaligned model, produces only a single token at a time.