The idea of running a JARVIS-like assistant on a gaming GPU is no longer science fiction. The latest proof comes from Reddit user Mrinohk, who documented a software architecture that turns a small Large Language Model into a reliable agent for complex tasks, all on a modest RX 6600 XT.

The central concept is as simple as it is counterintuitive: instead of overwhelming the model with dozens of generic tools, you force it to operate within dedicated “applications” – the user calls them workflows – each with a limited action set, its own clipboard, and a scratch pad to carry information between views. When the agent leaves an application, the context is trimmed and upon return the state is exactly as it was, eliminating the need to rebuild the session every time.

In practice, for web browsing the agent doesn't receive a URL to type – an operation where local models tend to misplace characters – but interacts with a text menu. A verb and a number (“open 1”, “copy 2”) are enough to guide the action without errors. The same applies to computer control: what previously required twenty different tools is now reduced to a few commands inside a simplified interface.

The test was run with Gemma 4 in two sizes: the E4B version (around 4 billion parameters) and the 26B, both quantized to Q4_K_XL using the Unsloth QaT method. Inference runs on llama.cpp with the Vulkan backend, on the aforementioned RX 6600 XT. By the end of the task – finding a rare component for a project car – the model was processing between 70 and 85 tokens per second with a context of about 10k tokens; prefill reached 800 t/s. Quite respectable numbers for consumer hardware without CUDA acceleration.

The biggest surprise, however, was the behavior: the smaller model outperformed its larger sibling. The 26B version showed a certain aversion to dedicated planning tools, while the 4B model, placed in the same application architecture, completed the task more effectively. It’s not the first time we’ve seen that clever agent design can compensate for an LLM's smaller size, but seeing it confirmed on a real-world test adds an important piece for those developing on-premise solutions.

This artisanal discovery hits a sensitive spot for local deployment: large models require expensive GPUs and abundant video memory; small models, when encased in architectures that limit their scope, can become surprisingly performant. It’s not a ready-to-use enterprise solution – the author himself admits to having built only two applications, a text browser and a system controller – but it points in a clear direction: an agent’s quality does not depend solely on the model’s parameters, but on how its action space is engineered.

In a landscape where companies evaluate the Total Cost of Ownership of an on-premise AI stack, experiments like this suggest that investing in design can reduce the need for extreme hardware. The road is still long, but the message is clear: you don’t necessarily need hundreds of billions of parameters to build a useful assistant. Sometimes, just give it good boundaries.