Microsoft Research has released Fara1.5-27B, a computer use agent for web browsers that marks a paradigm shift from the usual DOM- or API-based automation. The model makes decisions by observing only screenshots of the page, without ever accessing the accessibility tree or the underlying code. Actions like clicking, typing, scrolling, or visiting URLs are emitted with precise pixel coordinates and structured arguments, while the task history and internal reasoning flow in text form.

Fara1.5-27B is the result of supervised fine-tuning on Qwen3.5-27B using data generated by FaraGen1.5, a multi-agent pipeline that first synthesizes web tasks, then executes the trajectories needed to solve them, and finally verifies the results before including them in the training set. The whole system was co-designed with MagenticLite, the recommended deployment framework for both research and production.

The choice of purely visual perception brings both flexibility and well-defined risks. Without ties to the DOM, the model can operate on any rendered page, including applications that expose no programmatic interfaces. Yet that is precisely where the limitations highlighted by the researchers lie: deceptive renderings, prompt injections hidden in visible content, and ambiguity in interpreting the user interface can derail the agent. In a multi-step trajectory, a single wrong click can trigger cascading errors that are hard to recover from.

For those evaluating on-premise deployment, Fara1.5-27B’s profile is an interesting test of sovereignty and control. The model, with its 27 billion parameters, requires GPUs with adequate VRAM (likely at least one A100 80GB or multiple cards in parallel), and the MagenticLite inference pipeline further increases the computational load. But there is a structural advantage: all data – screenshots, actions, history – stays within the corporate perimeter, with nothing flowing to external clouds. It’s the classic trade-off that AI-RADAR analyzes in its frameworks for local deployment: greater data sovereignty in exchange for upfront hardware investment and in-house infrastructure skills.

Microsoft is careful to note that the model is English-only and not recommended for high-stakes domains (legal, healthcare, finance) or for commercial use without further safeguards. Despite these caveats, Fara1.5-27B sends a clear signal: major players are pushing visual agents from the lab to tools nearly ready for controlled environments. The fact that smaller variants (4B and 9B) are already available suggests a strategy to cover different hardware tiers, from on-premise servers down to more modest workstations.

It remains to be seen how much the exclusive reliance on vision will make these agents vulnerable to visual attacks and how the computational cost will weigh against hybrid approaches that combine DOM and vision. Meanwhile, for those automating repetitive web processes in corporate settings, the availability of a model that promises to “see the browser like a human” adds another arrow to the automation quiver – with all due caution.