Anthropic is entering the AI agent market with direct control capabilities over local computers. Claude Code and its user-friendly version, Claude Cowork, can now interact with the graphical interface to automate tasks.

These agents can "point, click, and navigate" on the screen, opening files, using the browser, and running dev tools. When possible, Claude Code and Cowork prioritize using Connectors to directly access external apps and data sources. Alternatively, they can request permission to interact directly with the machine.

Direct computer control can be initiated and managed remotely via Claude's Dispatch tool, provided the target computer is powered on. This feature is available to Claude Pro and Max subscribers using MacOS in a "research preview." Anthropic warns that the system may not always work perfectly and may require a second try for complex tasks. Performing tasks through direct computer interaction is slower and more error-prone than using Connectors.

For those evaluating on-premise deployments, there are trade-offs to consider carefully. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these aspects.