OrchestriAI
Back to the field guide
AI systemsComparison9 min read

Claude Code computer use vs OpenClaw and Hermes

Claude Code computer-use MCP controls your Mac with per-app approval and tiers, compared with self-hosted OpenClaw and Hermes agents.

Want Claude Code controlling Mac apps with per-app approval?

Free. Three short fields, and we reply by email within one business day.

By Shariq Riaz

In this guide

Claude Code computer-use MCP controls your Mac with per-app approval and tiers, compared with self-hosted OpenClaw and Hermes agents.

11 sections5 cited sources9 min read
On this page11 sections
  1. Short answer
  2. What Claude Code computer use is
  3. Claude picks the narrowest tool first
  4. How permission works
  5. What happens on screen
  6. The trust boundary is your own desktop
  7. OpenClaw and Hermes run a different kind of job
  8. Side by side
  9. When you want both
  10. How this fits implementation work
  11. Independence and limitations

This post covers Anthropic's built-in Claude Code `computer-use` MCP: an interactive coding session on your Mac where Claude sees approved apps and acts with per-app tiers. It is not an always-on OpenClaw or Hermes Gateway in chat channels, not Cua Driver as a client-agnostic background window driver (Hermes bundles Cua as `computer_use`), not LUCI recall-only memory, and not Cursor Self-Hosted where Cursor's cloud loop drives a registered worker.

Sources for Claude Code are Let Claude use your computer from the CLI and the Desktop computer use section. The OpenClaw and Hermes side reuses OpenClaw vs Hermes Agent and setup guides citing the OpenClaw docs and Hermes docs.

Short answer

Use Claude Code computer use when a developer is sitting at the Mac and needs the agent to check a native app, a simulator, or a GUI-only tool during a coding session. Use OpenClaw or Hermes when the agent has to run as a service on a host you administer, answer in chat channels, run on a schedule, or serve more than one person. If an API or MCP tool exists for the job, give the agent that tool first. Both setups are better off with screen control as the last resort.

What Claude Code computer use is

The CLI version has firm limits, and Anthropic states them up front:

  • It is a research preview.
  • The CLI version is macOS only. The Desktop app version runs on macOS and Windows.
  • It needs a Pro or Max plan. It is not available on Team or Enterprise plans.
  • It needs an interactive session, so it does not work in non-interactive mode with the `-p` flag.
  • You must sign in through claude.ai. It is not available when you reach Claude through Amazon Bedrock, Google Cloud, or Microsoft Foundry.

To turn it on, you run `/mcp` in a Claude Code session, find `computer-use` in the server list, and choose Enable. The setting is saved per project. The first time Claude tries to act, macOS asks you to grant Accessibility (to click, type, and scroll) and Screen Recording (to see the screen).

Claude picks the narrowest tool first

Anthropic calls computer use the broadest and slowest way to work with an app, so Claude tries other paths first. If you have an MCP server for the service, Claude uses it. A shell command goes to Bash. Browser work goes to Claude in Chrome if you have it set up. Screen control is left for native apps, simulators, and tools that have no API.

We made the same case in when an agent uses the browser instead of the MCP tool. A typed tool call is easier to check, log, and retry than a click on a screenshot. Anthropic has built that order into the product.

We'll set up computer use for your tasks and approval tiers, or a self-hosted agent if you need it running unattended.

How permission works

Turning on the server does not give Claude every app. The first time it needs an app in a session, the terminal shows which apps it wants, any extra access such as the clipboard, and how many other apps will be hidden while it works. You choose Allow for this session or Deny. Approvals end with the session.

Control is also capped by app type. The Desktop docs list three tiers that you cannot change:

TierWhat Claude can doApplies to
View onlySee the app in screenshotsBrowsers, trading platforms
Click onlyClick and scroll, but not type or use shortcutsTerminals, IDEs
Full controlClick, type, drag, and use shortcutsEverything else

Some apps get an extra warning before you approve them. Terminals and IDEs are flagged as equal to shell access. Finder is flagged because it can read or write any file. System Settings is flagged because it can change the system. They are not blocked. The warning is there so you decide whether the task needs that much access.

What happens on screen

  • One session at a time. A session takes a lock on its first computer-use action and keeps it until the session exits. A second session gets an error that names the session holding the lock.
  • Other apps are hidden. While Claude works, apps you did not approve are hidden and come back when the turn ends. In the CLI this is always on. The Desktop app lets you turn it off and keep a denied apps list, which the CLI does not have yet.
  • Your terminal stays out of the picture. The terminal window stays visible to you but is left out of the screenshots, so Claude never sees its own output.
  • You can stop it. A macOS notification says Claude is using your computer. Pressing `Esc` anywhere stops the current action, and the key press is consumed so text on screen cannot use it to dismiss a dialog. `Ctrl+C` in the terminal also works.
  • Screenshots are downscaled before they go to the model. Anthropic's example is a 16-inch MacBook Pro at 3456×2234 sent at about 1372×887. If text is too small to read, make it larger in the app.

The trust boundary is your own desktop

Anthropic's warning is direct. The sandboxed Bash tool in Claude Code isolates filesystem and network access. Computer use does not. It runs on your real desktop with whatever apps you approve. Claude checks each action and flags possible prompt injection from content on screen, but anything visible in an approved app can still try to steer it.

So the questions to settle before you approve an app are about your own machine. What is signed in inside that app? What could a page, email, or document on screen ask the agent to do? Would you approve Finder or a terminal for this task if a stranger were typing? Anthropic points to its computer use safety guide for practices.

OpenClaw and Hermes run a different kind of job

OpenClaw is a Gateway you run. It connects model providers, sessions, tools, messaging channels, the Control UI, and optional device nodes. Team use is configuration of that same Gateway. Sandboxing is off by default. When you turn it on, tool execution can move to Docker, Podman, SSH, OpenShell, or Crabbox while the Gateway stays on the host.

Hermes Agent is a process you run from a CLI, a messaging Gateway, an API server, or a supported editor. Tools come in toolsets you select, profiles keep roles apart, and terminal commands can run locally, in Docker, over SSH, or in several hosted sandboxes. Dangerous commands go through an approvals setting (`smart`, `manual`, or `off`) with a hardline blocklist that stays on underneath.

Both can schedule work, answer in chat, and keep running when nobody is at the keyboard. Claude Code computer use needs a person in an interactive session on that Mac, and the docs give you `Esc` to stop it at any moment.

Side by side

QuestionClaude Code computer useOpenClaw or Hermes
Where does it act?Your own Mac desktop (Windows too in the Desktop app)The host and sandboxes you configure
Who is present?A developer in an interactive sessionOften nobody: channels, schedules, API calls
How is access granted?Per app, per session, with fixed control tiersTool policy, toolsets, allowlists, pairing, and approvals in config
How many at once?One session holds the screen lockAs many sessions as the Gateway or process is set up to serve
What is the main risk?On-screen content steering an agent inside apps you approvedA broad tool or a weak sandbox on a host you run
Who maintains it?Anthropic ships the server; you approve appsYou install, patch, and audit the runtime

When you want both

The two fit together well. A developer can use Claude Code computer use to build and click-test the GUI for an internal tool on a Mac. The work that runs every day, such as filing tickets, syncing records, or answering a chat channel, belongs in a self-hosted agent with narrow MCP tools and clear approvals. See trust boundaries for self-hosted agents and permissions and human approval for that side.

If the job needs screen control without a person watching, look at products built around a separate machine. Our posts on Cursor Self-Hosted Machines, Muse Secure VM, and Simular Sai cover three ways vendors split who owns that computer. For enterprise EX Automation that remediates IT/HR/Ops on the employee desktop inside user RBAC, see Yellow.ai Nexus EDGE.

How this fits implementation work

OrchestriAI does not operate Claude Code, OpenClaw, or Hermes for clients. MCP development builds the typed tool that lets an agent skip the screen. AI agent systems decide which jobs an agent may run and where a person approves. Systems integration connects those tools to the systems that hold the data.

Independence and limitations

OrchestriAI is an independent implementation provider. Product names are used for identification only. OrchestriAI is not affiliated with, endorsed by, or sponsored by Anthropic, OpenClaw, or Nous Research / Hermes. This article summarizes Anthropic's Claude Code computer use page and the Desktop computer use section as checked on October 6, 2026, plus our existing OpenClaw and Hermes guides. Computer use is a research preview, so plans, platforms, and controls can change. Confirm the current docs before you enable it on a machine that holds client data. This is not a security certification or a compliance conclusion.

Free. Three fields. We reply within one business day.

Want Claude Code controlling Mac apps with per-app approval?

We'll set up computer use for your tasks and approval tiers, or a self-hosted agent if you need it running unattended.

You'll talk to Shariq. Replies come from shariq@orchestriai.com. Leave out passwords and patient records.

What happens next

  1. 1.You send one line about the task.
  2. 2.Shariq reads every message and replies by email within one business day.
  3. 3.If it fits, you get a written plan or a short call. Your choice.
  4. 4.If it does not fit, we tell you and point you to a better option.
“Shariq built our Open Dental to GoHighLevel sync and our AI routing system for new patient inquiries. Both projects were delivered fast, worked right away, and he was responsive whenever we needed adjustments. He understands dental practice workflows and doesn't overcomplicate things.”
Michael Ling, Owner, Smile Again Dental

References used in this article

5 links
Shariq Riaz, founder of OrchestriAI

Written by

Shariq Riaz

Founder, OrchestriAI · CPHIMS · PMP · CBAP

11 years in enterprise IT at Fortune 500 companies. He founded OrchestriAI, an AI automation agency that builds and runs automations and AI agents for healthcare, real estate, financial services, and freight forwarding teams.

Free automation checklist

Free automation checklist for your industry

A one-page checklist of the manual work clinics, real estate teams, financial firms and freight forwarders most often automate. Pick your industry and tick what is still done by hand.

Or read it now, no email needed: Healthcare & dental automation checklist.

Your email comes only to us. No spam, and we never sell it. See the Privacy Policy.

Want this built for you? AI Agents: From $4,000/project

When your process has branches, decisions, and exceptions that need judgment, a linear automation breaks down. We build multi-agent systems where specialized AI agents handle different parts of a complex workflow, coordinating with each other like a well-run team.

Not sure it fits yet? Send us one manual task and we'll email back one specific way to automate it, free.

Keep reading