Agents often pick computer-use or browser automation when a purpose-built MCP tool already exists. That swap replaces a typed, auditable, permission-scoped tools/call with a session that inherits a human's cookies, screen context, and brittle UI selectors. It also bypasses the MCP approval, schema, and audit path you built for that intent.
This is a common failure mode in OpenClaw, Hermes, MCP-connected agents, and Grok Bot-class setups: the model can see a logged-in UI, so it clicks through instead of calling the tool that was designed for the job. Adjacent patterns such as Claude desktop computer-use and other browser agents raise the same surface choice. Browsers belong in narrow visual or API-less jobs; using them as a silent substitute for MCP is the wrong surface.
For the permission and approval design that MCP tools are meant to use, see how to scope AI agent permissions and human approval and MCP tool permissions that survive retries and remaps. For what MCP servers are and how to harden them, see what MCP servers are and how to secure an MCP server for production.
What "wrong surface" means
An MCP tool call is a named operation with a schema, server-side authorization, and a durable record of tool name plus arguments. Computer-use / browser automation drives a DOM or desktop session: screenshots, clicks, typed text, and whatever cookies or tokens the session already holds.
Wrong surface means the agent chose the browser (or a generic computer-use tool) for an intent that already has an MCP tool, or that should have one before you grant session access. Right surface means the agent calls the MCP tool for that intent, and only opens a browser when the job truly requires rendered UI or has no API.
The surfaces are not interchangeable. One is a contract. The other is a shared human workspace with none of the same guarantees.
Why agents pick the browser
Several conditions push the model toward computer-use even when MCP is available:
- The MCP tool is missing, misnamed, or not listed in the tools the runtime exposes for this run.
- The prompt describes the goal in UI language ("open the dashboard and update the record") instead of naming the tool or the business operation.
- The model prefers visible UI: screenshots look like progress, while a tools/call looks like a black box.
- MCP auth is broken, expired, or scoped too narrowly, so the tool fails and the agent falls back to "just click through."
- Discovery is weak: dozens of tools, vague descriptions, or no route map from intent to tool name.
- Operators normalize browser fallbacks in playbooks, so the agent learns that UI automation is an acceptable substitute.
In stacks that mix MCP servers with computer-use, those fallbacks become the default under pressure. A timeout on tools/call should trigger reconcile and retry of the same tool, not a silent switch to the operator's logged-in browser. See the retry and remap controls for why remapping to a wider surface is a new proposal, not a retry.
What you lose when the browser substitutes for MCP
When the agent drives a session instead of calling the tool, you typically lose:
- Typed arguments and schema validation. Clicks do not carry the same argument contract as tools/call.
- Server-side authorization on a service identity. The browser often uses the operator's full session cookies and roles.
- Exact-args approval binding. A human who approved a narrow MCP payload did not approve an unbounded UI walk.
- Idempotency and reconcile hooks. UI steps rarely have durable keys or provider reconcile before a second attempt.
- Durable audit of tool name plus arguments. Screenshot trails are harder to prove and harder to replay.
- Least privilege. A purpose-built MCP tool can hold a narrow token. A browser session usually can do everything the human can do.
That is why the wrong surface is a security and operations problem, not only a reliability annoyance. You built MCP so side effects would go through a controlled boundary. Browser substitution undoes that boundary without a new design review.
When the browser is the right surface
Keep browser / computer-use narrow. Prefer it when:
- the admin UI has no API and no MCP server, and the change is a one-off;
- you need visual QA or to verify rendered state a human must see;
- the site or product has no MCP server and you are not ready to build one;
- a human-in-the-loop step deliberately reviews the live UI before a side effect.
Even then, isolate the session, prefer a least-privilege account, log that the surface was browser, and do not silently reuse that path for workflows that already have MCP tools. Desktop computer-use products and browser agents are useful for those narrow cases. They are a poor default for every CRM update, ticket write, or payment action that already has a typed tool.
Decision table: MCP, browser, or refuse
| Scenario | Wrong surface risk | Prefer |
|---|---|---|
| Intent maps to an existing MCP tool (CRM update, ticket create, search) | Browser inherits operator session; skips schema and approval binding | MCP tools/call with exact-args approval |
| MCP tool failed once (timeout, auth blip) | Agent opens UI "to finish the job" | Retry / reconcile the same MCP tool; do not remap to browser |
| One-off admin change, no API or MCP server | Over-building MCP for a single click | Narrow browser session with elevated approval and audit of surface choice |
| Visual QA or verify rendered layout | Automating clicks as if they were the business write | Browser for observation; MCP (or API) for the write if one exists |
| Money move, bulk delete, or message send with MCP tool available | Computer-use with operator cookies | MCP only; refuse browser for that intent |
| No MCP tool and impact is high (money, PII export, production config) | "Just click through" under operator identity | Refuse both until a scoped tool or human runbook exists |
| Tool exists but is undiscoverable / poorly described | Model never sees the tool and invents UI steps | Fix discovery and route maps; keep browser denied for that intent |
Prefer MCP when the intent is known, the tool exists or can be built once, and you need typed args, authz, approval, and audit. Prefer browser when the job is visual, one-off, or genuinely API-less. Refuse both when the impact is high and you have neither a scoped tool nor an explicit human procedure.
Controls that keep agents on the right surface
Put the preference in policy and tests, not only in prompts.
- 1Tool allowlists that prefer MCP. For golden workflows, expose the MCP tools that cover the intent and omit computer-use from the default tool set for that run.
- 2Deny or elevate computer-use when MCP covers the intent. If a route map says "update_crm_contact → mcp:crm.update_contact", a browser attempt for that intent should require elevated approval or fail closed.
- 3Discovery hygiene. Keep tool names and descriptions specific. Remove dead tools. Avoid a giant undifferentiated tool list that buries the right call.
- 4Route maps (intent → MCP tool). Maintain a small table operators and tests can read: business intent, MCP server, tool name, approval class. Agents and reviewers both use it.
- 5Golden-path tests. For critical workflows, assert the agent calls the MCP tool and does not open the browser. Treat a browser call on those paths as a test failure. Guidance on testing AI agents before production applies directly here.
- 6Log surface choice. Every side-effecting step should record whether it used MCP tools/call or browser/computer-use, plus tool name or URL, identity, and approval id. Without that field, you cannot prove which boundary ran.
These controls belong in the runtime and the MCP server, the same way production MCP security keeps authorization on the server rather than in model instructions alone.
How this fits the rest of the stack
Wrong-surface failures sit next to permission design and retry binding. A browser remap is a wider surface, so it must not inherit an MCP approval. Self-hosted agent checklists should treat computer-use as a separate capability with its own allowlist and identity, not as a free fallback when MCP is awkward.
For implementation work, custom MCP server development builds the typed tools that should own known intents, AI agent systems design the runtime allowlists and approval gates, and systems integration covers the APIs those tools call so the browser is not the integration layer.
Design rule
For every known business intent, name the preferred surface in a route map: MCP tool when one exists or should exist, browser only when the job is visual or API-less, and refuse when impact is high and neither path is scoped. Agents that open a session for an intent that already has an MCP tool are on the wrong surface by definition.
