GitHub Copilot Computer Use Preview Brings Desktop Control to CLI

Copilot can now operate desktop apps

GitHub has given its Copilot agent a way to work outside the terminal and the code editor. On October 1, 2026, the company released computer use as a public preview in GitHub Copilot CLI and the standalone GitHub Copilot app on macOS and Windows.56 With the feature turned on, Copilot can read an application's accessible content and visual context, click controls, type and edit text, press keys, scroll, drag, and carry a task across several applications.1 GitHub pitches it for legacy and GUI-only software that has no API, command-line interface, or Model Context Protocol (MCP) integration.1

The release came in a busy month for coding agents. OpenAI's Codex CLI, Anthropic's Claude Code, and the open-source OpenCode project all shipped updates in October.3 Copilot's launch examples are deliberately ordinary: summarizing notifications in a browser, updating a presentation, and moving information through a desktop workflow.1 Even so, the change shifts Copilot's position in the race. Before this, the agent could only act on text, files, and shell commands. Now it can reach almost anything a person can see on screen.

How it works and how tightly it is controlled

The feature is off by default. Developers switch it on with /computer on in the CLI, check its state with /computer show, and disable it with /computer off. The Copilot app has a matching toggle under Settings.12 Under the hood it is a bundled computer-use plugin backed by an MCP server. The agent reads the operating system's accessibility tree first and falls back to screenshots only when it needs them.58 When the feature isn't responding, GitHub's troubleshooting steps point users to /plugin and /mcp list to check that both pieces are loaded and connected.6

Permissions run per app. When the active permission mode requires it, Copilot asks before taking control of an application, and the user can allow it for the current session, choose "Always allow," or decline.4 Pressing Escape twice in the CLI stops an action that is already running.4 On macOS, the feature walks users through granting Accessibility and Screen Recording permissions.1 For organizations, a single managed setting, features.computerUse, blocks the feature, and a user's local setting cannot override it.4

The rollout has clear limits. It works only in local sessions on macOS and Windows. Linux is not supported even though the Copilot app runs there, and a cloud sandbox session cannot control apps on a user's own desktop.8 The managed-settings reference lists only the CLI and the Copilot app as supported surfaces. VS Code, the cloud agent, and JetBrains are not on that list.4

GitHub recommends using something else first

The launch is unusually modest about its own feature. GitHub tells developers to use a direct API, MCP server, terminal command, filesystem tool, or browser tool whenever one exists, because those return more structured and predictable results than clicking through a screen.2 One developer outlet summed up the message as advice to try something else first. It also noted that GitHub openly warns about misclicks, stalls, and sensitive on-screen information ending up in the agent's context.7

The details of the approval system deserve attention. An "Always allow" saved in the CLI also applies to the Copilot app on the same machine, so one click covers both.8 Deny rules still take priority over saved approvals.2 Deleting an app from the always-allowed list does not revoke access for a session that is already running.610 Several guides arrive at the same practical advice: decide which apps to permit and how to stop the agent before rolling the feature out, and start with read-only tasks or ones that are easy to undo.410 GitHub's own documentation suggests a starting prompt that tells Copilot to summarize an app's status without changing values or submitting forms.8

On this point the coverage splits. One October roundup said GitHub had not published a full technical breakdown of the screen-interaction layer or detailed guidance on what the feature should and shouldn't touch, which leaves teams to set their own boundaries.3 Other write-ups based on GitHub's documentation describe concrete details: the accessibility-tree-first design, permission modes, deny-rule precedence, and an enterprise kill switch.245 The better-documented reading holds up. GitHub has explained the mechanics and controls in reasonable depth. What it has not published are usage limits, a general-availability date, or a list of compatible apps.38

Catching up to Anthropic and OpenAI

The roundup called Copilot's computer use the most visible single feature of the month, but GitHub was not first.3 Its move follows computer-use features that OpenAI's Codex and Anthropic's Claude Code and Cowork had already shipped.7 In this category, GitHub is following, not leading.

Anthropic has been at this the longest. Computer use first appeared as a beta with Claude 3.5 Sonnet in October 2024.26 On March 23, 2026, Anthropic brought it to Claude Code and Cowork for Pro and Max subscribers, letting Claude point, click, and navigate the screen with no setup.21 Claude Code's changelog later recorded computer use arriving in the CLI itself as a research preview, aimed at confirming changes that only a GUI can show.25 On September 3, Anthropic added background computer use, so Claude can operate Mac apps while the user works on something else. That is still a macOS-only beta.27 On the API side, the computer use tool came out of beta on August 19 as a new toolset that supports batching several actions in one turn.22 On October 7, Anthropic's Python and TypeScript SDKs added beta classes that run the tool loop and approval callback for developers who build their own desktop automation.22

OpenAI's timeline is just as full. The Codex changelog describes GPT-5.4, released in March, as the first general-purpose Codex model with native computer use.16 Computer use reached macOS apps in April, and Windows followed on May 29.16 At DevDay on September 29, OpenAI added computer use to its Agents API, letting developers build agents that operate software directly.18 That API version runs a browser in an OpenAI-hosted sandbox and requires the app to approve each new website origin.15 OpenAI engineers say the capability only became practical once it was fast enough that users stopped noticing when an agent switched from a direct integration to clicking through an interface.12

Set against these timelines, GitHub's distinguishing features are breadth of platform and depth of governance, not novelty. It launched on Windows and macOS at the same time. Anthropic's desktop computer use has been described as macOS-only.2327 GitHub's per-app approvals, saved-permission rules, and enterprise policy key also suit companies that already manage Copilot centrally.48

Anthropic matters inside Copilot too

Anthropic shows up on both sides of this story. Claude Code competes with Copilot CLI, yet Anthropic models also run inside Copilot. The same October cycle that brought computer use to Copilot also made Claude Haiku 5.5 generally available there, alongside a wider shuffle of models within Copilot.3 Claude Code itself shipped version 2.1.296 this month with model pinning for workflow agents and context compaction controlled by subagents. Codex CLI 0.162.0 added managed Git worktrees and pinned tasks for parallel work.3

This looks like a market where the same capabilities appear across vendors within months and leadership is measured in quarters. The real differences are now in permissions, platform support, and how well each agent knows when not to click.

What to watch

The direction is clear even though the details are still settling. All three major vendors now treat computer use as a fallback for when structured tools don't exist, not as the main way an agent should work. GitHub says so in its documentation.2 OpenAI's Agents API pairs computer use with ordinary tool calling so agents can use an API when one is available.20 Anthropic's SDK toolsets put URL policies and approval callbacks in the developer's hands.22

For engineering leaders, the immediate choice is whether to allow the feature, not whether it works. One analysis argues the policy decision should be made before a developer first clicks "Always allow," since users can grant OS-level permissions themselves before any admin acts.4 That is the right order. Computer use is useful for the long tail of GUI-only software, but it is still a preview that can misread a screen. For now it belongs on bounded tasks that can be undone.