GitHub Launched New Computer Use Capabilities
The platform has introduced public preview features allowing Copilot agents to interact with desktop applications.
Updated on Oct. 2, 2026 in Artificial Intelligence
Live Poll
Would you trust an AI agent to control applications and sensitive data on your computer?
GitHub has launched computer use capabilities for its Copilot CLI and desktop app in public preview. The tool enables agents to interact directly with macOS and Windows desktop applications by reading content, clicking, and typing.
Why it matters
This feature allows artificial intelligence to automate tasks in applications that lack formal APIs or command-line interfaces. GitHub notes that while helpful, developers should prefer direct tools when available for more structured results.
The tool interacts with applications by reading the operating system accessibility tree and analyzing screenshots for visual context. macOS users must grant specific Accessibility and Screen Recording permissions for the agent to function.
The players
GitHub
This subsidiary of Microsoft provides a platform for software development, hosting, and AI-powered coding assistance tools.
OpenAI
This artificial intelligence research organization develops the underlying models that power many modern coding and automation agents.
The details
The agents can perform physical interface actions such as scrolling, dragging, and typing directly within desktop software. GitHub warns that the agents may occasionally stall or repeat actions when encountering dynamic interfaces or timing changes.
Timeline
April 2026: OpenAI added computer use to Codex.
October 1, 2026: GitHub launched the computer use feature in public preview.
October 22, 2026: New enterprise policy for unconfigured features takes effect.
The Tech Race
This release follows the integration pattern established by OpenAI's Codex computer use capabilities. It marks a broader industry pivot toward agents that can operate across fragmented desktop software ecosystems rather than relying strictly on structured API calls.
Users can now automate workflows in applications that previously required manual input, though they must manage local operating system permissions. Enterprise owners retain the ability to block these features or override developer settings to ensure corporate security.
The takeaway
The implementation of agent-based desktop control signifies that software automation is expanding beyond terminal commands into graphical user interfaces. Developers should carefully test these agents against complex workflows to avoid unintended repetitions or errors in dynamic desktop environments.
What happens next
Enterprise policies regarding unconfigured features will begin enforcement on October 22, 2026.
Further reading
For additional context on evolving automation tools, visit the Artificial Intelligence section.
Live Poll
Would you trust an AI agent to control applications and sensitive data on your computer?
