Cua – Open-Source Computer-Use Agent Infrastructure

TL;DR

TL;DR: Cua is an open-source TypeScript/Python framework that gives AI agents the ability to control full desktop operating systems — macOS, Linux, and Windows — through a unified sandbox, SDK, and benchmark toolkit.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: cua.ai
• Source repository: github.com/trycua/cua
• License: MIT (verified via GitHub API license.spdx_id)
• HN launch thread: news.ycombinator.com/item?id=46670181 (Lume 0.2 launch, 154 points)
• Docs: cua.ai/docs
• Source last checked: 2026-06-25 (commit main branch, pushed_at: 2026-06-24T05:40:09Z)

What Is Cua?

Cua (pronounced “coo-ah”) is an open-source infrastructure project for building, benchmarking, and deploying AI agents that control entire desktop operating systems. Rather than calling APIs, a Cua-powered agent can see a screen, click buttons, type into fields, and complete multi-step tasks inside a real OS environment.

From the project README:

Build agents that see screens, click buttons, and complete tasks autonomously. One API for any VM or container image — cloud or local.

The repository is organized into four main components:

| Component | Description | |---|---| | Cua Driver | Background computer-use agent for macOS, Windows, and Linux. Drives native desktop apps without stealing cursor or focus. | | Cua | Agent-ready sandboxes for any OS. Provides a Python SDK (pip install cua) with a unified API for Linux, macOS, Windows, and Android. | | Cua-Bench | Evaluation benchmarks on OSWorld, ScreenSpot, Windows Arena, and custom tasks. | | Lume | macOS/Linux VM management on Apple Silicon using Apple’s Virtualization.Framework. |

Setup Workflow

Prerequisites

• Python 3.11 or later (for the Cua Python SDK)
• macOS, Linux, or Windows host
• For Lume (macOS VMs on Apple Silicon): macOS with Virtualization.Framework support

Install the Cua Python SDK

pip install cua

Run an Ephemeral Linux Sandbox

from cua import Sandbox, Image

async with Sandbox.ephemeral(Image.linux()) as sb:
    result = await sb.shell.run("echo hello")
    screenshot = await sb.screenshot()
    await sb.automation.click("Submit")

The same code switches to .macos() or .windows() by changing Image.linux() — no other API calls change.

Install Cua Drivers (macOS/Linux background agent)

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"

This installs the background driver that lets agents control the desktop without grabbing the mouse or keyboard focus.

Install Lume (macOS VM management)

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"

Run a macOS VM

lume run macos-sequoia-vanilla:latest

Lume supports unattended setup — the VM configures itself from IPSW without user interaction.

Deeper Analysis

Why it matters for AI agent development:

The “computer-use” capability — having an AI actually operate a desktop OS rather than just call APIs — is a key frontier in agent research. Cua makes this reproducible and platform-agnostic.

• Unified API across OSes: Write once, run on Linux/macOS/Windows/Android.
• Background operation: Cua Drivers run without stealing cursor or focus, so the host machine stays usable.
• Benchmarking built-in: Cua-Bench integrates OSWorld, ScreenSpot, and Windows Arena for standardized evaluation.
• Apple Silicon support: Lume uses Apple’s own Virtualization.Framework for near-native performance on M-series chips.

Architecture:

Cua separates concerns into layers. The sandbox layer (Cua) handles VM/container lifecycle. The driver layer (Cua Driver) handles OS-level input simulation (click, type, screenshot). The benchmark layer (Cua-Bench) handles evaluation and RL environment management. Each layer is independently usable.

Practical Evaluation Checklist

• [ ] pip install cua succeeds on a clean Python 3.11+ environment
• [ ] Sandbox.ephemeral(Image.linux()) launches a container or VM within 60 seconds
• [ ] sb.screenshot() returns a usable image
• [ ] sb.shell.run("echo test") executes and returns output
• [ ] Cua Driver installs on macOS without privilege escalation (unless required)
• [ ] lume run macos-sequoia-vanilla:latest works on Apple Silicon hardware
• [ ] Cua-Bench cb image create linux-docker runs without Docker daemon errors
• [ ] Benchmarks (OSWorld, ScreenSpot) load and execute test tasks

Security Notes

• Sandboxes isolate agent activity from the host system — do not disable container/VM boundaries in untrusted workloads.
• Cua Driver runs as a background process with OS-level input simulation privileges — grant only to trusted agent processes.
• The MIT license permits commercial use with no attribution requirement beyond the LICENSE file.
• Third-party components include Kasm (MIT), OmniParser (CC-BY-4.0), and optionally ultralytics (AGPL-3.0) when installing cua-agent[omni].

FAQ

Q: Does Cua require a cloud VM or can it run locally? A: Both. Cua sandboxes can run as local containers/VMs via the Python SDK, or connect to cloud VM runtimes. Lume runs entirely locally on Apple Silicon using Apple’s Virtualization.Framework.

Q: Can multiple agents share one Cua sandbox? A: Each Sandbox.ephemeral() call creates an isolated instance. Sharing within a sandbox would require coordination at the agent level — Cua does not handle multi-agent session management directly.

Q: What is the difference between Cua and Cua Driver? A: Cua (the main package) provides the sandbox API and Python SDK. Cua Driver is the background macOS/Windows/Linux agent that handles low-level input simulation (clicking, typing, screenshots) within a live desktop session. Cua Driver is what lets agents operate a real desktop without grabbing the mouse or keyboard focus of the user.

Q: Is Cua production-ready? A: The project is open-source and actively maintained (18K+ GitHub stars, pushed as recently as June 2026). Linux and macOS backends are stable; Windows and Android are listed as supported but may have narrower platform coverage. Check the docs for the latest platform status.

Q: Does Cua work with Claude Code, Cursor, or other AI coding agents? A: Yes — the HN launch specifically mentions Claude Code, Cursor, Codex, and OpenClaw as compatible clients. Cua Driver exposes an MCP server that these agents can connect to for computer-use capabilities.

Conclusion

Cua fills a specific gap in the AI agent ecosystem: making computer-use reliable, multi-platform, and benchmarkable. Whether you are evaluating models on OSWorld, running a Cursor agent in a macOS VM, or building an automation pipeline that spans Linux and Windows, Cua’s unified sandbox API and driver layer remove the OS-specific plumbing that usually makes these setups fragile.

If you are building agents that need to operate a full desktop — not just call REST APIs — Cua is worth evaluating. The install is a single pip install cua away, and the cross-platform story (macOS, Linux, Windows, Android) is genuinely competitive with single-OS alternatives.

Start at cua.ai/docs and pick the component that matches your use case: the Python SDK for sandboxed agents, the Driver for background desktop control, Cua-Bench for evaluation, or Lume for Apple Silicon VM management.