Skip to content

feat(drivers): codex-computer-use, OpenAI's Codex Computer Use as a macOS driver (FD-5356) - #550

Merged
earakely-scale merged 5 commits into
mainfrom
edgararakelyan/fd-5356-cu2-codex-computer-use-driver
Nov 9, 2026
Merged

earakely-scale merged 5 commits into
mainfrom
edgararakelyan/fd-5356-cu2-codex-computer-use-driver

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

OpenAI's Codex Computer Use becomes a computer driver on macOS: codex-computer-use, in the drivers bundle beside agent-desktop and macos-harness. It's the ChatGPT app's unified-computer-use plugin. The plugin's MCP server, cua_repl, runs the JavaScript an agent sends against OpenAI's cua API, which reads apps' accessibility trees and acts on them in the background through the plugin's helper app.

  • Install ([install.macos]). ChatGPT 26.930.51102 is fetched from OpenAI's versioned URL (persistent.oaistatic.com/codex-app-prod/ChatGPT-darwin-arm64-<version>.zip), checked by sha256.

    • install.sh unpacks it beside the driver and never runs it, so it never updates itself.
    • It copies the plugin's helper app to /opt/agentenv/codex-computer-use/. That path doesn't depend on the driver folder's name, so the layer's grant can name it.
    • The helper app and the driver's Python both get Accessibility, Post Event and Screen Recording.
  • One cua_repl per agent session. The driver starts cua_repl as the app would, on the app's own Node with the environment the app builds, with only the computer surface on. Each agent session gets its own server, since the REPL keeps its bindings between calls. A server that stops is started afresh on the next call. Nothing says when a session ends, so a server unused for 30 minutes, or past the eight kept, is closed, the least recently used first.

  • Tools. Agents get the server's instructions and its js and js_reset, pinned in mcp.json. js_add_node_module_dir and turn_ended, which it serves only the ChatGPT app, are hidden. The computer_* tools come from MacOSDriver.

  • Approvals. Codex asks its host before each action on an app, as an MCP elicitation. The driver answers each one:

    • it approves the apps the allowed_apps setting names, by name or bundle identifier, case-insensitively, as the request's metadata gives them (the message's quoted name too, but a reworded message doesn't decline every app);
    • it approves every app when the setting is unset;
    • it declines anything else, including recording the computer's audio, and logs a request that names no app as a warning.
  • Sandbox. Each call carries Codex's managed read-only sandbox, unless js_sandbox = false:

    • everything is readable;
    • only ~/.codex and the temp folders are writable;
    • there's no network.

    The apps it drives are outside the sandbox, so Computer Use still works. Reads and commands still run, so a bench that wants GUI-only use needs its own guard.

  • No ChatGPT sign-in. It ran without one, and we chose to run it that way. Whether running Computer Use without a ChatGPT Work or Codex sign-in fits OpenAI's terms is an open question.

  • computer-ready checks:

    • the executables' sha256;
    • that cua_repl gives the instructions and lists the tools mcp.json pins, every field it pins;
    • that it answers cua.getState();
    • the TCC permissions.

Testing

  • Unit: tst/unit/drivers/codex_computer_use_driver_test.py, 10 tests, 9 of them against a fake cua_repl that sends approval requests shaped like Codex's real ones. They cover:

    • the forwarded and hidden tools;
    • the sandbox _meta;
    • approvals by name and by bundle id, from the metadata of a reworded request or the message of one without metadata, every app when unset, audio declined with a warning;
    • one REPL per session and a stopped one restarting, calls a session makes together sharing one, and idle and least recently used ones closed;
    • cua_repl's environment;
    • readiness, which flags changed instructions, descriptions or annotations;
    • that env.toml, install.sh and driver.py agree on the pinned version and the helper's path.

    The env, bundle and drivers suites pass locally, apart from three tests that fail the same way on main on macOS: two in cua_driver_test.py, and the Windows ISO test in test_disk_recipe.py. ruff check . is clean.

  • On an EC2 Mac host with lume, through agent-env run:

    • Compose: onto macos-26 in 4 min 6 s.
    • Gate 3's full probe: 53 passed, 0 failed, 0 unverified, 1 not applicable.
    • Deploy: deploy_env with allowed_apps = ["Calculator"] passed the driver's readiness checks.
    • An env_outcome_verifier driving the deployed driver over MCP, as an agent would, passed all 5 checks:
      • only js and js_reset of Codex's tools listed;
      • Calculator opened through js;
      • 7 × 6 shown as 42;
      • TextEdit refused ("Computer Use was not approved to use TextEdit");
      • a computer/v1 screenshot.

Docs

The site's computers or drivers page should gain codex-computer-use:

  • what it is;
  • macOS only;
  • its allowed_apps and js_sandbox settings, passed in deploy_env's driver_settings;
  • that no sign-in is needed;
  • that js can still read files and run commands, so a bench guards for GUI-only use.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new blocking issue was found.

Summary

Adds codex-computer-use as a macOS driver with app approvals, sandbox settings, pinned downloads, and readiness checks.

  • Agents can use Codex Computer Use through a separate JavaScript session for each agent.
  • App actions follow the driver’s allowlist, and JavaScript calls carry Codex’s sandbox rules.
  • MacOS arm64 deployments can install and select the pinned Codex Computer Use driver.

Reviews (10) · Last reviewed commit: "fix(drivers): a codex-computer-use call ..." · Reviewed by Greptile

…acOS driver (FD-5356)

The ChatGPT app's unified-computer-use plugin as a computer driver on macOS. ChatGPT 26.930.51102, OpenAI's
versioned download pinned by sha256, is unpacked beside the driver and never run; its helper app is copied to a fixed
path and granted Accessibility, Post Event and Screen Recording. The driver starts the plugin's cua_repl as the app
would, one server per agent session, and forwards its js and js_reset, hiding the two tools it serves only the
ChatGPT app. It answers Codex's per-app approvals from the allowed_apps setting, by name or bundle identifier (every
app when unset, audio recording never), and runs each call under Codex's managed read-only sandbox unless js_sandbox
is off. No ChatGPT sign-in is needed. computer/v1 is MacOSDriver's.

On a Mac host with lume, it composes onto macos-26 and passes gate 3's full probe (53 passed, 1 not applicable); a
verifier driving the deployed driver over MCP opened Calculator, computed 7 x 6 = 42, was refused TextEdit, which
allowed_apps didn't name, and took a computer/v1 screenshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner November 9, 2026 04:15
…s read-only

Nothing tells the driver when an agent session ends, so each session's cua_repl stayed open for the driver's life:
a REPL left unused for 30 minutes is now closed by a sweep, and past eight the least recently used is closed when a
new one starts, never one in a call, so active sessions keep their bindings. js keeps the schema and description
Codex serves, but its hints say it changes apps and reaches the world (readOnlyHint false, destructiveHint and
openWorldHint true), where Codex marks it read-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 3 commits November 8, 2026 20:31
…-use REPL

Closing other sessions' REPLs awaited between finding the caller's REPL and putting it back, so a second call from
the same session made a REPL of its own and the first call then replaced it. Eviction now takes REPLs out at once and
closes them in the background, so nothing awaits before a call is queued.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…data and checks its pins agree

- Approvals take the app's bundle identifier and name from the request's metadata, and the name its message quotes
  only beside them, so a reworded message no longer declines every app; a request that names no app is a warning.
- computer-ready compares cua_repl's instructions and every field mcp.json pins for its tools, not only their input
  schemas.
- The app's paths are relative to APP, install.sh names the helper's folder once, the README no longer repeats the
  version, and a unit test checks that env.toml, install.sh and driver.py agree on the version and the helper's path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arts a new one

A stopped server was still ending when the session's next call came, so the call joined its queue and failed with
the error it ended on, as on Linux CI. Each server now has its own queue, let go as soon as it takes no more calls,
so the next call starts a server afresh; close ends every server still ending. The test slows the old server's end
so the next call always arrives meanwhile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit b5e7235 into main Nov 9, 2026
114 of 117 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/fd-5356-cu2-codex-computer-use-driver branch November 9, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant