Skip to content

feat(codex): add Android runner with managed ChatGPT sign-in - #39

Open
wellorbetter wants to merge 2 commits into
google:mainfrom
wellorbetter:feat/codex-oauth-runner
Open

wellorbetter wants to merge 2 commits into
google:mainfrom
wellorbetter:feat/codex-oauth-runner

Conversation

@wellorbetter

@wellorbetter wellorbetter commented Sep 11, 2026 •

Copy link
Copy Markdown

Summary

ARTEMIS autonomous tasks currently require separately configured model API credentials even when the user is signed into Codex with ChatGPT. This adds an experimental CLI runner that reuses Codex-managed ChatGPT authentication to operate an explicitly selected Android device.

Closes #38.

uv run artemis codex login
uv run artemis codex status
uv run artemis codex run "Open Settings and read the battery percentage" --serial YOUR_DEVICE_SERIAL

Implementation

  • Use the official Codex app-server stdio API for account status, browser/device-code login, and dynamic Android tools. Codex manages credentials; ARTEMIS does not read its auth file or extract tokens.
  • Provide observation, tap, swipe, Unicode typing, and navigation tools with strict argument validation, coordinate bounds, action budgets, and consumed/expiring observation IDs.
  • Acquire the ARTEMIS device lock, record local screenshots/UI trees/action logs, and interrupt unfinished turns and release resources on timeout or cancellation.
  • Create an ephemeral thread with host shell, plugin, and configured MCP capabilities disabled; reject unsupported server requests.
  • Document setup, architecture, and limitations in docs/codex.md.

Validation

Tested on Windows with Python 3.12.14 and Codex CLI 0.153.4. Latest PR head: ca3848b.

Codex integration tests: 33 passed

Full deterministic suite:
Upstream 0860788: 2109 passed, 82 failed, 4 skipped, 8 deselected
This PR:          2142 passed, 82 failed, 4 skipped, 8 deselected
Failed-node comparison: identical 82 failures; no new failed tests

Tests cover real subprocess framing, API-key environment exclusion, account redaction, login success/failure event handling, device argument validation, observation invalidation, action budgets, unsupported requests, timeouts, cancellation, and cleanup.

Live Android validation reused an existing ChatGPT login, completed a Settings/battery task, and matched the result against its screenshot. A deliberate one-second timeout was followed by a successful observation-only task. Cancelling a live async task recorded a terminal trace status and allowed immediate device-lock reacquisition.

Formatting, changed-file Ruff, quality ratchet, protected-core Pyright, and new-module type checks passed. Repository-wide Ruff retains 11 pre-existing playground errors; the full repository suite is not green. Independent test-isolation fixes are in #42.

Scope and remaining validation

This is a separate CLI path; Flash/Pro, Web UI, daemon scheduling, and mobile_run_task retain their existing execution paths. It does not include recording, arbitrary ADB shell, or Pro checkpoint verification. Success is the model's assessment plus a required final observation, not an independent Checker verdict.

Dynamic tools are experimental. Fresh interactive browser/device-code sign-in, physical cable disconnection, other Codex versions, and broader device compatibility remain untested; login protocol paths have offline coverage. Cancellation was tested through the async API rather than a physical Ctrl+C keystroke. The PR remains a draft for these validation limits and maintainer feedback.

Private screenshots, UI XML, and raw device traces remain local. Only sanitized textual test evidence is shared.

@google-cla

google-cla Bot commented Sep 11, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@wellorbetter

Copy link
Copy Markdown
Author

Expanded regression validation

Retested ca3848b on Windows / Python 3.12.14 with the locked development environment. Added 10 permanent regression cases in commit ca3848b:

  • Browser and device-code login: success, failure cancellation, and ignoring completion events belonging to another login.
  • Timeout/cancellation: release the device connection and lock and persist terminal trace status.
  • Failed action: consume its observation and budget, preventing replay against an uncertain screen.
  • Failed observation: invalidate the previous screen before another action.
  • Unsupported approval request: reject and interrupt without invoking Android tools.
pytest -q tests/unit/test_codex_integration.py
33 passed in 0.85s

pytest -q
82 failed, 2142 passed, 4 skipped, 8 deselected, 80 warnings in 93.80s
Failed-node comparison with upstream 0860788: 0 new failures

Formatting, changed-file Ruff, quality ratchet, protected-core Pyright, and git diff --check passed. The full suite retains the same 82 baseline failures; #42 addresses 64 independently.

Live validation (Codex CLI 0.153.4, existing ChatGPT session, Android 16):

codex status: authenticated=true, auth_type=chatgpt
codex login: existing ChatGPT session reused
Settings/battery task: succeeded=true, battery=75%, actions=2, observations=4
Final screenshot manually checked: 75%
Intentional --timeout 1 run: exit=1, trace status=failed, end_time populated
Immediately following observation-only task: succeeded=true, actions=0, observations=1
Cancel live run after thread startup: CancelledError propagated
DeviceExecutionLock acquired immediately afterward: PASS
Cancellation trace: status=cancelled, end_time populated

These are sanitized log excerpts; private screenshots, account details, device serial, and raw device traces remain local. Cancellation was exercised through the async task API, not a physical Ctrl+C keystroke. A Samsung wake-lock inspection warning (Unknown command: set-wakelock) was followed by the existing USB stay-awake fallback succeeding; it did not prevent these runs. Fresh interactive sign-in and physical cable disconnection remain untested. The PR remains a draft for those protocol/compatibility limitations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Android automation with Codex-managed ChatGPT sign-in

1 participant