Skip to content

Add Codex desktop catalog integration tests - #1226

Merged
andy-xu-db merged 3 commits into
mainfrom
andy-xu-db/codex-desktop-e2e
Oct 22, 2026
Merged

andy-xu-db merged 3 commits into
mainfrom
andy-xu-db/codex-desktop-e2e

Conversation

@andy-xu-db

@andy-xu-db andy-xu-db commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

What

Adds black-box integration tests for how ug sets up the Codex desktop app, modeled on the Claude Desktop lifecycle test (tests/integration/test_ug_configure_claude_desktop.py, #1072). The desktop app reads the shared ~/.codex/config.toml, where ug points model_catalog_json at its stable catalog ~/.ucode/codex-model-catalog.json.

New file tests/integration/test_ug_configure_codex_desktop.py (markers managed_fixture and codex, like the other managed Codex tests; fixture managed_workspace_default):

  1. test_ug_configure_codex_desktop_repeat_configure: the user already has settings in ~/.codex/config.toml.
    • The first ug configure adds only the catalog pointer, keeps the user's settings, and prints the codex app-server daemon restart notice.
    • A repeat configure leaves config.toml and the catalog byte-identical and prints no restart notice.
  2. test_ug_configure_codex_desktop_keeps_user_catalog: the user already points the desktop app at their own catalog.
    • Configure warns (custom model catalog) and changes neither file.
    • Revert leaves both untouched.

Already covered elsewhere, so not repeated: the catalog's model list and a bare codex app-server model/list (test_ug_configure_managed_codex), and revert removing ug's pointer and catalog while keeping user settings (test_case_02_managed_codex_uses_admin_discovery_after_configure).

Not covered

  • This PR changes tests and docs only. ug revert leaves ~/.ucode/codex-model-catalog.json behind when the user keeps their own model_catalog_json pointer. That is not fixed here, so the user-catalog test does not assert it.
  • Desktop tracing and its login service: tests/AGENTS.md keeps tracing out of this suite, and CI runners have no systemd user bus.
  • The desktop GUI and inference.

Docs

  • tests/integration/README.md: file-tree entry and a Codex desktop coverage paragraph.
  • tests/README.md: the "Desktop Codex app" row now names this coverage.

Tests

  • The first revision's two tests passed live in CI on Linux (Full journeys · Codex · Linux and Managed config · Codex). The tests now run only in the Linux managed-config lane; the Windows failure (invalid TOML from a backslash path) is also fixed by JSON-quoting the path.
  • uv run pytest tests/test_integration_contract.py: 80 passed; the new file collects 2 tests. ruff check and format are clean.

This pull request and its description were written by Isaac.

Cover the shared ~/.codex/config.toml catalog pointer that the Codex
desktop app reads: configure twice beside user settings, bare
codex app-server model/list, unchanged repeat, revert, and a
user-owned catalog pointer that must survive.

Co-authored-by: Isaac <no-reply@databricks.com>
@andy-xu-db

Copy link
Copy Markdown
Collaborator Author

/ug-review

Comment thread tests/integration/test_ug_configure_codex_desktop.py Outdated
Comment thread tests/integration/test_ug_configure_codex_desktop.py
@github-actions

Copy link
Copy Markdown
Contributor

Polly AI Review

Blocking issues

  • Windows test failure — tests/integration/test_ug_configure_codex_desktop.py:69: can we encode the path with json.dumps(str(user_catalog))? Windows backslashes produce invalid TOML, so the selected Windows case fails before checking catalog preservation.

    The module’s live marker selects it in the Windows/Codex job. An offline TOML parse reproduced the failure; JSON encoding round-tripped the path correctly.

Security vulnerabilities

None identified in the changed code.

Non-blocking notes

  • tests/integration/test_ug_configure_codex_desktop.py:54: can we compare restored bytes with the original snapshot? Parsed TOML equality misses lost comments or formatting despite the exact-restoration claim.
  • tests/integration/test_ug_configure_codex_desktop.py:11: is full-lane execution intentional? live selects both cases in Linux/Windows full jobs as well as the managed lane.

Approach

The installed-CLI configure → repeat configure → revert journey is appropriate for checking command wiring and filesystem ownership. Existing session and managed-config primitives avoid new scaffolding.

Summary

Diff size (computed): 3 files; +87 / -1 lines (88 changed text lines); 0 binary files.

Category Files Added Deleted Share of changed text lines
Tests and test support 3 87 1 100.0%
Documentation 0 0 0 0.0%
Dependencies and lockfiles 0 0 0 0.0%
Implementation / other 0 0 0 0.0%

Shares use additions + deletions, including generated text and lockfiles. Binary files have no line count. Tests include fixtures/helpers under test paths and colocated test/spec files; categories are path-based, not coverage measurements.

Changes

Adds shared-config lifecycle and custom-catalog preservation coverage, plus coverage documentation. No production code, dependencies, or generated assets change. The size is proportionate; no unnecessary scaffolding was found.

Tests

Test-by-test assessment
Test / case Behavior protected Layer Needed? Action / rationale
tests/integration/test_ug_configure_codex_desktop.py::test_ug_configure_codex_desktop_catalog_lifecycle Existing settings survive; stable catalog pointer; initial restart notice; byte-identical repeat without notice; revert cleanup Installed-CLI integration keep Adds repeat-run and complete lifecycle assertions beyond existing catalog/model-list coverage. Compare original and restored bytes to establish exact restoration.
tests/integration/test_ug_configure_codex_desktop.py::test_ug_configure_codex_desktop_keeps_user_catalog Configure warns without changing a user-owned pointer/catalog; revert preserves both Installed-CLI integration keep Exercises a distinct ownership boundary through public commands. Encode the path safely so the scenario works on Windows.
tests/README.md:710 — documentation only Records shared-config coverage while excluding desktop GUI, tracing, and inference Documentation keep Updates the coverage inventory without claiming desktop execution.
tests/integration/README.md:29 — coverage paragraph; file-tree entry at line 467 Documents the new scenarios and their limits Documentation keep Makes the integration coverage discoverable and describes its intended boundary.

Existing tests cover catalog contents, bare app-server model/list, malformed shared configuration, and catalog-validation failures. No additional live failure matrix is warranted. The concrete assertion gap is exact-byte restoration, noted above.

No pytest, collection, lint, or live suite was independently executed. The reported results remain unverified. Only the synthetic Windows-path TOML failure and encoding fix were executed offline; the corrected lifecycle cases still need Linux and Windows integration verification.

Scope

All three changed files support the stated catalog-lifecycle coverage outcome. No unrelated changes were found. The broader CI selection caused by live needs clarification, but is not established as unrelated work.


Automated review by Polly · workflow run

- Quote the user catalog path as JSON so the TOML stays valid on Windows.
- Drop the live marker, matching the other managed-fixture Codex tests.
- Drop the revert half of the repeat-configure test; the managed model
  discovery test already covers it.
- ug revert now deletes ug's stable catalog even when the user's own
  model_catalog_json pointer is kept, and the user-catalog test checks it.

Co-authored-by: Isaac <no-reply@databricks.com>
@andy-xu-db andy-xu-db changed the title Add Codex desktop catalog lifecycle integration tests Add Codex desktop catalog integration tests; revert deletes ug's catalog beside a user pointer Oct 21, 2026
Drop the ug revert change, its unit test, and the leftover-catalog
assertion; this PR only adds integration tests.

Co-authored-by: Isaac <no-reply@databricks.com>
@andy-xu-db andy-xu-db changed the title Add Codex desktop catalog integration tests; revert deletes ug's catalog beside a user pointer Add Codex desktop catalog integration tests Oct 21, 2026
@andy-xu-db
andy-xu-db requested a review from masonc08 October 21, 2026 22:01
@andy-xu-db
andy-xu-db added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 73081d9 Oct 22, 2026
105 of 108 checks passed
@andy-xu-db
andy-xu-db deleted the andy-xu-db/codex-desktop-e2e branch October 22, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants