Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -707,7 +707,7 @@ desktop GUI, a real `launchctl` or `schtasks` run, or a login/reboot pass.
| Follow-up turns and conversation resume | Not covered; reopen proves startup, not conversation resume |
| Claude/Codex interactive smart routing | First-prompt routing is covered by the `managed_fixture` banner journeys and CUJ4. The live subagent-only journeys spawn native children and invoke the installed Smart Router skill to verify routing on -> off -> on within one Claude/Codex session. The scheduled harness-latest workflow repeats CUJ4, routing hooks, skill toggles, and banners with both latest agents. Plugin-refresh survival and native daemon/background dispatch remain deferred. |
| Full allow/deny tool-permission matrix | Not covered; onboarding/trust uses actual TUI choices |
| Desktop Codex app, Isaac itself, auto-upgrades | Not covered by command forwarding or pinned-version tests |
| Desktop Codex app, Isaac itself, auto-upgrades | Not covered by command forwarding or pinned-version tests. Codex desktop's shared-config catalog pointer (repeat configure and a user-owned pointer) is covered by `integration/test_ug_configure_codex_desktop.py`; desktop tracing, GUI, and inference are not |
| Native macOS/Windows live TUI, managed settings, resize/signals | Windows fresh-install, CLI, local helpers, and the Claude headless gateway journey are advisory; Codex Windows CI is blocked on npm proxy access. Live PTY/TUI, managed settings, and signal behavior still need separate platform implementation and coverage |
| Other agents | Live scope is Claude Code and Codex, plus a non-blocking OpenCode headless case |
| OpenCode `--model` / `-m` | Unit/component tests cover raw CLI forwarding, configured model selection, launch arguments, rejection of unknown models before starting the native process, and the existing failure when no default model is available; no dedicated live integration journey |
Expand Down
8 changes: 8 additions & 0 deletions tests/integration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ selection, and an unrelated profile. It does not launch Desktop. The
macOS/Windows GUI, Cowork, model-switching, and browser-authentication work; those GUI and
virtualization prerequisites are not satisfied by the headless suite.

Codex desktop has managed-fixture coverage in `test_ug_configure_codex_desktop.py`. It
configures twice beside user-owned `~/.codex/config.toml` settings and checks the stable catalog
pointer and an unchanged repeat configure with no restart notice; a second case keeps a user-owned
catalog pointer through configure and revert. Revert of ug's pointer is covered by
`test_ug_codex_managed_model_discovery.py`. It does not cover desktop tracing (its systemd/launchd
service needs a user session), the GUI, or inference.

The separate [dedicated-workspace CUJ](../e2e_cuj/AGENTS.md) lives in `tests/e2e_cuj/`
and runs directly with pytest. It does not use this suite's runner or config fixtures.
It reuses the session, terminal, file-task, and transcript helpers in `utils/`.
Expand Down Expand Up @@ -458,6 +465,7 @@ test_ug_codex_app_server.py # actual client/server initialize exchan
test_ug_smart_routing_hooks.py # live hook contract plus skill-driven subagent toggles
test_ug_configure_claude_lifecycle.py # repeat setup, revert, rejected credentials
test_ug_configure_claude_desktop.py # native-path Desktop profile lifecycle, no GUI
test_ug_configure_codex_desktop.py # stubbed managed Codex desktop catalog pointer, no GUI
test_ug_configure_claude_workspace_switch.py # real skills MCP cleanup across two workspaces
test_ug_configure_codex_lifecycle.py # repeat setup, revert, rejected credentials
test_ug_claude_managed_model_discovery.py # Claude MPS policy cases (claude_mps.json)
Expand Down
75 changes: 75 additions & 0 deletions tests/integration/test_ug_configure_codex_desktop.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
"""Black-box lifecycle coverage for the Codex desktop app's shared config and model catalog."""

import hashlib
import json
import tomllib

import pytest
from utils.managed import use_managed_config_fixture
from utils.terminal import TerminalProcess

pytestmark = [pytest.mark.managed_fixture, pytest.mark.codex]

USER_SETTINGS = "# user-owned Codex settings\n[notice]\nhide_rate_limit_model_nudge = true\n"
RESTART_NOTICE = "codex app-server daemon restart"


def _digest(path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()


def test_ug_configure_codex_desktop_repeat_configure(live_session, workspace):
"""Scenario: configure managed Codex twice beside user-owned settings.

Expected: the shared `~/.codex/config.toml` that the desktop app reads gains one pointer to
ug's stable catalog and keeps the user's settings; a repeat configure rewrites neither file
and prints no daemon-restart notice. The catalog's models and a bare `codex app-server`
`model/list` are covered by `test_ug_configure_managed_codex`, and revert by
`test_case_02_managed_codex_uses_admin_discovery_after_configure`. Desktop tracing, GUI
rendering, and inference are not covered.
"""
session = live_session
shared_config = session.home / ".codex" / "config.toml"
catalog_path = session.home / ".ucode" / "codex-model-catalog.json"
# Ordinary pre-existing user settings, not manufactured gateway state.
shared_config.parent.mkdir(parents=True)
shared_config.write_text(USER_SETTINGS)
use_managed_config_fixture(session, "managed_workspace_default")
configure_args = ("configure", "--workspace", workspace, "--skip-upgrade")

first = session.run(*configure_args, timeout=240)
assert RESTART_NOTICE in " ".join(first.stderr.split()), first.stderr
configured = tomllib.loads(shared_config.read_text())
assert configured.pop("model_catalog_json") == str(catalog_path), configured
assert configured == tomllib.loads(USER_SETTINGS)

before = (_digest(shared_config), _digest(catalog_path))
second = session.run(*configure_args, timeout=240)
assert RESTART_NOTICE not in " ".join(second.stderr.split()), second.stderr
assert (_digest(shared_config), _digest(catalog_path)) == before


def test_ug_configure_codex_desktop_keeps_user_catalog(live_session, workspace):
"""Scenario: the user already points the desktop app at their own model catalog.

Expected: configure warns and leaves the user's pointer and catalog file byte-identical, and
revert neither removes the pointer nor deletes the user's file.
"""
session = live_session
shared_config = session.home / ".codex" / "config.toml"
user_catalog = session.home / "my-catalog.json"
# User-owned inputs: a catalog file and the pointer to it. JSON quoting keeps Windows paths
# valid TOML.
user_catalog.write_text(json.dumps({"models": []}))
shared_config.parent.mkdir(parents=True)
shared_config.write_text(f"model_catalog_json = {json.dumps(str(user_catalog))}\n")
original = (shared_config.read_text(), user_catalog.read_text())
use_managed_config_fixture(session, "managed_workspace_default")

result = session.run("configure", "--workspace", workspace, "--skip-upgrade", timeout=240)
assert "custom model catalog" in " ".join(result.stderr.split()), result.stderr
assert (shared_config.read_text(), user_catalog.read_text()) == original

with TerminalProcess(session, "ug", [str(session.binary), "revert"], "desktop-keep") as tty:
tty.finish()
assert (shared_config.read_text(), user_catalog.read_text()) == original
Comment thread
andy-xu-db marked this conversation as resolved.
Loading