[Bug]: transient code-mode-host process makes catalog staleness check return "unknown", emitting a false "catalog changed after Codex started" restart prompt (2.11.1, Windows)

[mihneaptu]

Client or integration

Codex App

Area

Service lifecycle

Summary

On OpenCodex 2.11.1 (Windows), v2 guidance intermittently emits:

<multi_agent_mode>The model catalog changed after Codex started; do not set model or reasoning_effort overrides until Codex restarts.</multi_agent_mode>

even though the catalog is not stale. In my session this alternated with the normal permissive injectionPrompt guidance from turn to turn, so spawn_agent model overrides were blocked unpredictably. Restarting Codex does not help, because the underlying state is unknown, not stale.

Root cause appears to be that collectCodexAppServerCatalogState() treats a missing process start time as unknown, and the guidance branch in src/server/responses/collaboration.ts maps unknown to the same user-facing text as stale:

// src/server/responses/collaboration.ts:248-252
const catalogState = await (deps.collectCatalogState ?? defaultCollectCatalogState)();
if (catalogState.state === "stale" || catalogState.state === "unknown") {
  return "<multi_agent_mode>The model catalog changed after Codex started; do not set "
    + "model or reasoning_effort overrides until Codex restarts.</multi_agent_mode>";
}

The message asserts a specific cause ("the catalog changed") and prescribes a specific remedy ("restart Codex"), but in the unknown branch neither is established. A restart cannot clear unknown.

Why unknown is reached

collectCodexAppServerCatalogState() (src/codex/app-server-processes.ts) performs two separate PowerShell/CIM queries:

1. listWindowsSnapshots() — enumerate processes + Invoke-CimMethod GetOwner (line 319).
2. readProcessStartMsBatch() — fetch start times via Get-CimInstance Win32_Process -Filter "ProcessId=..." (line 513).

Then:

// line 631
if (catalogMtimeMs === null || withStarts.some(proc => proc.startedAtMs === null)) {
  return { state: "unknown", processes: withStarts, catalogMtimeMs };
}

isCodexAppServerCommandLine() deliberately includes codex-code-mode-host (line 217, via isCodeModeHostProcess). On this machine codex-code-mode-host.exe is transient — it spawns and exits around code-mode tool activity. If such a PID is captured in step 1 but exits before step 2, the ProcessId= filter returns no row, startedAtMs is null, and the whole state collapses to unknown.

The window between the two queries is not small. I timed step 1 ten consecutive times on this machine:

Run   Ms   OK  Incomplete
  1  2781  16           0
  2  2281  16           0
  3  2391  16           0
  4  2815  16           0
  5  2605  16           0
  6  2282  16           0
  7  2257  16           0
  8  2237  16           0
  9  2286  16           0
 10  2335  16           0

So there is a consistent ~2.2-2.8s gap during which a transient PID can disappear. This matches the observed behavior: the false message appeared far more often during bursts of tool calls (when code-mode hosts churn) and was stable when the session was idle.

Evidence that the state is genuinely not stale

ocx doctor disagrees with the injected guidance, in the same session, minutes apart:

[OK] Codex app-server model catalog is current with the on-disk catalog.

Timestamps also rule out staleness under the code's own rule (stale = startedAtMs <= catalogMtimeMs):

Item	Time (local)
.codex\opencodex-catalog.json mtime	13:40:25
codex.exe PID 13300 start	14:13:48
codex.exe PID 5172 start	14:15:34
codex-code-mode-host.exe PID 21368 start	14:20:38

Every app-server process started after the catalog was written, so stale is impossible here. Config files were likewise untouched after process start:

File	Last write
.codex\opencodex-catalog.json	13:40:25
.codex\config.toml	14:10:01
.opencodex\config.json	11:14:29

Functionally, the permissive path works: when guidance was in its normal state, spawn_agent with model="gpt-5.6-sol", reasoning_effort="medium", fork_turns="none" ran to completion successfully. So this is a false negative in detection, not a real allowlist mismatch.

Reproduction

1. Windows 11, OpenCodex 2.11.1, Codex CLI 0.148.0-alpha.5, subagentModels configured with gpt-5.6-sol / gpt-5.6-terra.
2. Ensure the on-disk catalog mtime is older than every running app-server process (i.e. a genuinely fresh state), and confirm with ocx doctor → [OK] ... catalog is current.
3. Run a v2 session that issues frequent tool calls, so codex-code-mode-host.exe processes spawn and exit repeatedly.
4. Observe the injected guidance across turns: it intermittently flips to the "catalog changed after Codex started" text and back to the configured injectionPrompt.
5. Restarting Codex does not stop the recurrence.

Suggested fix

• Do not collapse unknown into the stale message. Either emit a distinct, accurate text (e.g. "could not verify the model catalog state") or omit the positive roster without prescribing a restart that cannot help.
• Treat a PID that vanishes between enumeration and start-time lookup as process exited, not as an unreadable start time — drop it from consideration rather than poisoning the aggregate state.
• Consider excluding short-lived codex-code-mode-host processes from the staleness aggregate. They are relevant for restart targeting, but their lifetime makes them a poor signal for "does the running app-server hold an old catalog".
• Optionally capture start time in the same query as enumeration, removing the race entirely.

Version

2.11.1

Operating system

Windows 11 Pro, 10.0.26200 (build 26200)

Provider and model

OpenCodex v2 routed subagents; subagentModels = gpt-5.6-sol, gpt-5.6-terra. Codex CLI 0.148.0-alpha.5.

Logs or error output

Injected guidance observed intermittently (roughly five times in one session, interleaved with normal guidance):

<multi_agent_mode>The model catalog changed after Codex started; do not set model or reasoning_effort overrides until Codex restarts.</multi_agent_mode>

Contradicted by, in the same session:

[OK] Codex app-server model catalog is current with the on-disk catalog.

Screenshots and supporting files

Not required; process/catalog timestamps and timings above are text-reproducible.

Redacted configuration

{
  "subagentModels": ["gpt-5.6-sol", "gpt-5.6-terra"],
  "injectionPrompt": "spawn_agent supports optional \"model\" and \"reasoning_effort\" arguments. When setting either, set fork_turns to \"none\" or a positive turn count, and make the message self-contained.{{roster}}",
  "websockets": false
}

Notes

Related to #857 (closed), which added this staleness detection. The detection logic itself looks correct for the genuine stale case; the gap is the unknown fallback sharing the stale wording, plus the enumerate-then-stat race against transient processes.

One caveat on attribution: the code paths, timings, ocx doctor output, and timestamps above are directly verified. The specific race between the two CIM queries is a strong inference from those observations rather than a captured failure — the injection debug log endpoint returned 404 in this version, so I could not read the internal per-turn state directly. If there is a supported way to dump collectCodexAppServerCatalogState() results per turn, I am happy to capture it.

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.

[github-actions]

_{Automated translation bookkeeping — detected language: English.}

[Ingwannu]

Confirmed on current dev: this is a real guidance correctness bug. multiAgentGuidanceText() currently maps unknown and stale to the same catalog-change/restart message, while collectCodexAppServerCatalogState() can return unknown when a process start time cannot be read. The message therefore asserts both a cause and a remedy that were never established.

PR #1407 is the correctly scoped first fix: it keeps the conservative suppression of model/reasoning overrides for an unverifiable turn, but gives unknown distinct and truthful wording. It does not eliminate the underlying Windows enumerate-then-start-time race, so it should not be treated as a process-lifecycle fix. A broader process-filtering/query change would need its own Windows regression evidence.

Leaving this open until the linked fix lands.

[Wibias]

Fixed by #1407