diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/000_plan.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/000_plan.md new file mode 100644 index 0000000000..5072479f3b --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/000_plan.md @@ -0,0 +1,186 @@ +# 260816 — codex-rs multi-agent v2 + history performance: opencodex response + +## Objective + +Upstream codex-rs shipped two changes that opencodex must answer: multi-agent v2 +delegation now targets **every** catalog model that is not explicitly disabled +(Luna included), and the local conversation-history stack was redesigned around +paginated rollouts, ordinals, and a SQLite projection. This unit determines what +opencodex must change, and stages it as dependency-ordered implementation phases. + +> **Revision 2 (2026-08-16), after an adversarial A-phase audit.** An independent +> reviewer returned FAIL with 11 blockers against revision 1; all were verified against +> real code and folded in. The substantive corrections: the roster eligibility predicate +> (`sync.ts:105`) was the actual defect and revision 1 left it untouched; the subagent +> fallback chain can silently downgrade a v2 child; G5's stated impact was unreachable +> and is now conditional; three phase dependencies were not real; and the research docs +> carried prescriptive roadmaps that belong here. Details in each decade doc's amendment +> note and in `006_audit_round1.md`. + +## Evidence baseline + +| Repo | Commit | Verified | +| --- | --- | --- | +| upstream codex-rs (조사 시점) | `9dd22890f5ff47e4af128c20e32b9758a61d78d2` | `git log -1`, 2026-08-12 | +| upstream codex-rs (재검증 후, ff 완료) | `49db349ff` | 2026-08-15, +181 커밋. 두 P0 모두 유효함을 재확인 — `008`/`009` | +| opencodex | `7612e4c4f81544a250c3eea9fe8ca85d8022e765` | `git log -1`, `fix(routing): source capability evidence from explicit catalog provenance (#1799)` | + +Research documents (evidence only — no prescriptions; see LEXICO-SPLIT-01): + +- `001_upstream_multiagent_v2_evidence.md` — v2 wire surface, catalog contract, tool schemas. +- `002_upstream_history_perf_evidence.md` — rollout format, migration, pagination, proxy-visibility verdict. +- `003_opencodex_subagent_catalog_inventory.md` — current opencodex catalog/subagent surface. +- `004_opencodex_history_responses_inventory.md` — current opencodex rollout/Responses surface. +- `005_public_web_evidence.md` — public claim ledger. +- `006_audit_round1.md` — the A-phase reviewer's blockers and their disposition. + +## The two findings that drive this unit + +### 1. `multi_agent_version` no longer means "may I be a delegation target" + +Before `6d4d9442c` (2026-08-04), a v2 parent could only spawn a model whose catalog +value equalled `v2`. Now `model_supports_multi_agent_backend` +(`codex-rs/core/src/tools/handlers/multi_agents_common.rs:36-42`) admits every model +*except* explicit `Disabled`. The value instead decides whether the **child** gets +collaboration tools: `collab_tools_enabled` +(`codex-rs/core/src/tools/spec_plan.rs:599-610`) gives a child recursive tools only +when its own catalog value is exactly `Some(V2)`. + +| Value | Offered to a v2 parent | Child gets collab tools | Meaning | +| --- | --- | --- | --- | +| `"v2"` | yes | yes | recursive delegator | +| `"v1"` | yes | no | **leaf worker** | +| absent/null | yes | no | **leaf worker** | +| `"disabled"` | no | no | ineligible | + +opencodex encodes the OLD rule in two places, not one: + +1. `isEligibleV2SubagentEntry` (`src/codex/catalog/sync.ts:105-108`) returns true only + for `v2`/null/undefined — so an explicit `v1` pin like Luna's is **excluded from the + roster entirely**. This is the load-bearing defect. +2. `applyMultiAgentMode` (`src/codex/catalog/parsing.ts:382-388`) stamps every unpinned + row `"v2"` when the feature flag is on, claiming every routed third-party model is a + recursive delegator. + +Both must change together: fixing only the stamp leaves Luna excluded, and fixing only +the predicate leaves routed models over-claiming recursion. + +Classification: **silent degradation** — nothing errors; the roster is simply wrong. + +### 2. Paginated rollouts reject ordinal-less appended records + +`6bb6e9045` + `4bb7ee347` introduce paginated rollouts: each JSONL line carries a +monotonically increasing `ordinal`, `SessionMeta.history_mode` becomes `"paginated"`, +and a SQLite projection materializes turns/items. `read_projection_steps` +(`thread_history_materialization.rs:170-186`) returns a hard `Internal` error for a +paginated line missing an ordinal. + +opencodex's `updateSessionMeta` (`src/codex/history-provider.ts:523+`) **always** +appends an ordinal-less `session_meta` line, and every thread SELECT +(`history-provider.ts:589,714,722`) omits `history_mode`. Upstream's equivalent +(`codex-rs/thread-store/src/local/update_thread_metadata.rs:74`) branches on `paginated` and updates only the threads table. + +Classification: **compat-break** — corrupts a paginated thread's projection. + +### What is explicitly NOT our problem + +The "~98% fewer requests" figure is an N+1 elimination in *local SQLite* summary paging +(`332eac4b8`): ~749 queries to ~8 for a 741-turn thread at the 100-turn page cap. It is +not a reduction in `/v1/responses` calls, and `ResponsesApiRequest` +(`codex-rs/codex-api/src/common.rs:252`) is unchanged by the entire history series. + +The `27.6s → 1.7s` and 741-turn figures come from an **official OpenAI announcement** +(user-confirmed). They do not appear in any public PR body — `gh api search/issues` returns +`total_count: 0` for both `27.6s` and `"98% fewer"` — so treat them as an internal benchmark +whose *code* is public, not as a fabricated claim. `009_gh_pr_review.md` maps the figures to +the PRs that produced them (#36384 N+1 removal, #32234/#33364 pagination, #36948-36951 TUI +bounded hydration, #38604 resume round-trip removal, #34361 clone avoidance). What remains +opencodex-relevant is unchanged: those requests are local SQLite/app-server calls, not +`/v1/responses` calls. + +opencodex is a provider proxy, not Codex's app-server. It must **not** implement +`thread/turns/list`, `thread/items/list`, or `includeTurns`. + +## Gap matrix + +| # | Gap | Class | Phase | +| --- | --- | --- | --- | +| G1 | `default` mode stamps unpinned rows `v2`, erasing leaf semantics | silent-degradation | 010 | +| G2 | No typed per-model multi-agent capability, and no creation path for one | missed-opportunity | 010 | +| G3 | `updateSessionMeta` appends ordinal-less lines to paginated rollouts | compat-break | 020 | +| G4 | Thread SELECTs omit `history_mode`; bulk updates cannot skip unknown rows | compat-break | 020 | +| G5 | `extractUserMessagePreview` misses canonical `ItemCompleted` records | **conditional** — no reachable consumer found; see 030 gate | 030 | +| G6 | Rollout filename assumed to carry thread id; `4ef836f88` separates them | compat-break (latent) | 030 | +| G7 | `hasAgentsMaxThreads` claims a boot refusal upstream reverted in `1f304dd1f` | stale-guard | 040 | +| G8 | `/v1/responses/compact` full-field forwarding unverified | missed-opportunity | 050 | +| G9 | `structure/03_catalog-and-subagents.md:133-153` documents the superseded rule | docs-drift | 010 | +| **G12** | Quota fallback can rewrite a v2 child to a v1/disabled model, keeping collab tools | **compat-break** | 010 | +| G13 | `collaboration.ts:349` fork-override guidance | **no-action (optional polish)** — upstream's own hint (`config/mod.rs:253`) says forks "do not accept overrides", so our text mirrors it; the implementation honors overrides but the guidance is not wrong | 010 | +| **G14** | `model_messages.multi_agent` (role/mode 지시문)이 카탈로그에서 공급됨 (#38619); opencodex는 `model_messages` 를 `metadata.ts:300-308` 에서 변형하고 `upstream-models.json` 스냅샷으로 공급하는데 이 서브트리를 모름 | missed-opportunity → 잠재 silent-degradation | 060 (신규, 미작성) | +| G10 | Storage cleanup refuses paginated history | no-action (keep refusal) | — | +| G11 | App-server pagination protocol | no-action (out of proxy scope) | — | + +G12 and G13 were found by the A-phase reviewer, not by the research swarm. + +## Phase map + +Ordering follows build order where a real dependency exists. Revision 1 asserted three +dependencies that the reviewer disproved; the corrected map is mostly **parallel**, which +is itself a useful finding — these are independent defects, not a chain. + +| Phase | Doc | Outcome | Depends on | +| --- | --- | --- | --- | +| 1 | `010_phase1_catalog_capability_contract.md` | Leaf semantics restored end-to-end: eligibility, stamp, creation path, fallback capability class | — | +| 2 | `020_phase2_history_mode_awareness.md` | Never write ordinal-less records to a paginated rollout | — | +| 3 | `030_phase3_rollout_identity_and_previews.md` | Rollout-id/thread-id separation locked; G5 gated | — | +| 4 | `040_phase4_config_alias_and_docs.md` | `max_threads` alias truth through its full consumer chain | — | +| 5 | `050_phase5_compact_wire_verification.md` | `/v1/responses/compact` field-fidelity regression | — | + +Recommended execution order (risk-first, not dependency-forced): 1 → 2 → 4 → 3 → 5. +Phase 1 carries two compat-breaks (G12) and the headline defect; Phase 2 carries the +data-corruption risk. Phases 3 and 5 may both close as NOOP with recorded evidence. + +Each phase is one full PABCD cycle and closes with an independently verifiable gate. + +## Environment precondition (applies to every phase) + +This worktree has **no `node_modules/`**. Every verifier command therefore exits 1 for an +environmental reason (`Cannot find module 'zod/v4'`; `TS2688: bun-types`). Each phase's B +must run `bun install` first and re-record real exit codes beside its verifier table. +Note `package.json:41` defines `"test": "bun scripts/test.ts"` — use `bun run test` for a +full run, `bun test ` for focused iteration. + +## Scope boundary + +IN, by phase: + +- **catalog/capability (010):** `src/codex/catalog/parsing.ts`, `src/codex/catalog/sync.ts`, + `src/codex/catalog/provider-fetch.ts`, `src/codex/catalog/effort.ts` (the + `applyCatalogModelMetadata` bridge), `src/codex/subagent-model-fallback.ts`, + `src/server/responses/core.ts` (both fallback call sites), + `src/server/responses/collaboration.ts`, `src/server/management/model-rows.ts`, + `gui/src/pages/models-shared.ts`. +- **history (020/030):** `src/codex/history-provider.ts`, `src/codex/sqlite-columns.ts` (new), + `src/codex/history-worker.ts`, `src/codex/history-job.ts`, `src/codex/history-transition.ts`, + `src/codex/history-migration-guardian.ts`, `src/codex/inject.ts`, + `src/codex/convergence-types.ts` and `src/codex/transition-state.ts` (the durable + reason vocabulary + its schema migration, Phase 2 Change 6), + `src/storage/cleanup.ts` (read paths + the `columnExists` re-export). +- **config/docs (040):** `src/codex/features.ts`, `src/cli/v2.ts`, + `src/server/management/agent-settings-routes.ts` (GET and PUT), `gui/src/pages/Models.tsx` + and its locale strings. +- **compact (050):** `src/server/responses/compact.ts`, `src/adapters/openai-responses.ts`. +- **docs:** `structure/03_catalog-and-subagents.md`, `structure/05_gui-and-management-api.md`. +- matching `tests/` for each. + +OUT: the codex-rs checkout (read-only), app-server protocol reimplementation, **other** +provider adapters (`src/adapters/openai-responses.ts` is explicitly IN for Phase 5's +passthrough regression; no other adapter is), GUI redesign beyond the copy/type changes +listed above, release actions, any `git push`. + +## Terminal outcome for this docs cycle + +`DONE` when 000-007 plus every decade doc exist at diff-level precision, every audit-round +blocker is folded or explicitly rebutted, and the unit is committed locally. `006` and `007` +are audit history: the decade docs are canonical, and where an amendment corrected an +earlier instruction the canonical text was rewritten rather than appended to. diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/001_upstream_multiagent_v2_evidence.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/001_upstream_multiagent_v2_evidence.md new file mode 100644 index 0000000000..0a41999fdc --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/001_upstream_multiagent_v2_evidence.md @@ -0,0 +1,241 @@ +# Lane A — upstream multi-agent v2 wire surface + +Research baseline: upstream checkout `/Users/jun/Developer/codex/121_openai-codex`, branch `main`, HEAD `9dd22890f5ff47e4af128c20e32b9758a61d78d2` (2026-08-12). OPENCODEX baseline: `/Users/jun/.codex/worktrees/e80c/opencodex`, detached worktree HEAD as inspected on 2026-08-16. Both repositories were read-only during research; this file is the only write. + +## Executive result + +Multi-agent runtime selection is **per model**, then made sticky per thread. The provider catalog wire is `models[].multi_agent_version` (snake case); app-server JSON-RPC `model/list` exposes the same value as `data[].multiAgentVersion` (camel case), with values `"disabled" | "v1" | "v2" | null`. + +The decisive upstream change is `6d4d9442c7142c08ac5c5098dfd6e82d8cd9f65a`: a v2 parent may target every catalog model except one explicitly marked `disabled`. A target marked `v2` can recursively delegate; a target marked `v1` or with the field absent/null runs as a **leaf worker** and receives no collaboration tools. Thus `gpt-5.6-luna` (`multi_agent_version: "v1"`) is a valid child of a v2 parent, but is a leaf. + +OPENCODEX already carries the catalog field, but its default-mode implementation still encodes the superseded pre-`6d4d9442` equality rule: when the v2 feature is enabled it stamps every unpinned row as `v2`. That makes routed leaf workers look recursively v2-capable. The first implementation change should remove that blanket stamp in `default` mode while retaining explicit `v1`, `v2`, and `disabled` values. + +## Verified commits + +All requested anchors resolve in the checkout and were verified with `git show --stat`. + +| Commit | Date | Verified relevance | +| --- | --- | --- | +| `51e36d2ec23c0eff710053d28c400d447500a41a` | 2026-08-07 | Adds nullable `multiAgentVersion` to app-server v2 `model/list`, forwarding the model preset value. | +| `6d4d9442c7142c08ac5c5098dfd6e82d8cd9f65a` | 2026-08-04 | Changes v2 target eligibility from “catalog value equals v2” to “catalog value is not disabled”; suppresses collaboration tools for non-v2 child models. | +| `92b83e226df59dc5ec43a49259d7716821e20c85` | 2026-08-06 | Tracks v2 usage-hint hashes in world state so resume/config changes re-emit the right instructions; moves wait guidance into configurable instructions. | +| `0da13c6c993cbb6de3ce88591b316a40cbd411b1` | 2026-07-22 | Tracks effective multi-agent mode instructions as a durable/diffed world-state section. This is prompt/history state, not catalog eligibility. | +| `4462b9deef211723b781b426f5e5d36a5777115f` | 2026-07-23 | Adds default-on `features.multi_agent_v2.wait_agent_enabled`; omits v2 `wait_agent` when false. | +| `c4f42d161ae44a8d696ee9fb595709661979d187` | 2026-08-05 | Uses `gpt-5.6-luna` for API-key Guardian reviews. It does not alter delegation eligibility or tool schemas. | +| `3fe19fcd81559e543ea0747c1927b2d3a36a6885` | 2026-08-12 | Lazily resolves subagent analytics connections. No catalog/tool wire change. | +| `1151b23f01accb19e55c090a3349a32fdf2b4685` | 2026-08-06 | Lazily starts cached MCP servers for subagents. No catalog/tool schema change. | +| `4ffeddcbcc0dd72251433abff1ca9423e8017008` | 2026-08-06 | Fixes TUI subagent MCP startup-status settling. No catalog/tool schema change. | + +Supporting provenance: `3f1fb7ed8b641542add19bb841e4e4be5651693e` introduced the runtime metadata types; `92938d880eccbad1242a86a63f819f67780f68c0` added backend-aware spawn-model filtering and the five-model cap; `ea1545628404e448347bae336771eaf649614105` exposed v2 model/effort overrides; `6ddb747e7687e9e6e3a2482631028c07ddc89cb6` renamed v2 `assign_task` to `followup_task`; `8d415050fce4b4ebc6da1ba247379844235fa453` renamed v2 `close_agent` to `interrupt_agent`; `5f4d06ef186b896d316620556e561d59206c3ebf` marked v2 message payloads encrypted. + +## 1. Exact catalog and app-server wire shape + +### Provider `/models` catalog + +`codex-rs/protocol/src/openai_models.rs:L372-L461` — `ModelInfo` is the backend `/models` row. Its field is: + +```json +{ + "models": [ + { + "slug": "gpt-5.6-luna", + "multi_agent_version": "v1" + } + ] +} +``` + +- Field: `models[].multi_agent_version`. +- Values: `"disabled"`, `"v1"`, `"v2"`, or omitted/null. +- Enum definition: `codex-rs/protocol/src/protocol.rs:L2822-L2829` — `MultiAgentVersion`, serialized with `#[serde(rename_all = "snake_case")]`. +- Forward compatibility: `ModelInfo.multi_agent_version` uses `deserialize_optional_model_selector` at `openai_models.rs:L322-L331,L455-L460`; an unknown future string becomes `None`, proven by `model_info_treats_unknown_multi_agent_version_as_omitted` at `L1491-L1504`. +- Conversion: `impl From for ModelPreset` at `openai_models.rs:L723-L755` copies it unchanged. + +### App-server JSON-RPC `model/list` + +`codex-rs/app-server-protocol/src/protocol/v2/model.rs:L27-L34,L89-L121` — `MultiAgentVersion` and `Model` serialize with model fields in camel case: + +```json +{ + "data": [ + { + "id": "gpt-5.6-luna", + "model": "gpt-5.6-luna", + "multiAgentVersion": "v1" + } + ], + "nextCursor": null +} +``` + +- Field: `data[].multiAgentVersion`. +- Values: `"disabled" | "v1" | "v2" | null`. +- Mapping: `codex-rs/app-server/src/models.rs:L27-L62` — `model_from_preset`, specifically `preset.multi_agent_version.map(Into::into)`. +- Generated contract: `app-server-protocol/schema/typescript/v2/Model.ts` declares `multiAgentVersion: MultiAgentVersion | null`; `MultiAgentVersion.ts` declares the three strings. The JSON Schema permits the property to be absent for tolerant readers, but Rust serialization emits `null` for `None` because the field has no `skip_serializing_if`. + +This is **per-model**, not per-account. Account/auth state can change which models reach the list: `ModelPreset::filter_by_auth` at `openai_models.rs:L786-L795` keeps all models in ChatGPT mode but requires `supported_in_api: true` otherwise. The `multi_agent_version` value itself belongs to each model row; there is no account-level multi-agent field. + +## 2. V1 versus V2 decision and precedence + +The core decision function is `Config::multi_agent_version_for_model` in `codex-rs/core/src/config/mod.rs:L1543-L1550`: + +```rust +self.multi_agent_version_override() + .or(model_multi_agent_version) + .unwrap_or_else(|| self.multi_agent_version_from_features()) +``` + +The full precedence is: + +1. `features.multi_agent_v2` enabled -> `V2` (`multi_agent_version_override`, `L1523-L1531`). This wins even if `[agents] enabled = false`. +2. Otherwise `[agents] enabled = false` -> `Disabled`. +3. Otherwise use the selected model's catalog `multi_agent_version`, if present. +4. Otherwise stable feature `multi_agent`/`Feature::Collab` -> `V1`; if that feature is disabled -> `Disabled` (`multi_agent_version_from_features`, `L1533-L1541`). + +`Feature::Collab` is stable/default-on and `Feature::MultiAgentV2` is stable/default-off at `codex-rs/features/src/lib.rs:L1067-L1078`. + +Selection is sticky per thread. `Session::resolve_multi_agent_version_for_model` at `codex-rs/core/src/session/mod.rs:L3402-L3425` reuses the thread's `OnceLock` value if set; otherwise it resolves from the model and stores it. `resolve_multi_agent_version` at `L469-L485` restores persisted/inherited metadata and defaults older resumed/forked threads without metadata to V1. Consequently a mid-thread model switch does not normally switch the collaboration protocol. + +## 3. Leaf models in v2 + +At this HEAD, “leaf” is behavioral, not a separate enum or JSON field. + +- Eligibility: `model_supports_multi_agent_backend` at `codex-rs/core/src/tools/handlers/multi_agents_common.rs:L36-L42` allows every model under a v2 parent except `multi_agent_version == Some(Disabled)`. +- Recursion: `collab_tools_enabled` at `codex-rs/core/src/tools/spec_plan.rs:L599-L610` exposes collaboration tools to a v2 root, but for a child (`session_source.get_agent_path().is_some()`) requires the child model's catalog value to be exactly `Some(V2)`. + +Therefore: + +| Catalog value on target | Offered to v2 parent? | Child receives v2 collaboration tools? | Meaning | +| --- | --- | --- | --- | +| `"v2"` | Yes | Yes | Recursive/delegating v2 model. | +| `"v1"` | Yes | No | Leaf worker under a v2 parent. | +| omitted/null | Yes | No | Leaf worker under a v2 parent. | +| `"disabled"` | No | No | Explicitly ineligible for v2 delegation. | + +The checked-in upstream catalog `codex-rs/models-manager/models.json` represents `gpt-5.6-sol` and `gpt-5.6-terra` as `v2`, `gpt-5.6-luna` as `v1`, and gpt-5.5/gpt-5.4/gpt-5.4-mini/gpt-5.2 with the field absent. Luna and the absent-value models are thus leaf targets, not rejected targets. + +## 4. Exact tool-surface diff + +Tool specifications come from `codex-rs/core/src/tools/handlers/multi_agents_spec.rs`. Every input object schema sets `additionalProperties: false`; all tools set `strict: false`. + +### Naming and namespace + +- V1 is always the Responses namespace `multi_agent_v1` (`MULTI_AGENT_V1_NAMESPACE`, `L14-L15`). Its members are `spawn_agent`, `send_input`, `resume_agent`, `wait_agent`, and `close_agent`. +- V2's exact spawn name is **`spawn_agent`**, not `spawn`. V2 also has `send_message`, `followup_task`, `wait_agent`, `interrupt_agent`, and `list_agents`. +- V2 defaults to namespace `collaboration` (`DEFAULT_MULTI_AGENT_V2_TOOL_NAMESPACE`, `config/mod.rs:L254`). `multi_agent_v2_handler` at `spec_plan.rs:L1293-L1323` wraps tools in that namespace only when provider namespace tools are enabled and the configured namespace is non-null; otherwise they are flat functions. + +### Input schemas + +| Tool | Required fields | Optional fields | Notes | +| --- | --- | --- | --- | +| V1 `spawn_agent` | Schema declares none; runtime requires exactly one of `message` or `items`. | `message: string`, `items: UserInput[]`, `agent_type: string`, `fork_context: boolean`, `model: string`, `reasoning_effort: string`, `service_tier: string` | `fork_context=true` means full-history fork. Output `{agent_id, nickname}`. | +| V1 `send_input` | `target: string`; runtime also requires exactly one of `message`/`items`. | `message: string`, `items: UserInput[]`, `interrupt: boolean` | Output `{submission_id}`. `interrupt=true` redirects immediately. | +| V1 `resume_agent` | `id: string` | none | Output `{status}`. | +| V1 `wait_agent` | `targets: string[]` | `timeout_ms: number` | Waits for whichever listed agent finishes; output `{status: {id: AgentStatus}, timed_out}` and may include final content. | +| V1 `close_agent` | `target: string` | none | Shuts down target and descendants; output `{previous_status}`. | +| V2 `spawn_agent` | `task_name: string`, `message: string` | `agent_type: string`, `model: string`, `reasoning_effort: string`, `service_tier: string`, `fork_turns: string` | `message` schema carries `"encrypted": true`. `fork_turns` accepts `none`, `all` (default), or a positive integer string. Output `{task_name}` by default (`hide_spawn_agent_metadata=true`), else `{task_name,nickname}`. | +| V2 `send_message` | `target: string`, `message: string` | none | `message` encrypted; queues promptly but does not trigger a turn. No output schema. | +| V2 `followup_task` | `target: string`, `message: string` | none | `message` encrypted; triggers an idle non-root target or delivers at a boundary. No output schema. | +| V2 `wait_agent` | none | `timeout_ms: number` | Mailbox-wide wait, not target-specific. It returns no agent content: `{message, timed_out}`. | +| V2 `interrupt_agent` | `target: string` | none | Interrupts current turn but leaves the agent available; output `{previous_status}`. | +| V2 `list_agents` | none | `path_prefix: string` | Additional v2 tool omitted from the question's list; output `{agents:[{agent_name,agent_status}]}`. | + +V1 property builders are at `multi_agents_spec.rs:L586-L629` and V1 wait parameters at `L848-L874`. V2 properties are at `L631-L673` and v2 wait parameters at `L876-L885`. Output schemas are at `L360-L543`. + +### Dynamic schema controls + +`create_spawn_agent_tool_v2` at `multi_agents_spec.rs:L102-L146` changes the exposed fields: + +- `features.multi_agent_v2.expose_spawn_agent_model_overrides=false` removes `model` and `reasoning_effort` (default is true). +- If no agent roles exist, `agent_type` is removed. +- `hide_spawn_agent_metadata=true` (default) removes `service_tier` and hides nickname in output; current code still exposes model/reasoning when their separate flag is true. +- `wait_agent_enabled=false` omits v2 `wait_agent` entirely (`spec_plan.rs:L1166-L1173`). + +Default guidance at `config/mod.rs:L253` tells the model not to combine model/effort overrides with a full-history fork, but this is **guidance, not a runtime rejection at this HEAD**. `handle_spawn_agent` applies requested overrides before forking (`multi_agents_v2/spawn.rs:L39-L99`), and `spawned_full_history_v2_child_uses_model_precedence_without_dropping_context` (`core/tests/suite/subagent_notifications.rs:L1040-L1087`) proves an explicit override with `fork_turns:"all"` is honored. The hard parser rules are narrower: `SpawnAgentArgs::fork_mode` at `multi_agents_v2/spawn.rs:L191-L238` rejects legacy `fork_context` and validates `fork_turns` as `none`, `all`, or a positive integer string. + +### `deny_unknown_fields` + +- Every v2 runtime argument struct has `#[serde(deny_unknown_fields)]`: spawn (`multi_agents_v2/spawn.rs:L191-L202`), send/followup (`message_tool.rs:L26-L40`), wait (`wait.rs:L123-L127`), interrupt (`interrupt_agent.rs:L104-L108`), and list (`list_agents.rs:L57-L61`). Unknown input keys fail parsing. +- V1 runtime structs do **not** use `deny_unknown_fields`: spawn (`multi_agents/spawn.rs:L234-L244`), send (`send_input.rs:L130-L137`), wait (`wait.rs:L273-L278`), close (`close_agent.rs:L161-L164`), resume (`resume_agent.rs:L163-L166`). Their published schemas still say `additionalProperties:false`, but serde will ignore an unknown key if it reaches the runtime. This is a real behavioral difference. +- V2 `SpawnAgentArgs` deliberately includes `fork_context: Option` even though it is absent from the schema, solely to emit the specific migration error “use fork_turns instead”; it is therefore a recognized-but-rejected legacy key, not an unknown key. + +## 5. Config and feature gates + +Primary gates and defaults: + +- `[features] multi_agent_v2 = true` or `[features.multi_agent_v2] enabled = true`: force V2. Stable feature, default false. +- `[features] multi_agent = ...`: legacy V1 feature (`Feature::Collab`), stable/default true. +- `[agents] enabled = false`: disables collaboration only when v2 is not explicitly enabled. +- `features.multi_agent_v2.wait_agent_enabled`: default true; controls only v2 `wait_agent`. +- `features.multi_agent_v2.expose_spawn_agent_model_overrides`: default true. +- `features.multi_agent_v2.hide_spawn_agent_metadata`: default true. +- `features.multi_agent_v2.tool_namespace`: default `"collaboration"`. +- `features.multi_agent_v2.non_code_mode_only`: default true (direct-model-only exposure). +- Other v2 config fields include min/max/default wait timeouts, usage/root/subagent hints, `subagent_developer_instructions`, and `multi_agent_mode_hint_text`. The TOML struct is `MultiAgentV2ConfigToml` at `codex-rs/features/src/feature_configs.rs:L74-L119`, with `deny_unknown_fields`. + +Runtime defaults are in `MultiAgentV2Config::defaults_for_max_concurrency` at `core/src/config/mod.rs:L1291-L1325`; parsing/default resolution is `resolve_multi_agent_v2_config` at `L2678-L2777`. + +### Concurrency semantics + +- V1 default: `DEFAULT_AGENT_MAX_THREADS = 6` (`config/mod.rs:L209`). The root is not counted; this is the maximum spawned/open child threads. +- V2 default: `features.multi_agent_v2.max_concurrent_threads_per_session = 4` (`L210`). The root **is counted**, so `effective_agent_max_threads` subtracts one (`L1552-L1565`), yielding three child slots. +- Current canonical shared key: `[agents] max_concurrent_threads_per_session`; legacy `[agents] max_threads` is a serde/config-normalization alias (`config/src/config_toml.rs:L660-L672`, `config/src/key_aliases.rs:L11-L21`). Under V1 its value is the child limit. When used as the fallback for V2, `resolve_multi_agent_v2_config` adds one (`config/mod.rs:L2680-L2689`) so the same configured N child slots become N+1 total slots. +- A value directly under `[features.multi_agent_v2]` is already the total including root and is not incremented. + +## REQUIRED CATALOG CONTRACT + +This is the proxy-author implementation contract at upstream HEAD `9dd22890f`. + +### A. Transport shape + +Return provider catalog JSON as an object with `models: ModelInfo[]`. For each row, use snake-case `multi_agent_version`. Do not send app-server camel-case `multiAgentVersion` to `/models`; that name belongs only in JSON-RPC `model/list` output. + +The delegation-specific minimum is: + +```json +{ + "slug": "provider/model-id", + "display_name": "Model name", + "description": "What the model is for", + "visibility": "list", + "supported_in_api": true, + "priority": 10, + "supported_reasoning_levels": [ + { "effort": "medium", "description": "Balanced" } + ], + "default_reasoning_level": "medium", + "service_tiers": [], + "multi_agent_version": "v1" +} +``` + +That snippet shows delegation-relevant fields, not the entire `ModelInfo` parse contract. A standalone proxy row must also satisfy the non-defaulted `ModelInfo` fields at this HEAD: `shell_type`, `support_verbosity`, `truncation_policy`, `supports_parallel_tool_calls`, and `experimental_supported_tools` (plus the fields above). OPENCODEX's `ensureStrictCatalogFields` already fills most of these on generated rows. + +### B. Exact eligibility conditions for being *offered* in `spawn_agent` + +The list comes from `TurnContext.available_models`, loaded from `ModelsManager::try_list_models`/`list_models` (`turn_context.rs:L574,L327-L332`). `ModelsManager::build_available_models` (`models-manager/src/manager.rs:L125-L138`) sorts by ascending `priority`, converts to presets, and auth-filters. + +A row is described to the model as an available override iff all are true: + +1. It survives auth filtering: ChatGPT mode accepts all; non-ChatGPT/API mode requires `supported_in_api: true` (`openai_models.rs:L786-L795`). +2. `visibility == "list"`, which converts to `ModelPreset.show_in_picker=true` (`openai_models.rs:L750`). +3. Under a v2 parent, `multi_agent_version` is not explicit `"disabled"`; under non-v2, this particular backend filter imposes no additional catalog condition (`multi_agents_common.rs:L36-L42`). +4. It falls within the first five eligible rows after ascending-priority ordering: `MAX_SPAWN_AGENT_MODEL_OVERRIDES=5` and `spawn_agent_models_description` at `multi_agents_spec.rs:L781-L846`. +5. V2 model overrides are exposed (`features.multi_agent_v2.expose_spawn_agent_model_overrides=true`). If false, the model list and `model`/`reasoning_effort` fields are absent from the v2 spawn schema (`L102-L119`). + +No other capability bit gates the target list. In particular, `supports_parallel_tool_calls`, search support, tool mode, service tiers, and reasoning support do not decide eligibility. Reasoning and service-tier metadata only annotate/validate the selected override. + +Runtime selection is slightly broader than “offered”: `find_spawn_agent_model_name` at `multi_agents_common.rs:L431-L456` accepts any exact model present in `available_models` and backend-compatible, even if hidden or beyond the first five; the visible/top-five filter is used only to construct the advertised list and error message. A proxy should not rely on hidden status as an authorization barrier. + +### C. Capability values a proxy should emit + +- Emit `"v2"` only when the child should receive collaboration tools and may recursively delegate. +- Emit `"v1"` or omit/null when it may be selected as a v2 child but should remain a leaf. Preserve upstream `v1` for Luna. +- Emit `"disabled"` only to exclude the model from v2 delegation entirely. +- Preserve explicit upstream/provider values. Unknown future values are treated as absent by this client, which currently means leaf eligibility under a v2 parent. + + +--- + +Evidence only (LEXICO-SPLIT-01). The prescriptive roadmap that once ended this file +lives in [000_plan.md](000_plan.md) and [010_phase1_catalog_capability_contract.md](010_phase1_catalog_capability_contract.md). diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/002_upstream_history_perf_evidence.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/002_upstream_history_perf_evidence.md new file mode 100644 index 0000000000..6dc78c41b7 --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/002_upstream_history_perf_evidence.md @@ -0,0 +1,263 @@ +# Lane B — upstream conversation-history performance rework + +Research target: `openai/codex` checkout at `9dd22890f5ff47e4af128c20e32b9758a61d78d2` (2026-08-16 checkout state). +Planning target: OPENCODEX checkout at `7612e4c4f81544a250c3eea9fe8ca85d8022e765`. + +## Executive reconstruction + +This was not one provider-API optimization. It was a local history-stack redesign with three related layers: + +1. Rollout JSONL remained the durable source, but paginated rollouts gained monotonically increasing `ordinal` values, canonical turn lifecycle records, and `SessionMeta.history_mode = "paginated"`. +2. A separate SQLite projection (`thread_history_1.sqlite`) materializes turns/items and supports bounded turn/item pages. App-server and TUI callers can request only the page they need instead of hydrating an entire transcript. +3. The largest directly provable request-count reduction is commit `332eac4b858a40575a55f21026ecf8cb662e539c`, which removed an N+1 SQLite query in summary paging: the old code issued one item query for each returned turn; the new query joins the first user and final agent item into the turn-page query. + +The supplied `27.6s -> 1.7s` and “~98% fewer requests” benchmark numbers do not appear in the reachable commit messages, source comments, or tests. They are therefore **UNVERIFIED as benchmark measurements**. The mechanism and approximate percentage are independently reproducible from code: for 741 turns with the app-server maximum of 100 turns/page, the dominant SQL count falls from about `8 page queries + 741 per-turn summary queries = 749` to about `8 joined page queries`, a `98.93%` reduction. These are local SQLite requests, not `/v1/responses` provider requests. + +## Commit ledger + +All requested abbreviated anchors resolve to these exact commits in the upstream checkout: + +| Commit | Date | Verified subject | Role | +|---|---:|---|---| +| `99915080b6ce1d7c0788edc72f116455bd77fcd6` | 2026-08-11 | Store model history in response item envelopes (#38045) | Introduces the in-memory envelope; initially it contains only `item`. | +| `63002bdb26c939925f3fa59b9575cc0a3564cb45` | 2026-08-10 | Extract persisted history types into a dedicated crate (#37871) | Moves persisted rollout types from `protocol` to `codex-history`. | +| `4bb7ee3472830bda4d58996e4ff0106e7bef0f3f` | 2026-08-07 | Add rollout migration tooling and background migration (#37348) | Manual CLI, startup migration, journal/recovery path. | +| `6bb6e9045f728f172ee1c68e6dc56ce6a53cbdb8` | 2026-08-05 | Add legacy rollout migration to paginated history (#37175) | Canonical legacy-to-paginated conversion and one-way state promotion. | +| `aac9f842473ac6a05d417dd76ce8b89bdb3b707d` | 2026-08-06 | Preserve legacy semantics during rollout migration (#37191) | Preserves rollback/subagent/compaction semantics during canonicalization. | +| `e87e2b495bcf8aa1950e2bb24cc95bfdc6fd473a` | 2026-08-04 | Support includeTurns reads for paginated threads (#36993) | Makes `thread/read(includeTurns:true)` hydrate paginated turns. | +| `449f099f1cad470fa4f63dcc198f8d6cb8944cde` | 2026-08-04 | Harden paginated history handling in the TUI (#36951) | Bounded TUI history paging/fallback hardening. | +| `4496ba3fd5645af57779fc6a468c270a0b23cfb7` | 2026-08-11 | Use session metadata to validate thread history paths (#38033) | Validates a rollout path by its first `SessionMeta.id`. | +| `ed390a5dc4b5e859ca7411ea5a81cd96470fb748` | 2026-08-11 | Filter live rollout items in place (#38034) | Allocation reduction in live-item filtering; no wire/schema change. | +| `4ef836f883c38ba6d39e6920f335ce6452b7de33` | 2026-08-12 | Distinguish rollout IDs from thread IDs (#38127) | Allows multiple rollout IDs/files for one thread ID. | +| `3a6f747d77d770c74203e051abeccc66d4a06d3b` | 2026-08-11 | Preserve harness metadata across conversation history (#38058) | Adds persisted optional metadata beside response payloads. | +| `d6ca19d99b8188062795315850aed34d25c8f037` | 2026-08-11 | Add turn-aware response item injection (#38047) | Makes injection persistence/flush turn-aware. | +| `c2bcb9a26b5bdbdc66c48cf3bc3bb382568e800c` | 2026-08-07 | Reuse parent compactions in Guardian review sessions (#37513) | Seeds Guardian review history from parent compaction. | +| `bd5b55e403e867f957e371840377cd284023f98c` | 2026-07-22 | Track compaction time in turn profiles (#34835) | Adds local turn-profile timing. | +| `b9ba969f3996cc73b6c8129bbc1606caad2192a9` | 2026-08-04 | Enable remote compaction for Amazon Bedrock (#36981) | Makes `/v1/responses/compact` provider-visible for Bedrock. | +| `4bd5b9fd0933c4399351bd59364a96a8fb99cd33` | 2026-08-04 | Keep image resize notices attached during remote compaction (#36956) | Changes compact-request `input` retention, not its schema. | + +Additional causal commits not in the supplied anchor list: + +| Commit | Verified subject | Why it matters | +|---|---|---| +| `5267e805fb830891c0b23376bcd9cbd382c3473c` | feat(app-server): add history_mode to thread (#29927) | Establishes the compatibility floor: metadata discovery remains available, unsupported full-history operations fail closed. | +| `332eac4b858a40575a55f21026ecf8cb662e539c` | Load turn summaries with paginated queries (#36384) | Direct N+1 SQL removal behind the request-count reduction. | +| `8bfa49e350edb065889332c72854d06f0e7ce50f` | Paginate transcript history in TUI | Starts bounded TUI transcript loading. | +| `3b8d22ec2c75bf8fcd6048c34039344795ff7a0a` | Improve paginated TUI history loading | Continues the incremental-loading path. | +| `dbcd837c20cd891f3de4bb3976c7f643d0b2fb93` | Paginate TUI transcript history | Establishes the paged transcript flow later hardened by `449f099f1`. | + +## 1. Old and new rollout/history formats + +### Ownership and serde types + +At `63002bdb2`, persisted history types move out of `codex-rs/protocol/src/protocol.rs` into the dedicated `codex-rs/history` crate. At current HEAD: + +- `codex-rs/history/src/lib.rs:36` (`ResponseItemEnvelope`) owns model-history envelopes. +- `codex-rs/history/src/lib.rs:46` (`CodexHarnessMetadata`) owns harness-only metadata. +- `codex-rs/history/src/lib.rs:91` (`RolloutItem`) owns persisted rollout variants. +- `codex-rs/history/src/lib.rs:137` (`CompactedItem`) owns compacted replacement history. +- `codex-rs/history/src/lib.rs:195` (`RolloutLine`) owns each JSONL record: `timestamp`, optional `ordinal`, and flattened `item`. +- `codex-rs/history/src/rollout_payload.rs:20` (`RolloutItemWire`) is the serde-facing tagged wire representation. +- `codex-rs/history/src/rollout_payload.rs:117` (`CompactedItemWire`) provides the compatibility sidecar for replacement-history metadata. + +The important distinction is that “new history” is two things, not one replacement file: + +- JSONL remains authoritative durable storage. +- `codex-rs/state/thread_history_migrations/0001_thread_history.sql:1` projects it into `thread_turns`, `thread_items`, and `thread_history_projection_state`, whose checkpoint is `(next_rollout_byte_offset, next_rollout_ordinal)`. +- The projection database filename is `thread_history_1.sqlite` (`codex-rs/state/src/sqlite.rs:34`, `SqliteDb::for_thread_history` at line 158). + +### Concrete JSONL shape + +An old/legacy rollout response item has no ordinal. Removing only whitespace, this is a complete valid legacy line: + +```json +{"timestamp":"2025-01-03T12:00:00.000Z","type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"hello"}]}} +``` + +A paginated line has an explicit ordinal. This exact shape is asserted by `codex-rs/history/src/tests.rs:39` (`response_item_rollout_line_preserves_shape`): + +```json +{"timestamp":"2025-01-03T12:00:00.000Z","ordinal":7,"type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"hello"}]}} +``` + +Migration does more than add a number. `codex-rs/thread-store/src/local/rollout_migration/canonicalizer.rs:81` (`write_head_session_meta`) rewrites the head metadata as paginated, clears `history_base` and the subagent boundary after canonicalizing inherited history, and `write_item` at line 458 serializes every output line with `ordinal: Some(next_ordinal)`. Migration tests at `codex-rs/thread-store/src/local/rollout_migration_tests.rs:287` verify a paginated first `SessionMeta`, contiguous ordinals, and SQLite materialization. + +The new state is marked by `SessionMeta.history_mode`, whose enum is `Legacy` (serde default) or `Paginated` at `codex-rs/protocol/src/protocol.rs:700` (`ThreadHistoryMode`). It is not a replacement for `RolloutLine.ordinal`: both are part of the contract. + +## 2. The response item envelope + +Current in-memory fields (`codex-rs/history/src/lib.rs:36`, `ResponseItemEnvelope`) are: + +```rust +pub struct ResponseItemEnvelope { + pub item: ResponseItem, + pub metadata: Option, +} +``` + +`CodexHarnessMetadata` is currently a closed, empty struct (`codex-rs/history/src/lib.rs:46`). The envelope intentionally separates context consumed by Codex/harnesses from the `ResponseItem` sent to a model. + +Chronology matters: + +- `99915080b` introduced an in-memory envelope containing only `item`; it did **not** itself change the persisted `response_item` JSON shape. +- `3a6f747d7` added optional envelope metadata and persisted it as a sibling of `payload`, while preserving the old payload shape. + +Exact metadata-bearing wire shape, asserted at `codex-rs/history/src/tests.rs:66` (`response_item_envelope_stores_metadata_beside_rollout_payload`): + +```json +{"timestamp":"2025-01-03T12:00:00.000Z","ordinal":7,"type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"hello"}]},"metadata":{}} +``` + +There is no envelope `version` field. Compatibility/version markers live around it: + +- `SessionMeta.history_mode`: legacy vs paginated file semantics. +- `RolloutLine.ordinal`: projection order/checkpoint contract. +- Startup migration ID `legacy_to_paginated_v1` (`codex-rs/thread-store/src/local/rollout_migration/startup.rs:33`). +- Pending migration journal under `$CODEX_HOME/rollout-migrations/.pending`. +- SQLite migration/projection tables, including `rollout_migration_state` and `rollout_migration_skips` from `codex-rs/state/migrations/0047_rollout_migration_state.sql:1`. + +For compacted histories, current `CompactedItemWire` stores `replacement_history` as raw response items and optional aligned `replacement_history_metadata` as a sibling sidecar. `TryFrom` at `codex-rs/history/src/rollout_payload.rs:170` accepts a missing legacy sidecar and rejects a mismatched sidecar length. Tests at `codex-rs/history/src/tests.rs:137` (`response_item_replacement_history_preserves_shape`) and line 166 (`compacted_replacement_history_stores_metadata_in_an_aligned_sidecar`) cover both shapes. + +Unknown fields inside current empty metadata are ignored on read (`codex-rs/history/src/tests.rs:100`, `response_item_envelope_ignores_unknown_harness_metadata_fields`). The inverse claim—every arbitrarily old binary safely reading every future metadata extension—is **UNVERIFIED**; it depends on that binary's serde policy and predates this contract. + +## 3. Migration, publication, and compatibility + +### Background versus on-read + +Legacy-to-paginated conversion is not an on-read mutation. + +- Manual entry point: `codex migrate-rollouts`; arguments include `--apply`, `--thread`, `--max-mib-per-second`, `--json`, and `--verbose` (`codex-rs/cli/src/migrate_rollouts.rs:20`). It is dry-run unless `--apply` is supplied. +- Background entry point: `4bb7ee347` adds startup migration behind `BackgroundPaginatedRolloutMigration`; `codex-rs/core/src/thread_manager.rs:388` checks it and spawns migration at line 399. Current feature metadata marks it under development and disabled by default (`codex-rs/features/src/lib.rs:962`). +- Startup traversal: `codex-rs/thread-store/src/local/rollout_migration/startup.rs:51` reads migration cursor/skips/pending state, scans the startup window, applies eligible files, and advances the pre-scan cursor. `CURSOR_LOOKBACK_SECONDS` at line 36 sets the lookback to 48 hours. +- Read-time work only materializes or advances the SQLite projection for a rollout already marked paginated (`codex-rs/thread-store/src/local/thread_history_materialization.rs:19`, `materialize_to_sqlite`). It does not silently convert a legacy rollout. + +### Where migration writes + +`codex-rs/thread-store/src/local/rollout_migration.rs:321` (`migrate_rollout_path`) and line 467 (`migrate_one_rollout`) implement this publication sequence: + +1. Read/validate `SessionMeta` and current mode. +2. Record a pending journal in `$CODEX_HOME/rollout-migrations/`. +3. Canonicalize into a sibling staging file named `..paginated.tmp`. +4. Build and verify the `thread_history_1.sqlite` projection. +5. Re-stat the legacy source. If length or modification time changed, abort with a conflict rather than replacing the newer source (`migrate_one_rollout`, lines 603–613). +6. Atomically rename the staged paginated rollout over the source (lines 616–629), then finalize the journal. + +Journal recovery can reconstruct a missing projection from a rollout that was published before a crash; `codex-rs/thread-store/src/local/rollout_migration_tests.rs:1828` exercises that case. The JSONL file remains the durable recovery source. + +`aac9f8424` adds rollback plans/replay and preserves legacy compaction, fork/history-base, subagent, and rollback semantics while canonicalizing. `6bb6e9045` deliberately treats promotion as one-way in SQLite so stale legacy metadata cannot downgrade an already paginated thread. + +### Forward/backward compatibility matrix + +| Writer / file | Reader | Verified behavior | +|---|---|---| +| Older Codex writes a legacy file; newer Codex reads it | Newer | `history_mode` defaults to `Legacy`; normal legacy replay remains available. Manual/background migration may later promote it. | +| Older Codex appends while newer Codex is migrating | Newer migrator | New writers use locks; an older writer does not. The final source length/mtime recheck detects an observed change and aborts publication, preserving the legacy file. Safety against a write in the narrow interval after the final stat and before rename is **UNVERIFIED** from the inspected code. | +| Newer Codex reads paginated JSONL with absent/stale SQLite projection | Newer | Materialization resumes from `thread_history_projection_state`; pending-journal recovery can rebuild after interrupted publication. | +| History-aware older release sees a newer paginated thread | Compatibility-floor releases | Foundational commit `5267e805fb830891c0b23376bcd9cbd382c3473c` records fail-closed behavior: metadata-only list/read remains possible, while full history read/resume/mutation is rejected when the release cannot honor the history mode. | +| Pre-`history_mode` binary reads paginated JSONL | Truly old | **UNVERIFIED / unsupported.** Ignoring unknown `ordinal` or `metadata` does not prove it understands canonical `ItemCompleted` events, inheritance collapse, or rollout IDs. | +| Newer writer adds sibling `metadata`; immediately older envelope-aware reader reads it | Older | Payload shape is deliberately preserved by `3a6f747d7`; serde readers that ignore unknown sibling fields remain compatible. Universal compatibility with all old binaries is **UNVERIFIED**. | +| Newer Codex writes paginated; older Codex writes a separate legacy rollout for the same thread | Newer | `4ef836f88` separates rollout identity from thread identity, and the state DB's selected `rollout_path` is authoritative. Startup scanning can consider the new legacy file; no last-file-wins guarantee should be inferred from filename alone. | + +Rollout filenames now encode this distinction through `codex-rs/rollout/src/rollout_file_name.rs:1` (`RolloutFileName`): ordinary rollouts can use the thread ID, while reverted/alternate rollouts carry a distinct rollout ID. Code must not parse a filename suffix and assume it is always the thread ID. + +## 4. Paginated thread history and `includeTurns` + +The app-server protocol is camelCase on the wire. + +### Existing read with optional full hydration + +`codex-rs/app-server-protocol/src/protocol/v2/thread.rs:1487` (`ThreadReadParams`) and line 1497 (`ThreadReadResponse`): + +```text +thread/read params: { threadId, includeTurns? } +thread/read response: { thread } +``` + +`e87e2b495` makes `includeTurns: true` work for paginated stored and loaded threads. The compatibility implementation is intentionally expensive: `codex-rs/app-server/src/request_processors/thread_processor.rs:2806` (`paginated_thread_full_turns`) hydrates all projected turns. It should not be confused with bounded pagination. + +`Thread` exposes experimental `historyMode` at `codex-rs/app-server-protocol/src/protocol/v2/thread_data.rs:222` and `turns` in the same `Thread` struct; turns are populated only for operations that explicitly need them (resume/rollback/fork or read with `includeTurns`). + +### New bounded methods + +Registration is in `codex-rs/app-server-protocol/src/protocol/common.rs:680`: + +```text +thread/turns/list params: + { threadId, cursor?, limit?, sortDirection?, itemsView? } +response: + { data, nextCursor?, backwardsCursor? } + +thread/items/list params: + { threadId, turnId, cursor?, limit?, sortDirection? } +response: + { data: [{ turnId, item }], nextCursor?, backwardsCursor? } +``` + +The exact Rust definitions are `ThreadTurnsListParams/Response` at `codex-rs/app-server-protocol/src/protocol/v2/thread.rs:1518` / line 1537 and `ThreadItemsListParams/Response` at line 1552 / line 1580. `itemsView` values are `notLoaded`, `summary`, and `full` (`codex-rs/app-server-protocol/src/protocol/v2/thread_data.rs:268`, `Turn`, and line 293, `TurnItemsView`). + +Resume also gained bounded bootstrap fields: + +- Request: `excludeTurns?` and `initialTurnsPage?` (`ThreadResumeParams`, `codex-rs/app-server-protocol/src/protocol/v2/thread.rs:326`). +- Page request: `{ limit?, sortDirection?, itemsView? }` (`ThreadInitialTurnsPageParams`, line 466). +- Response: `initialTurnsPage?`, `turnsBackwardsCursor?`, `itemsBackwardsCursor?` (`ThreadResumeResponse`, line 405). + +Current TUI history code requests five turns and up to 100 items at a time, with a 400-item scan cap (`codex-rs/tui/src/app_server_session/history.rs:26`). Its `thread_turns_page` and `thread_items_page` calls are at lines 116–151; `hydrate_initial_thread_history` at line 194 uses bounded pages for paginated threads and retains the full-read fallback for legacy threads. This is the path established by `8bfa49e3` / `3b8d22ec` / `dbcd837c` and hardened by `449f099f1`. + +## 5. Source of the “~98% fewer requests” + +### The proven N+1 + +Commit `332eac4b858a40575a55f21026ecf8cb662e539c` states the defect precisely: loading the summary view issued a separate item query for every returned turn. + +Before that commit, `list_turns` first fetched a page of turn rows, then for `itemsView = Summary` looped over those rows and called `load_summary_items` once per turn. `load_summary_items` issued a separate `SELECT ... FROM thread_items` query for that turn. + +After that commit/current HEAD: + +- `codex-rs/thread-store/src/local/thread_history/read.rs:95` (`list_turns`) consumes `turn.summary_items` returned by paging; only a synthetic inherited fork boundary needs a fallback item query. +- `codex-rs/thread-store/src/local/thread_history/segment_paging.rs:44` (`page_turn_rows`) uses a `page_turns` CTE and two joins to `thread_items` to return each turn's first user item and final agent item in the same query. +- App-server caps `thread/turns/list` at 100 turns (`THREAD_TURNS_DEFAULT_LIMIT` / `THREAD_TURNS_MAX_LIMIT`, `codex-rs/app-server/src/request_processors/thread_processor.rs:4714`); `thread_turns_page_size` at line 4721 defaults to 25 and clamps at 100. + +For a 741-turn flat transcript requested at the 100-turn cap: + +```text +old dominant SQL count: ceil(741 / 100) + 741 = 8 + 741 = 749 +new dominant SQL count: ceil(741 / 100) = 8 +reduction: (749 - 8) / 749 = 98.93% +``` + +This arithmetic explains “~98% fewer requests” without treating the supplied benchmark as verified. Fixed metadata/lineage queries and synthetic fork-boundary fallbacks are omitted from both sides, so this is a mechanism-level count, not an instrumented end-to-end total. + +### What it did not optimize + +- It did not remove one LLM/provider request per conversation turn. There was no such replay loop on `/v1/responses`. +- It optimizes `itemsView = Summary`. The full compatibility path still loads full items per turn: `paginated_thread_full_turns` calls `paginated_turn_full_items`, which loops over turns and calls `thread_store.list_items` (`codex-rs/app-server/src/request_processors/thread_processor.rs:2767`). +- TUI pagination separately reduces app-server RPC payload and startup hydration: five turns/100 items are loaded initially instead of the whole transcript. That is related user-visible latency work, but it is not the N+1 SQL commit. +- `ed390a5d` is an in-place filtering/allocation optimization, not the source of the request reduction. + +The exact `27.6s` and `1.7s` timings, hardware, database cache state, RPC sequence, and profiler trace remain **UNVERIFIED** from the local checkout. + +## 6. Provider-wire impact + +### Core persistence/pagination path is local-only + +The normal Responses request schema remains `ResponsesApiRequest` in `codex-rs/codex-api/src/common.rs:252`: `model`, `instructions`, `input`, `tools`, `tool_choice`, `parallel_tool_calls`, `reasoning`, `store`, `stream`, `stream_options`, `include`, `service_tier`, `prompt_cache_key`, `text`, and `client_metadata`. + +None of the envelope, migration, SQLite projection, `includeTurns`, turn-list, item-list, rollout-ID, or N+1-query commits changes those fields. Before model submission, `ContextManager::for_prompt` maps envelopes back to raw `ResponseItem`s (`codex-rs/core/src/context_manager/history.rs:172`), stripping harness metadata. `3a6f747d7` explicitly keeps harness metadata away from model providers. + +No inspected history commit introduces or changes `store`, `previous_response_id`, or `prompt_cache_key` reuse. In particular, pagination cursors are app-server/local-history cursors; they are not provider continuation IDs. + +### Supplied anchors with provider-visible effects + +1. **Remote compaction (`b9ba969f3`)**: Bedrock gains remote compaction support. Codex sends `POST /v1/responses/compact`. `CompactionInput` at `codex-rs/codex-api/src/common.rs:26` contains exact fields `model`, `input`, `instructions`, `tools`, `parallel_tool_calls`, `reasoning`, `service_tier`, `prompt_cache_key`, and `text`; the endpoint path is `fn path()` at `codex-rs/codex-api/src/endpoint/compact.rs:35` and the sending function is `compact_input` at line 71 [corrected in round-2 audit]. A Responses proxy sees this request. +2. **Image resize notice retention (`4bd5b9fd0`)**: changes which response items stay together in the remote compact request's existing `input` array. It adds no field, but a proxy can observe changed compact-body content. +3. **Guardian parent compaction reuse (`c2bcb9a26`)**: feature-gated Guardian review sessions can start with the parent's encrypted compaction response item. That item becomes review-session model input, so it is provider-visible content with no new request field. +4. **Turn-aware injection (`d6ca19d99`)**: changes when injected response items are associated/flushed in local history. Injected items remain model-visible on a later turn as before; this commit does not introduce a new provider-wire field. +5. **Compaction timing (`bd5b55e4`)**: `compaction_ms` is turn-profile telemetry, not model request data. + + +--- + +Evidence only (LEXICO-SPLIT-01). The prescriptive roadmap that once ended this file +lives in [000_plan.md](000_plan.md), [020_phase2_history_mode_awareness.md](020_phase2_history_mode_awareness.md), +[030_phase3_rollout_identity_and_previews.md](030_phase3_rollout_identity_and_previews.md), and +[050_phase5_compact_wire_verification.md](050_phase5_compact_wire_verification.md). diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/003_opencodex_subagent_catalog_inventory.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/003_opencodex_subagent_catalog_inventory.md new file mode 100644 index 0000000000..014b69c452 --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/003_opencodex_subagent_catalog_inventory.md @@ -0,0 +1,415 @@ +# Lane C — OpenCodex current-state inventory: subagents and model catalog + +## Scope and provenance + +This is a current-state inventory only. It records what the checked-out OpenCodex code models, synthesizes, writes, and forwards today; it makes no recommendation about future changes. + +| Checkout | Verified revision | Evidence | +| --- | --- | --- | +| OpenCodex | `7612e4c4f81544a250c3eea9fe8ca85d8022e765` (`2026-08-16T09:15:49+09:00`) | `git log -1`; tracked working tree was clean before the authorized report write | +| upstream codex-rs | `9dd22890f5ff47e4af128c20e32b9758a61d78d2` (`2026-08-12T14:09:44Z`) | `git -C /Users/jun/Developer/codex/121_openai-codex log -1`; its pre-existing untracked `codex-rs/.codexclaw/` was not read as authority or modified | +| upstream leaf-model commit (context only) | `6d4d9442c7142c08ac5c5098dfd6e82d8cd9f65a`, “Support leaf models in multi-agent v2 (#36892)” | verified with upstream `git show --format=fuller --stat`; this report does not compare or prescribe changes from it | + +The repository search was source-local. Literal-hit inventories below cover tracked `src/`; tests and GUI coverage are inventoried separately. `devlog/` was not used as behavioral authority. + +## Direct answers + +| Question | Current answer | Primary evidence | +| --- | --- | --- | +| What is written for subagents? | A Codex-shaped JSON model catalog whose subagent contract is encoded primarily by `priority`, `visibility`, `multi_agent_version`, and `supported_reasoning_levels`; a root `model_catalog_json` pointer; optional marker-owned `[agents].default_subagent_*`; and native v1/v2 agent settings in `[agents]` / `[features.multi_agent_v2]`. | `src/codex/catalog/sync.ts:435` `buildCatalogEntriesFromObservedState`; `src/codex/catalog/parsing.ts:362` `applyMultiAgentMode`; `src/codex/inject.ts:481` `setRootModelCatalogPath`; `src/codex/subagent-defaults.ts:419` `transformManagedSubagentDefaults`; `src/codex/features.ts:357,750,767,893` | +| Does OpenCodex know v1 vs v2? | **Yes, explicitly.** It has a persisted three-state catalog override (`v1/default/v2`), reads/toggles native `multi_agent_v2`, migrates thread-limit storage, recognizes v1/v2 tool shapes, filters the effective v2 roster, and applies v2-only effort policy. | `src/types.ts:822-827`; `src/codex/catalog/parsing.ts:342-395`; `src/codex/features.ts:121-178,1426-1507`; `src/server/responses/collaboration.ts:152-179`; `src/server/effort-policy.ts:57-80` | +| How does fallback trigger? | Only on requests identified as spawned children by exact Codex headers; it selects from an ordered configured chain using routability, provider/account usability, cooldown, cached failure, and quota checks, then rewrites only `parsed.modelId` and raw-body `model`. | `src/server/effort-policy.ts:20-43`; `src/codex/subagent-model-fallback.ts:269-296,357-362,498-545`; `src/server/responses/core.ts:1727-1784` | +| User surfaces? | OpenCodex JSON config, `ocx agent ...`, `ocx v2 ...`, `/api/subagent-models`, `/api/subagent-model-fallback`, `/api/injection-model`, `/api/effort-caps`, `/api/v2`, and the GUI Subagents/Models controls. | detailed tables below | + +## 1. Exact catalog and Codex-config writes + +### 1.1 Catalog file and pointer + +| Artifact | Produced shape / behavior | Builder or writer | Evidence | +| --- | --- | --- | --- | +| Active catalog JSON | Existing root object is preserved and `catalog.models` is replaced; serialized as pretty JSON plus newline. Abstract type is `{ models?: Record[]; [k: string]: unknown }`. | `prepareCatalog`; `writeRetainedCatalogSync`; `RawCatalog` | `src/codex/catalog/parsing.ts:135-137`; `src/codex/convergence.ts:211-326`; `src/codex/catalog/sync.ts:1485-1535` | +| Catalog target in Codex TOML | Root scalar `model_catalog_json = ""`; removed when no OpenCodex catalog is available. | `setRootModelCatalogPath`; call in `injectCodexConfig` | `src/codex/inject.ts:478-489,744-750` | +| Codex model cache | `{ fetched_at: "2000-01-01T00:00:00Z", client_version: "0.0.0", models: [...] }`, forcing stale reload while retaining hidden account observations. | `invalidateCodexModelsCacheWithPermit` | `src/codex/catalog/sync.ts:1717-1763` | + +`src/codex/catalog.ts` is a 14-line barrel, not a builder. It re-exports the parsing, metadata, effort, and sync symbols (`src/codex/catalog.ts:1-14`). + +### 1.2 Subagent-relevant catalog entry shape + +`RawEntry` is deliberately open (`Record`), so there is no single closed TypeScript interface for serialized rows (`src/codex/catalog/parsing.ts:135`). The following is the union/conditional subagent-relevant projection produced by `deriveEntry` + `ensureStrictCatalogFields` + `applyMultiAgentMode`; template-backed rows may preserve an existing value, while literal values below are the no-template or missing-field defaults: + +```ts +{ + slug: string, // bare native, provider/model, or selector/native + display_name: string, + description: string, + priority: number, // ascending; configured featured models receive the low band + visibility: "list" | "hide", + supported_in_api?: boolean, // no-template fallback: true + shell_type?: string, // no-template fallback: "shell_command" + supported_reasoning_levels: Array<{ + effort: string, + description: string + }>, + default_reasoning_level?: string, + multi_agent_version?: "v1" | "v2" | null, + + // Routed rows: + tool_mode?: "code_mode_only", + supports_search_tool?: boolean, + web_search_tool_type?: "text_and_image", + + // OpenCodex-private extensions ignored by Codex: + opencodex_spawn_priority?: number, // natural priority when display-only order moved the row + opencodex_catalog_kind?: string, + + // Strict parser/default fields also written or healed: + supports_reasoning_summaries: boolean, + default_reasoning_summary: string, + support_verbosity: boolean, + default_verbosity: string, + apply_patch_tool_type: string, + truncation_policy: object, // missing-field default: {mode:"tokens",limit:10000} + supports_parallel_tool_calls: boolean, + supports_image_detail_original: boolean, + experimental_supported_tools: unknown[], + input_modalities: Array<"text" | "image" | "audio">, + context_window: number, + max_context_window: number, + effective_context_window_percent: number, + auto_compact_token_limit: number, + comp_hash: string +} +``` + +| Field/group | Exact production rule | Evidence | +| --- | --- | --- | +| Base routed row | With a native template, clones it, replaces `slug/display_name/description/priority`, sets `visibility="list"`, normalizes native-only fields, applies effort and metadata, then strict fields. Without a template, creates the fallback fields shown above. | `src/codex/catalog/sync.ts:260-376` `deriveEntry` | +| Strict fields | Missing parser booleans/defaults are filled; modalities are restricted to `text/image/audio`; context defaults to 128k for unknown rows. | `src/codex/catalog/parsing.ts:300-340` `ensureStrictCatalogFields` | +| Reasoning ladder | Routed reasoning-capable rows gain mock `max` and `ultra` unless `preserveExact`; `none`-only rows do not. Catalog objects are `{effort, description}`. | `src/codex/catalog/effort.ts:201-240` `applyReasoningLevels`; canonical descriptions `src/reasoning-effort.ts:4-12` | +| Native GPT-5.6 metadata | Loaded from `src/codex/data/upstream-models.json`; current snapshot pins Sol=`v2`, Terra=`v2`, Luna=`v1`, GPT-5.5=`null`. | loader `src/codex/catalog/metadata.ts:114-133,173-177`; literal rows `src/codex/data/upstream-models.json:4,21,118,135,230,247,338,355` | +| Featured roster priority | `featured` becomes a rank map; bare native rows receive rank directly; routed rows receive `rank * priorityStride`; account clones use `rank * selectorCount + selectorIndex`. This is how `subagentModels` affects Codex's advertised first five. | `src/codex/catalog/sync.ts:452-457,517-520,543-570,593-610` | +| Five-row model | OpenCodex's own effective roster takes visible rows, applies v2 compatibility, sorts by natural spawn priority, then `.slice(0, 5)`. | constant `src/codex/catalog/sync.ts:68`; `effectiveSubagentRoster` `src/codex/catalog/sync.ts:142-212` | +| Display-order decoupling | If `modelPickerOrder` moves a non-featured routed row, natural priority is saved as `opencodex_spawn_priority`, while only public `priority` is changed. | `src/codex/catalog/sync.ts:70-80,603-611` | +| Final multi-agent pass | Routed normalization first deletes inherited `multi_agent_version`; final `applyMultiAgentMode` then restores/pins/stamps it. | delete `src/codex/catalog/parsing.ts:398-404`; final build `src/codex/catalog/sync.ts:615-627`; final merge `src/codex/catalog/sync.ts:1084-1102` | + +### 1.3 Three-state `multi_agent_version` output + +Built by `applyMultiAgentMode(entries, mode, v2FeatureEnabled)` (`src/codex/catalog/parsing.ts:362-395`): + +| OpenCodex `multiAgentMode` | Native `multi_agent_v2` flag | Serialized row result | +| --- | --- | --- | +| `"v1"` | either | Every entry receives `multi_agent_version = "v1"`. | +| `"v2"` | either | Every entry receives `multi_agent_version = "v2"`. | +| `"default"` / absent | off | Upstream snapshot pin is restored when present; otherwise the property is deleted. Current pins: Sol/Terra v2, Luna v1, others generally absent/null. | +| `"default"` / absent | on | Upstream snapshot pin is restored when present; otherwise the entry receives `multi_agent_version = "v2"`. A genuine v1 pin remains v1. | + +The mode input is captured in both write paths: retained sync (`src/codex/catalog/sync.ts:1393-1429,1485-1511`) and evidence-bound convergence (`src/codex/convergence.ts:211-230,254-318`). + +### 1.4 OpenCodex-owned JSON config that drives the catalog/runtime + +| Key | Shape/default | Consumer | Evidence | +| --- | --- | --- | --- | +| `subagentModels` | `string[]`, documented max five; fresh default `['gpt-5.5','gpt-5.6-sol','gpt-5.6-terra','gpt-5.6-luna','gpt-5.4-mini']`; explicit `[]` is retained. | catalog ranking and guidance roster | `src/types.ts:667-673`; `src/config.ts:1650-1659,3298-3300`; `src/codex/catalog/sync.ts:1390-1395` | +| `multiAgentMode` | `"v1" | "default" | "v2"`; fresh config omits it, equivalent to default. | final `multi_agent_version` pass and effort gate | `src/types.ts:821-827`; `src/codex/catalog/sync.ts:1396`; `src/server/effort-policy.ts:77-80` | +| `subagentModelFallback` | ordered `string[]` | global runtime fallback chain and optional injected guidance | `src/types.ts:688-693`; `src/codex/subagent-model-fallback.ts:116-145,548-553` | +| `subagentModelFallbackByModel` | `Record` | per-primary fallback stage before global chain | `src/types.ts:694-704`; `src/config.ts:1294-1299`; `src/codex/subagent-model-fallback.ts:427-456` | +| `subagentModelFallbackPollMs` | number; runtime default 60,000 ms; invalid/<1000 falls back to default; API accepts 5,000–600,000. | quota prime/failure health TTL | `src/types.ts:705-708`; `src/codex/subagent-model-fallback.ts:46,99-105`; `src/server/management/agent-settings-routes.ts:669-686` | +| `injectionModel`, `injectionEffort`, `injectionPrompt` | optional strings | proxy-authored guidance; model/effort also feed optional native defaults | `src/types.ts:709-720,758-771`; `src/server/responses/core.ts:1137-1149`; `src/codex/inject.ts:140-155` | +| `multiAgentGuidanceEnabled` | optional boolean; effective default true; fresh config writes true. | suppresses both v1 and v2 OpenCodex-authored guidance when false | `src/types.ts:772-776`; `src/config.ts:3271-3274,3299-3300`; `src/server/responses/collaboration.ts:236-254` | +| `syncCodexSubagentDefaults` | optional boolean; effective only when true and `injectionModel` is nonblank; default off. | authorizes marker-owned native `[agents]` defaults | `src/types.ts:710-714`; `src/config.ts:2245-2249`; `src/codex/inject.ts:140-155` | +| `subagentEffortCap` | optional Codex ladder string | hard cap on header-marked child turns where v2 gate applies | `src/types.ts:783-790`; `src/server/effort-policy.ts:45-80` | + +### 1.5 Codex `config.toml` shapes OpenCodex can write + +#### Marker-owned native defaults + +Built by `configuredManagedSubagentDefaults` and `transformManagedSubagentDefaults` (`src/codex/inject.ts:140-155,778-807`; `src/codex/subagent-defaults.ts:398-550`): + +```toml +# Managed by opencodex: native subagent defaults table +[agents] +# Managed by opencodex: native subagent default +default_subagent_model = "" +# Managed by opencodex: native subagent default +default_subagent_reasoning_effort = "" # omitted when unset +``` + +This write occurs only when `syncCodexSubagentDefaults === true`, `injectionModel` is nonblank, and OpenCodex owns active Codex routing. Unmarked user values are conflicts and are not overwritten (`src/codex/subagent-defaults.ts:461-473`; ownership gates `src/codex/inject.ts:664-686,778-807`). + +#### Native agent/v2 settings + +The `/api/v2` and `ocx v2` surfaces can produce the following keys: + +```toml +[agents] +enabled = true | false +max_depth = # v1-only semantics +max_threads = = 1> # v1 storage; removed while v2 is active + +[features.multi_agent_v2] +enabled = true | false # flag toggle itself is delegated to `codex features` +max_concurrent_threads_per_session = = 1> # v2 total includes root +subagent_developer_instructions = "" +multi_agent_mode_hint_text = "" +``` + +Equivalent `[features] multi_agent_v2 = true|false|{...}` encodings are read and safely upgraded/edited (`src/codex/features.ts:121-178,352-413,883-1003`). Exact writers are `setAgentsEnabled` (`:749-758`), `setAgentsMaxDepth` (`:760-778`), `setMaxConcurrentThreads` (`:351-413`), `setSubagentDeveloperInstructions` (`:1006-1008`), `setMultiAgentModeHintText` (`:1010-1037`), and `transitionMultiAgentV2` (`:1426-1507`). + +OpenCodex **does not currently write** `model_fallback` into `$CODEX_HOME/agents/*.toml`. It only reads legacy role files for compatibility and explicitly places supported per-primary metadata in `subagentModelFallbackByModel` because newer Codex rejects the unknown role field (`src/types.ts:694-704`; `src/codex/subagent-model-fallback.ts:427-456,703-769`). + +## 2. v1/v2, `leaf`, and `spawn_agent` awareness + +### 2.1 Behavioral awareness + +| Mechanism | Current behavior | Evidence | +| --- | --- | --- | +| Catalog mode | Explicit `MultiAgentMode = "v1" | "default" | "v2"`, persisted as `OcxConfig.multiAgentMode`. | `src/codex/catalog/parsing.ts:342`; `src/types.ts:821-827` | +| Native feature | Parses dedicated-table, boolean, and inline `multi_agent_v2`; toggles through native Codex CLI. | `src/codex/features.ts:121-178`; `src/cli/v2.ts:43-86,212-227` | +| Tool-shape detection | Namespaced `spawn_agent` or v1-only companions (`send_input/resume_agent/close_agent`) => v1; flat `spawn_agent` or v2-only companions (`send_message/followup_task/interrupt_agent/list_agents`) => v2; contradictory/no-spawn => null. | `src/server/responses/collaboration.ts:152-179` `collabSurface` | +| Leaf child recognition | Child effort caps do not require collab tools, because depth-limited leaves can have no collaboration tools; exact spawn headers still identify the child. | `src/server/effort-policy.ts:57-80` | +| V2 roster eligibility | OpenCodex treats `multi_agent_version` `v2`, null, or absent as eligible; explicit v1 is excluded. | `src/codex/catalog/sync.ts:90-108` `isEligibleV2SubagentEntry` | +| Prompt inventory | Models `multi-agent-mode` as a feature-gated prompt layer keyed by `features.multi_agent_v2.enabled`; this module inventories it but does not generate runtime guidance. | `src/codex/prompt-layers.ts:87-103` | + +### 2.2 Exact tracked `src/` literal-hit index + +This is the exact grouped output of source-only `git grep`; line lists include comments, API DTO fields, and barrel/plumbing imports as well as executable references. + +#### `multi_agent` + +| File | Lines | +| --- | --- | +| `src/cli/help.ts` | 45 | +| `src/cli/registry.ts` | 313, 316 | +| `src/cli/v2.ts` | 2, 43, 66, 91, 92, 97, 98, 99, 121, 125, 127, 144, 145, 160, 161, 167, 215, 219 | +| `src/codex/catalog/metadata.ts` | 175 | +| `src/codex/catalog/parsing.ts` | 352, 355, 364, 381, 383, 385, 387, 393, 402 | +| `src/codex/catalog/sync.ts` | 93, 106 | +| `src/codex/data/upstream-models.json` | 21, 135, 247, 355, 461, 562, 658, 754 | +| `src/codex/features.ts` | 6, 24, 121, 123, 124, 125, 133, 152, 154, 158, 162, 170, 172, 228, 229, 261, 262, 294, 300, 302, 328, 352, 367, 371, 376, 377, 381, 392, 396, 695, 781, 783, 794, 802, 813, 877, 879, 883, 903, 904, 906, 909, 912, 922, 926, 946, 950, 985, 988, 994, 1001, 1011, 1014, 1020, 1036, 1054, 1082, 1245, 1264, 1272, 1283, 1289, 1343, 1346, 1398, 1401, 1408, 1410, 1412, 1413, 1419, 1427, 1477, 1489 | +| `src/codex/prompt-layers.ts` | 7, 102 | +| `src/server/effort-policy.ts` | 6 | +| `src/server/management/agent-settings-routes.ts` | 218, 290, 327, 362, 365 | +| `src/server/responses/collaboration.ts` | 346, 365, 372, 461, 464 | +| `src/types.ts` | 760 | + +#### `multiAgent` + +| File | Lines | +| --- | --- | +| `src/cli/agent.ts` | 66 | +| `src/cli/v2.ts` | 95, 112, 193, 194, 203 | +| `src/codex/catalog/sync.ts` | 387, 392, 407, 422, 427, 442, 447, 627, 734, 735, 764, 765, 1089, 1090, 1116, 1155, 1156, 1396, 1421, 1429, 1434, 1474, 1479, 1498, 1499 | +| `src/codex/convergence.ts` | 48, 181, 205, 216, 229, 230, 261, 266, 276, 281, 306, 307, 394 | +| `src/codex/features.ts` | 130, 134 | +| `src/config.ts` | 1284, 3271, 3272, 3274, 3300 | +| `src/server/effort-policy.ts` | 63, 78 | +| `src/server/index.ts` | 970 | +| `src/server/management-api.ts` | 10 | +| `src/server/management/agent-settings-routes.ts` | 11, 234, 238, 248, 252, 257, 261, 263, 266, 267, 291, 292, 293, 295, 298, 309, 312, 331, 332, 347, 375, 379, 458, 477, 485, 493, 494, 495, 497, 535, 548 | +| `src/server/management/combo-routes.ts` | 10 | +| `src/server/management/config-routes.ts` | 10 | +| `src/server/management/logs-usage-routes.ts` | 10 | +| `src/server/management/model-routes.ts` | 81 | +| `src/server/management/oauth-account-routes.ts` | 10 | +| `src/server/management/provider-routes.ts` | 12 | +| `src/server/management/shared.ts` | 10 | +| `src/server/responses.ts` | 6 | +| `src/server/responses/collaboration.ts` | 5, 184, 236, 241 | +| `src/server/responses/compact.ts` | 5 | +| `src/server/responses/core.ts` | 8, 213, 1138, 1139, 1150 | +| `src/server/responses/encrypted-payload.ts` | 5 | +| `src/server/responses/fetch-helpers.ts` | 11 | +| `src/types.ts` | 776, 827 | + +Many line-10-style management/response hits are imports from barrel modules, not separate behavior owners. + +#### `agents.max_threads` and `max_concurrent_threads_per_session` + +| Token | Exact `src/` hits | +| --- | --- | +| `agents.max_threads` | `src/codex/features.ts:228,1417`; `src/server/management/agent-settings-routes.ts:218` | +| `max_concurrent_threads_per_session` | `src/cli/registry.ts:318`; `src/cli/v2.ts:127,167`; `src/codex/features.ts:24,294,303,304,352,359,381,387,390,391,402,407,411,1251,1267,1269,1270,1289,1344,1347,1412,1420` | + +#### whole-word `leaf` + +There are 23 tracked `src/` whole-word hits. Only `src/server/effort-policy.ts:68` is a multi-agent semantic hit (“leaf guard”). The rest use “leaf” for JSON/config path segments or module-architecture descriptions: + +| File | Lines | +| --- | --- | +| `src/cli/config-command.ts` | 56, 57, 58, 59 | +| `src/codex/injected-marker.ts` | 5 | +| `src/config.ts` | 3091, 3099, 3176 | +| `src/integrations/merge.ts` | 65, 66, 67 | +| `src/integrations/omp-yaml-source.ts` | 6, 111, 295 | +| `src/integrations/registry.ts` | 45 | +| `src/integrations/state.ts` | 62, 65, 140, 166 | +| `src/integrations/writer.ts` | 393 | +| `src/lib/upstream-reachability.ts` | 23 | +| `src/lib/upstream-retry.ts` | 14 | +| `src/server/effort-policy.ts` | 68 | + +#### `spawn_agent` + +| File | Exact lines | +| --- | --- | +| `src/codex/catalog/effort.ts` | 211 | +| `src/codex/catalog/parsing.ts` | 354 | +| `src/codex/catalog/provider-fetch.ts` | 1833 | +| `src/codex/catalog/sync.ts` | 72, 76, 79, 453, 458, 462, 476, 563, 596, 603 | +| `src/config.ts` | 1652 | +| `src/server/effort-policy.ts` | 5 | +| `src/server/management/agent-settings-routes.ts` | 442, 584 | +| `src/server/responses/collaboration.ts` | 164, 173, 259, 270, 349 | +| `src/types.ts` | 676, 683, 716 | + +## 3. Quota-aware subagent model fallback + +### 3.1 Trigger and call path + +| Stage | Exact behavior | Evidence | +| --- | --- | --- | +| Spawn classification | True only when `x-openai-subagent` is exactly `collab_spawn`, or parsed `x-codex-turn-metadata.subagent_kind` is exactly `thread_spawn`; malformed JSON and other subagent categories are false. | `src/server/effort-policy.ts:20-43` `isThreadSpawnRequest` | +| Entry point | `handleResponses` computes `threadSpawn`; exact account selectors skip pool-wide priming/fallback. Non-combo, non-fixed-account spawned turns run quota priming, preview account selection, then fallback before route-dependent normalization. | `src/server/responses/core.ts:1722-1784` | +| Encrypted assignment | Initial fallback can be restricted to canonical native ChatGPT routes while task ciphertext is unreadable; after recovery, selection reruns with the full configured chain. | `src/server/responses/core.ts:1768-1776,1853-1875`; `src/codex/subagent-model-fallback.ts:275-288` | +| Main turn | No spawn headers => `applySubagentModelFallback` returns null without inspecting chains. | `src/codex/subagent-model-fallback.ts:510-520` | + +### 3.2 Chain order and availability + +For a primary model `P`, the effective order is: + +```text +P +→ config.subagentModelFallbackByModel[P] +→ config.subagentModelFallback +→ legacy $CODEX_HOME/agents/*.toml model_fallback values for roles whose model matches P +``` + +The chain trims empties and de-duplicates; ordinary model ids are case-insensitive for de-duplication, while configured account-selector prefixes remain case-sensitive (`src/codex/subagent-model-fallback.ts:107-145,396-456,510-539`). + +| Candidate rejection | Rule | Evidence | +| --- | --- | --- | +| Disabled | Matches `disabledModels`, including bare native, provider/model, and selector-qualified semantics. | `src/codex/subagent-model-fallback.ts:80-97` | +| Unroutable/disabled provider | Route fails, provider is disabled, or an unknown slash prefix is neither configured nor a known registry provider id. | `src/codex/subagent-model-fallback.ts:63-69,184-198,241-245` | +| Cached health block | Quota/rate-limit failures mark `(account when pool-scoped, model)` unavailable until poll TTL. | `src/codex/subagent-model-fallback.ts:216-232,298-322` | +| Pool account unavailable | No resolved account, paused, unusable/reauth-required, fixed-account model cooldown, or account cooldown without a probe lease. | `src/codex/subagent-model-fallback.ts:246-266` | +| Quota threshold | Known usage score at/above `autoSwitchThreshold` (default 80) rejects; unknown usage does not. | `src/codex/subagent-model-fallback.ts:147-150,200-214` | +| Encrypted native-only pass | Rejects candidates whose resolved provider is not canonical OpenAI Codex forward. | `src/codex/subagent-model-fallback.ts:275-288` | + +The first available candidate wins. If every candidate is skipped, the original primary is retained and the skipped list is returned (`src/codex/subagent-model-fallback.ts:269-296`). + +### 3.3 Priming, failure feedback, and rewrite shape + +| Mechanism | Current behavior | Evidence | +| --- | --- | --- | +| Quota priming | Single-flight best-effort `primeCodexPoolQuotas(config, "subagent-spawn")`; success cached for poll interval; blocked when native-main reads are forbidden or ChatGPT host circuit is open; failures are swallowed and remain retryable. | `src/codex/subagent-model-fallback.ts:421-496,780-784` | +| Failure feedback | On spawned-child failures classified as quota/rate-limit (including 429/402 paths), selected model is health-blocked for poll TTL. | `src/codex/subagent-model-fallback.ts:298-322,498-508`; call sites `src/server/responses/core.ts:2555-2562,2724-2731,2805-2812,3761-3770` | +| Rewrite | Mutates `parsed.modelId` and `_rawBody.model`; does not rewrite effort. | `src/codex/subagent-model-fallback.ts:357-362,543-545`; documented SOT `structure/03_catalog-and-subagents.md:231-234` | +| Guidance | Global fallback list can be rendered into v2 OpenCodex guidance; it does not perform selection itself. | `src/codex/subagent-model-fallback.ts:548-553`; `src/server/responses/collaboration.ts:339-360` | +| Account selection dependency | Uses `previewCodexAccountForRequest` / quota-health helpers from `src/codex/routing.ts`; routing itself does not define multi-agent surfaces. | preview SOT comment `src/codex/routing.ts:1373-1379`; call `src/server/responses/core.ts:1757-1767` | + +## 4. Test coverage in `tests/*.test.ts` + +The table lists direct behavioral suites, not files with incidental words in unrelated fixtures. + +| File | Covered scenarios (test-name evidence) | +| --- | --- | +| `tests/codex-v2-gate.test.ts` | native/route ultra ladders (`:100-139`); feature TOML readers (`:140-298`); v2 thread reader/writer (`:299-387`); mode-hint read/write/capability probe (`:388-769`); v1/v2 thread-limit migration and root-slot translation (`:770-980`); `[agents]` and subagent-instruction parity (`:981-1191`); management and CLI v2 surfaces (`:1192-1741`); three-state catalog mode and stale-pin restoration (`:1781-1986`). | +| `tests/codex-catalog.test.ts` | pinned GPT-5.6 exact per-slug specs (`:2610-2669`); snapshot upgrades and priority preservation (`:2995-3048`); catalog effort normalization and model metadata throughout the larger catalog suite. | +| `tests/codex-catalog-model-picker-order.test.ts` | display ordering does not displace or change spawn candidates, including all-routed reverse ordering (`:52-184`). | +| `tests/native-alias-maintainer-regressions.test.ts` | native aliases retain upstream multi-agent pins in default mode (`:106-121`). | +| `tests/multi-agent-compat.test.ts` | v1/v2 tool-shape classification; v1 top-tier guidance; v2 catalog-freshness gate; candidate/advertised roster intersection, account projection, five-item cap, visibility/version exclusions; injection/fallback placeholders; guidance kill switch; developer-message placement (`:97-1104`). | +| `tests/effort-policy.test.ts` | exact child-header detection (`:54-83`); child/global caps; v2 gating including tool-less leaf children and forced-v1 kill switch (`:319-421`); `/api/effort-caps` (`:442-487`). | +| `tests/subagent-defaults.test.ts` | marker-owned `[agents]` create/update/remove; escaping and CRLF; user-owned conflicts; ambiguous TOML rejection; table ownership (`:12-380`). | +| `tests/codex-inject-integration.test.ts` | opt-in native defaults, removal/restore, user-owned conflict, residue cleanup, ambiguous-marker refusal, and proof injection does not enable v2 (`:222-342,577-585`). | +| `tests/injection-model-api.test.ts` | model/effort/prompt round trips; invalid input no-mutation; guidance kill switch partial updates; native-default opt-in/model binding/normalization; persistence across catalog sync/reload (`:45-415`). | +| `tests/subagent-model-fallback.test.ts` | chain ordering/dedup; quota/account/health/disabled/routability filters; native-only encrypted selection; priming TTL/single-flight/failure; request rewrite vs main no-op; config-keyed and legacy TOML per-role chains; malformed/quoted TOML; guidance text (`:121-1170`). | +| `tests/subagent-model-fallback-api.test.ts` | atomic validation: invalid/empty entry leaves prior config intact; valid chain accepted (`:50-96`). | +| `tests/subagent-fallback-handle-responses.test.ts` | exact-account bypass; cooled primary; final-route normalization; native↔routed fallback; account preview; encrypted native-only behavior; terminal 429/402 health recording (`:201-1003`). | +| `tests/agent-task-recovery-fallback.test.ts` | recovered encrypted task routes through healthy routed fallback (`:17-66`). | +| `tests/cli-headless-parity.test.ts` | headless `ocx agent effort` and `ocx agent subagents` use the same live routes as GUI (`:314-323`). | +| `tests/codex-sync-api.test.ts` | native-default conflict warnings propagate through sync (`:375-415`) and preflight/refusal protects catalog/config (`:126-229`). | +| `tests/native-model-toggle.test.ts` | `/api/subagent-models.available` removes disabled native slugs (`:471-533`). | + +Related GUI tests live under `gui/tests/`, outside the requested `tests/*.test.ts` glob: `subagents-classic.test.tsx`, `subagents-busy-race.test.tsx`, `subagents-ultra-mode.test.tsx`, and `multi-agent-guidance.test.tsx` cover the Subagents workspace, save-race guard, Ultra mode, and guidance toggle. + +## 5. User configuration surfaces + +### 5.1 CLI + +| Command | Payload / effect | Evidence | +| --- | --- | --- | +| `ocx agent status [--json]` | GETs `/api/v2`, `/api/injection-model`, `/api/effort-caps`, `/api/subagent-models`, `/api/subagent-model-fallback`, and sidecars. | `src/cli/agent.ts:30-44` | +| `ocx agent subagents status|set|clear` | GET/PUT `/api/subagent-models`; set takes comma-separated ids; CLI rejects >5. Alias: `roster`. | `src/cli/agent.ts:94-115,171-180` | +| `ocx agent fallback status|set|clear [models] [--poll-ms]` | GET/PUT `/api/subagent-model-fallback`; API range 5,000–600,000 ms. | `src/cli/agent.ts:117-142` | +| `ocx agent injection status|set --model --effort --prompt --guidance` | GET/PUT `/api/injection-model`; `-` clears string selections. Alias: `guidance`. The CLI does **not** expose `syncCodexSubagentDefaults`; GUI/API/config do. | `src/cli/agent.ts:46-70,171-180` | +| `ocx agent effort status|set --main --subagent` | GET/PUT `/api/effort-caps`. | `src/cli/agent.ts:72-92` | +| `ocx v2 status|on|off|mode v1|default|v2|threads N|mode-hint ...` | Reads/toggles native v2, migrates thread limit, persists catalog mode, resyncs catalog, edits mode hint. | `src/cli/v2.ts:89-230` | + +Exact account-qualified roster ids, which the GUI does not offer, are intentionally available through `ocx agent subagents set` or direct config (`structure/03_catalog-and-subagents.md:220-229`). + +### 5.2 Management API + +| Route | GET shape | PUT shape/effect | Evidence | +| --- | --- | --- | --- | +| `/api/subagent-models` | `{ chosen, available, catalogState }` | `{models}` → string values sliced to first five, persists `subagentModels`, converges catalog, syncs Claude defs/Desktop best-effort; response `{ok, applied, catalogRefresh}`. | `src/server/management/agent-settings-routes.ts:584-618` | +| `/api/subagent-model-fallback` | `{ models, pollMs, available }` | optional `{models, pollMs}` with full validation; persists global chain/poll interval. | `src/server/management/agent-settings-routes.ts:620-687` | +| `/api/injection-model` | `{multiAgentGuidanceEnabled,syncCodexSubagentDefaults,model,effort,prompt,efforts,available}` | partial patch of those settings; clearing model also clears effort and native-default sync. | `src/server/management/agent-settings-routes.ts:440-554` | +| `/api/effort-caps` | `{effortCap,subagentEffortCap,efforts}` | partial set/clear of main/global and child-only caps. | `src/server/management/agent-settings-routes.ts:556-582` | +| `/api/v2` | `{enabled,agentsMaxThreadsConflict,maxConcurrentThreadsPerSession,multiAgentMode,agentsEnabled,agentsMaxDepth,subagentDeveloperInstructions,multiAgentModeHintText,agentsMaxDepthAppliesWhenV2Disabled}` | partial flag/mode/thread/agent-depth/instruction/hint write, catalog convergence, warnings. | `src/server/management/agent-settings-routes.ts:218-383` | + +### 5.3 GUI + +| Panel/control | Endpoint | Exact behavior | Evidence | +| --- | --- | --- | --- | +| Subagents page — Featured roster | `/api/subagent-models` | Loads available/chosen, lets user toggle and reorder up to `FEATURED_MAX=5`, PUTs `{models: chosen}`. | `gui/src/pages/Subagents.tsx:116-176`; `gui/src/components/subagents-workspace/SubagentsWorkspace.tsx:56-71,88-149,152-213` | +| Subagents page — preferred model/effort | `/api/injection-model` | Selects `injectionModel` and `injectionEffort`. | `gui/src/pages/use-subagent-delegation.ts:42-92`; `gui/src/components/subagents-workspace/SubagentDelegationSection.tsx:66-97` | +| Subagents page — native defaults toggle | `/api/injection-model` | Toggles `syncCodexSubagentDefaults`; disabled when no model is selected. | `gui/src/components/subagents-workspace/SubagentDelegationSection.tsx:99-114` | +| Subagents page — guidance toggle | `/api/injection-model` | Toggles `multiAgentGuidanceEnabled`. | `gui/src/components/subagents-workspace/SubagentDelegationSection.tsx:116-131` | +| Subagents page — Ultra mode/hint | `/api/v2` | Writes/clears `multiAgentModeHintText`; enabling is allowed only when native v2 is enabled and catalog mode is explicitly v2. | `gui/src/pages/Subagents.tsx:28-99`; `gui/src/components/subagents-workspace/SubagentDelegationSection.tsx:133-169,227-229` | +| Models page — surface mode | `/api/v2` | Segmented `multiAgentMode` control for v1/default/v2. | `gui/src/pages/Models.tsx:776-793,1268-1281` | + +There is **no GUI control for `subagentModelFallback`** in tracked `gui/src` (zero `subagent-model-fallback` hits). It is exposed through CLI, management API, and direct OpenCodex config. + +### 5.4 Direct files users can edit + +| File | User-owned surface | OpenCodex behavior | +| --- | --- | --- | +| OpenCodex `config.json` | all `OcxConfig` keys above | loaded/preserved by `src/config.ts`; drives catalog/runtime | +| `$CODEX_HOME/config.toml` | `[agents]`, `[features.multi_agent_v2]`, root catalog pointer | OpenCodex uses scoped, format-preserving writers; marker-owned native defaults never overwrite unmarked user keys | +| `$CODEX_HOME/agents/*.toml` | role `model`; legacy `model_fallback` may exist | fallback reads matching roles; does not write `model_fallback` | + +## 6. Production module inventory + +| Module | Role in this surface | Writes? | Evidence | +| --- | --- | --- | --- | +| `src/codex/catalog.ts` | Barrel exporting catalog parsing/metadata/effort/sync APIs. | No direct write | `:1-14` | +| `src/codex/catalog/parsing.ts` | Defines open catalog types, strict row shape, routed normalization, three-state `multi_agent_version`. | Mutates candidate rows | `:94-137,300-404` | +| `src/codex/catalog/effort.ts` | Builds/reads `supported_reasoning_levels`; adds mock top rungs; clamps against observed Codex support. | Mutates candidate rows | `:84-113,201-274,309-384` | +| `src/codex/catalog/metadata.ts` | Loads pinned upstream native rows and exposes native multi-agent pins/ladders/context. | No file write in relevant path | `:102-177` | +| `src/codex/catalog/sync.ts` | Builds rows, ranks featured models, computes effective five-model roster, merges/writes catalog and cache. | Yes | `:68-212,401-627,751-1168,1355-1535,1717-1763` | +| `src/codex/convergence.ts` | Evidence-bound alternative catalog gather/prepare/commit path; consumes same featured/mode/v2 inputs and same builders. | Yes | `:172-208,211-326,524-577` | +| `src/codex/sync.ts` | Orchestrates refresh before Codex config injection; forwards native-default warnings and desired-state skips. | Through dependencies | `:60-216` | +| `src/codex/subagent-defaults.ts` | Pure marker-aware TOML transform for native default model/effort. | Returns candidate content; caller writes | `:1-41,398-550` | +| `src/codex/features.ts` | Reads/writes native v1/v2 flag-adjacent config, threads, agent keys, subagent instructions, mode hint; migrates thread units. | Yes | `:121-178,226-413,749-1037,1205-1507` | +| `src/codex/subagent-model-fallback.ts` | Builds chain, checks availability/quota/health, primes quota, reads legacy role metadata, rewrites spawned-child model. | Runtime request mutation; no catalog/config write | `:1-145,200-322,357-545,703-784` | +| `src/codex/routing.ts` | Supplies effective account, quota health, cooldown, and side-effect-free preview used by fallback. It does not select v1/v2. | Runtime state only | `:673-719,1361-1467` | +| `src/codex/prompt-layers.ts` | Inventories Codex prompt layers including v2 multi-agent mode; owns generic prompt toggles/projection, not delegation guidance. | Generic prompt config writes elsewhere in module | `:1-27,87-117,384-460` | +| `src/server/responses/collaboration.ts` | Detects v1/v2 tool surface, computes effective roster/guidance, injects fallback text. | Returns guidance | `:144-191,216-373` | +| `src/server/responses/core.ts` | Calls guidance and effort policy; primes/applies fallback before final routing; records quota failures. | Mutates proxied request | `:1137-1160,1722-1784,2535-2563,3761-3770` | +| `src/server/effort-policy.ts` | Exact spawned-child header predicate and v2-only/global/subagent effort cap policy, including tool-less leaves. | Mutates request effort through `applyEffortCap` | `:20-80,106-190` | +| `src/server/management/agent-settings-routes.ts` | Owns all listed management routes and persistence/convergence calls. | Yes | `:218-383,440-687` | +| `src/reasoning-effort.ts` | Canonical low→ultra ladder and configured ladder sanitation used by catalog/API/caps. | No | `:4-43,68-111` | +| `src/providers/openai-tiers.ts` | Migrates legacy `openai-multi/` ids in `subagentModels` and `injectionModel` to current bare identity. | Mutates migration projection/config | `:111-120,172-197` | +| `src/config.ts` / `src/types.ts` | Defines persisted OpenCodex fields, defaults, validation/degradation, and effective flags. | OpenCodex config writer elsewhere in file | `src/types.ts:667-720,758-790,821-827`; `src/config.ts:1284-1299,1650-1659,2047-2114,2245-2249,3271-3304` | +| `src/cli/agent.ts` | Headless API client for roster, fallback, injection/guidance, and effort caps. | Via management API | `:16-24,30-142,171-184` | +| `src/cli/v2.ts` | Native v2 flag/mode/thread/hint CLI and catalog resync. | Yes | `:89-230` | +| `structure/03_catalog-and-subagents.md` | Repository SOT matching current catalog, three-state mode, first-five roster, fallback, and native defaults. | Documentation | `:82-96,133-153,218-243` | +| `structure/08_openai-provider-tiers.md` | Identity/migration SOT: legacy selected ids rewrite; Pool/Direct does not alter bare catalog ids; selected virtual ids remain in subagent state. | Documentation | `:69-89,108-122` | + +## Inventory boundary + +- Catalog generation and request-time collaboration are separate: the catalog controls what Codex sees/validates; `collaboration.ts` only adds OpenCodex-authored developer guidance. +- Fallback is request-time and header-gated; it does not change the configured roster or catalog. +- `modelPickerOrder` is display-only by design; `subagentModels` is the spawn-candidate ranking input. +- Native default synchronization is opt-in and independent of guidance enablement. +- No claim in this report is based on an unverified external page or search snippet. diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/004_opencodex_history_responses_inventory.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/004_opencodex_history_responses_inventory.md new file mode 100644 index 0000000000..d65505a671 --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/004_opencodex_history_responses_inventory.md @@ -0,0 +1,239 @@ +# Lane D — OPENCODEX current-state inventory: history, rollout, usage, and Responses + +## Evidence baseline and scope + +| Checkout | Verified commit | Evidence | +|---|---|---| +| OPENCODEX | `7612e4c4f81544a250c3eea9fe8ca85d8022e765` | `git rev-parse HEAD`; commit subject `fix(routing): source capability evidence from explicit catalog provenance (#1799)` | +| UPSTREAM codex-rs | `9dd22890f5ff47e4af128c20e32b9758a61d78d2` | `git -C /Users/jun/Developer/codex/121_openai-codex rev-parse HEAD`; commit subject `Add an LRU baseline to skill shadow selection (#38197)` | + +This is an inventory of OPENCODEX only. No upstream behavior is inferred here. “Conversation history” is separated into three distinct stores because OPENCODEX treats them differently: + +| Store | Owner | Contents | Used on live `/v1/responses` request path? | +|---|---|---|---| +| `$CODEX_HOME/sessions/**/rollout-*.jsonl`, `$CODEX_HOME/archived_sessions/*` and `state_5.sqlite` | Codex | Codex thread/session history and listing metadata | No. Read or modified by integration migration/residue/storage workflows, not to construct ordinary Responses requests. | +| `$OPENCODEX_HOME/responses-state.json` and `responses-state-spill/` | OPENCODEX | Bounded `previous_response_id` continuation items plus provider-private continuation state | Yes. Expanded before request parsing. | +| `$OPENCODEX_HOME/usage.jsonl` and `routing-history.sqlite` | OPENCODEX | Request metadata, route attempts, status, timing, normalized usage; no prompts | No prompt replay. Used for usage summaries and management request-history views. | + +`$CODEX_HOME` resolves from `CODEX_HOME`, otherwise the platform default `~/.codex`; `sqlite_home` in Codex `config.toml`, then `CODEX_SQLITE_HOME`, can relocate `state_5.sqlite` (`src/codex/paths.ts:6` `resolveCodexHome`, `src/codex/paths.ts:26` constants, `src/codex/paths.ts:76` `resolveCodexSqliteHome`, `src/codex/paths.ts:107` `resolveCodexStateDbPath`). `$OPENCODEX_HOME` is separate and defaults to `~/.opencodex` (`src/config.ts:643` `resolveConfigDir`, `src/config.ts:1661` `getConfigDir`). + +## 1. Codex rollout/session JSONL parsing + +### Direct answer + +**Yes.** OPENCODEX parses rollout JSONL in two production subsystems, delegates that parser into a third, and has one isolated runtime-smoke script that parses a temporary Codex session: + +1. history-provider migration/restore (`src/codex/history-provider.ts`), +2. native-residue classification (`src/codex/native-residue.ts`), and +3. archived-session restore, which calls `readThreadFieldsFromRollout` (`src/storage/cleanup.ts`), and +4. the OpenAI-provider-option runtime smoke (`scripts/openai-provider-option-runtime-smoke.ts`). + +The parsers are schema-sensitive. They require the current top-level envelope `{type, payload}` and specific payload keys. A new envelope that moves or renames `type`/`payload` would not be understood: + +| Consumer | Required current shape | Failure behavior if envelope changes | +|---|---|---| +| `parseSessionMetaLine` | top-level `type === "session_meta"`; object `payload` | Returns `null`; provider migration skips the rollout or restore cannot reconstruct fields (`src/codex/history-provider.ts:355`). | +| `extractUserMessagePreview` | top-level `event_msg` with payload `type/message/content`, or top-level `response_item` with payload message/role/content | Returns no preview; reconstructed `first_user_message` becomes empty (`src/codex/history-provider.ts:419`). | +| `rolloutSessionMetaPayload` | every nonblank line must parse as an object; session metadata is top-level `type === "session_meta"` with object `payload` | Marks classification `indeterminate` on malformed/unknown session-meta payload; an envelope that hides the record type is treated as a non-session-meta line, eventually “no session_meta metadata” (`src/codex/native-residue.ts:156`, `src/codex/native-residue.ts:502`). | + +No rollout parser found in the ordinary `/v1/responses` ingress or adapter path. The only response-history expansion there is the OPENCODEX continuation store described later. + +### Rollout/session inventory + +| File and symbol | Read/write | Exact extraction or reasoning | Scope and bounds | +|---|---|---|---| +| `src/codex/history-provider.ts:355` `parseSessionMetaLine` | Parse | Parses one JSONL line; accepts only `type: "session_meta"` plus object `payload`. Exposes the whole payload and reads `model_provider`/`source`. | Malformed or other record types return `null`. | +| `src/codex/history-provider.ts:374` `readLatestSessionMeta` | Read/parse | Reads the whole plain rollout, scans backward, returns the last parseable `session_meta` (last-writer-wins). | No size cap in this function. | +| `src/codex/history-provider.ts:405` `textFromContentParts` | Parse | Extracts `text` or `input_text` strings from content arrays. | Used only for first-user-message reconstruction. | +| `src/codex/history-provider.ts:420` `extractUserMessagePreview` | Parse | Extracts the first user preview from `event_msg.payload.message/content` or `response_item.payload` where `type: message`, `role: user`. | Does not reconstruct full conversation history. | +| `src/codex/history-provider.ts:459` `readThreadFieldsFromRollout` | Read/parse | Reads plain JSONL or bounded zstd JSONL, then reconstructs thread listing fields. | `.jsonl.zst` decompression is capped at 64 MiB (`src/codex/history-provider.ts:9`, `src/codex/history-provider.ts:472`). | +| `src/codex/history-provider.ts:486` `parseThreadFieldsFromRolloutText` | Parse | Last session meta: `id`, `model_provider` (default `openai`), `source` (default `cli`), optional `cwd`, `history_mode`, `cli_version`; first user preview; derives `hasUserEvent`. | Full text split into lines; returns `null` without a parseable session meta/id. | +| `src/codex/history-provider.ts:72` `appendRolloutLine` | Write | Appends one JSONL line with `O_APPEND`; fsync best effort. | Used for provider/source migration metadata, not conversation turns. | +| `src/codex/history-provider.ts:107` `patchFirstLineProviderInPlace` | Read/write/parse | Reads line 1, verifies session id, length-preservingly rewrites `model_provider`, reparses, writes at offset 0. | Hard stop at 16 MiB for a newline-less/corrupt first line (`src/codex/history-provider.ts:111`). | +| `src/codex/history-provider.ts:530` `updateSessionMeta` | Read/write/reason | Clones the latest session-meta record, verifies `payload.id`, changes only provider/source, refreshes timestamp, patches line 1 best effort, appends the new record. | An id mismatch or unparseable latest meta makes it a no-op. | +| `src/codex/history-provider.ts:689` `syncCodexHistoryProvider` | Read/write | Coordinates `state_5.sqlite` thread rows and rollout metadata for legacy `openai`↔`opencodex` history tagging. | Integration migration/restore only. Uses DB busy timeout/retry. | +| `src/codex/history-provider.ts:705` `syncCodexHistoryProviderUnsafe` | Read/write | Selects `threads(id, rollout_path, model_provider, source, has_user_event)`, backs up originals, updates rollouts and rows. | Only resumable `cli`/`vscode` or selected legacy `exec` rows. | +| `src/codex/history-provider.ts:780` `restoreCodexHistoryProvider` | Read/write | Restores backed-up provider/source/user-event values and updates referenced rollouts. | Backup is OPENCODEX-owned `codex-history-backup-.json` (`src/codex/history-provider.ts:23`). | +| `src/codex/native-residue.ts:156` `rolloutSessionMetaPayload` | Parse | Validates JSON object envelope and session-meta payload shape. | Unknown non-session-meta record types are ignored; malformed lines fail closed. | +| `src/codex/native-residue.ts:176` `consumeRolloutLines` | Parse | Streams lines and retains first and latest session-meta payloads. | Does not retain conversation items. | +| `src/codex/native-residue.ts:416` `classifyReferencedRollout` | Read/parse/reason | Reads the rollout in 64 KiB chunks, validates file identity/races, requires first/latest ids and provider metadata, reports residue if either provider is `opencodex`. | Refuses rollouts over 64 MiB (`src/codex/native-residue.ts:82`, `src/codex/native-residue.ts:439`). | +| `src/codex/native-residue.ts:531` `classifyHistoryDatabase` | Read/parse/reason | Reads all `threads(id, rollout_path, model_provider)`, validates referenced rollouts, and detects routed residue. | Read-only DB open; any schema/read uncertainty is `indeterminate`. | +| `src/storage/cleanup.ts:222` `normalizeArchivedRolloutPath` | Path reasoning | Accepts only one file directly under `archived_sessions/`, normalizes `.jsonl.zst` to logical `.jsonl`, explicitly rejects active `sessions/`. | No content read. | +| `src/storage/cleanup.ts:276` `listArchivedCandidates` | Read metadata | Lists archived `.jsonl`/`.jsonl.zst`, groups physical variants, orders by mtime/size. | Never walks active sessions. | +| `src/storage/cleanup.ts:444` `collectPinnedArchivedRolloutPaths` | DB/path reasoning | Reads pinned thread rollout paths and excludes matching archived candidates. | Content not parsed. | +| `src/storage/cleanup.ts:2297` `reconstructThreadRowFromRollout` | Read/parse | Calls `readThreadFieldsFromRollout` to reconstruct old quarantined thread rows. | Used only when a legacy cleanup manifest lacks complete satellite snapshots. | +| `src/storage/cleanup.ts:2484` legacy restore loop | Read/parse/write | Locates restored plain/zstd rollout and reconstructs required `threads` columns. | Fails restore if required session metadata cannot be reconstructed. | +| `src/storage/scanner.ts:89` `walkFiles`, `src/storage/scanner.ts:172` `scanStorage` | Read metadata | Measures active/archived session bytes/counts/mtime/largest paths; opens newest state/log DB immutably for row counts. | Does not parse JSONL content; performs zero writes (`src/storage/scanner.ts:17`). | +| `scripts/openai-provider-option-runtime-smoke.ts:343` isolated runtime evidence | Read/parse | Recursively finds the one temporary session JSONL, parses every line, extracts first `session_meta.payload` and first `turn_context.payload`, then asserts `turn_context.model` and `session_meta.model_provider` (`scripts/openai-provider-option-runtime-smoke.ts:350`). | Diagnostic/final-gate script only; schema-sensitive and would throw if the expected temporary rollout/envelope changed. | +| `src/codex/inject.ts:1012` history-job dispatch | Orchestration | Integration apply/restore can schedule the history-provider job after Codex config/catalog handling. | Not part of request ingress. | +| `src/codex/sync.ts:60` `syncModelsToCodex` | Orchestration | Syncs Codex catalog/config and reports whether config, catalog, cache, or history changed. | It does not parse rollouts itself. | + +## 2. `~/.codex` state inventory relevant to this lane + +| Surface | Owner/symbol | Current behavior | +|---|---|---| +| Home/path resolution | `src/codex/home.ts:135` `defaultCodexHome`; `src/codex/home.ts:143` `resolveCodexHomeDir` | Uses explicit `CODEX_HOME` or platform `~/.codex`; has WSL Windows-profile discovery. | +| Canonical paths | `src/codex/paths.ts:26` | Defines `config.toml`, `opencodex.config.toml`, `opencodex-catalog.json`, `models_cache.json`. | +| SQLite root | `src/codex/paths.ts:76` | Parses root `sqlite_home`; precedence is Codex config, `CODEX_SQLITE_HOME`, effective home. | +| Codex config/profile | `src/codex/inject.ts:641` `injectCodexConfig` | Reads `config.toml`, computes managed routing/profile changes, and atomically writes config/profile after admission; preserves external model-provider configs. | +| Catalog/cache | `src/codex/sync.ts:60` plus `src/codex/catalog/sync.ts:1592` `syncCatalogModels` | Refreshes OPENCODEX catalog and Codex model cache before injection. This is model metadata, not conversation content. | +| Main Codex credentials | `src/codex/auth-collision.ts:36` `readCodexTokensResult` | Reads and parses `$CODEX_HOME/auth.json` tokens without logging them. These credentials authorize native forwarding and WHAM quota probes; they are not usage history. | +| Thread DB and rollouts | `src/codex/history-provider.ts:689`; `src/codex/native-residue.ts:531` | Migration/residue logic described above. | +| Active/archived session storage | `src/storage/scanner.ts:172`; `src/storage/cleanup.ts:276` | Scanner measures both; cleanup mutates only archived sessions and associated DB rows, never active sessions. | +| Management storage view | `src/server/management-api.ts:35-38` imports; routed through management handlers at `src/server/management-api.ts:208` | Resolves Codex home and exposes scan/cleanup operations through management APIs. It does not expose rollout conversation bodies. | + +## 3. Usage/token accounting and quota display + +### Accounting data flow + +| Stage | File and symbol | Source and normalized fields | +|---|---|---| +| Canonical shape | `src/types.ts:410` `OcxUsage` | `inputTokens`, `outputTokens`, optional absolute `contextTotalTokens`, `totalTokens`, cached read/write split, reasoning output, estimated flag. Cache detail is a subset and is not added twice (`src/types.ts:401`). | +| Native Responses passthrough extraction | `src/server/request-log.ts:542` `applyResponseLogMetadata`; `src/server/request-log.ts:563` `usageFromResponsesPayload` | Reads upstream `response.usage`/`usage`, supporting Responses `input_tokens/output_tokens` and chat-style `prompt_tokens/completion_tokens`, plus cached/write/reasoning details. | +| Routed OpenAI Responses adapter | `src/adapters/openai-responses.ts:1277` `usageFromResponsesPayload` | Reads upstream Responses usage into `OcxUsage` for adapter events. | +| OpenAI Chat adapter | `src/adapters/openai-chat.ts:1147` `usageFromOpenAIChat` | Reads provider `prompt_tokens`, `completion_tokens`, cached and reasoning details. Requests streaming usage with `stream_options.include_usage` (`src/adapters/openai-chat.ts:1337`). | +| Other adapters | Examples: `src/adapters/anthropic.ts:520`, `src/adapters/google.ts:288`, `src/adapters/command-code.ts:311`, `src/adapters/kiro-events.ts:72` | Normalize provider-specific usage frames; Cursor/Kiro can be marked estimated when authoritative per-turn usage is unavailable (`src/usage/log.ts:162`). | +| Bridge capture | `src/server/responses/core.ts:4122` streaming and `src/server/responses/core.ts:4193` JSON | Receives raw adapter usage before Responses wire normalization and places it on request/attempt log context. | +| Local fallback estimate | `src/server/request-log.ts:989` `finalizedUsage` | If usage is absent, can persist an input-token estimate with output 0; combines an estimate with reported usage conservatively and caps estimates at known context windows. | +| Durable ledger | `src/usage/log.ts:154` `usageLogPath`; `src/usage/log.ts:442` `appendUsageEntry` | Appends normalized request rows to `$OPENCODEX_HOME/usage.jsonl`, mode `0600`. | +| Summary | `src/usage/summary.ts:290` onward | Aggregates requests/attempts, input/output/cache/reasoning/total tokens, models/providers/accounts, and estimated cost. | +| Management usage API | `src/server/management/logs-usage-routes.ts:197` | `GET /api/usage` reads a bounded snapshot of `usage.jsonl`, summarizes it, and caches by exact ledger revision and pricing-overlay version. | +| Management request history | `src/server/management/request-history-routes.ts:1` | `GET /api/request-history` is request telemetry, not chat content. It queries a derived SQLite projection. | +| Request-history index | `src/routing/history/indexer.ts:1` `Derived request-history index` | Incrementally indexes complete rows from canonical `usage.jsonl` into rebuildable `$OPENCODEX_HOME/routing-history.sqlite`; records over 1 MiB are omitted from the projection, never from the ledger (`src/routing/history/indexer.ts:76`). | + +### Does accounting read Codex-local state? + +| Surface | Answer | Evidence | +|---|---|---| +| Per-request usage/token accounting | **No rollout/session reads.** Usage comes from upstream provider response frames or OPENCODEX estimates and is persisted in `$OPENCODEX_HOME/usage.jsonl`. | `src/server/request-log.ts:542`, `src/server/request-log.ts:563`, `src/server/request-log.ts:989`, `src/usage/log.ts:442`. | +| Context admission estimate | **No Codex-local history reads.** It walks the already-received parsed request: prompts, messages, thinking, tool calls/results, tools, and image approximations. | `src/server/responses/input-admission.ts:89` `estimateInputTokens`. | +| Native model metadata used by logging/admission | **Limited Codex-local metadata read, not conversation state.** Service-tier support can read Codex catalog metadata (`src/server/request-log.ts:520`); native context windows come from static catalog metadata (`src/server/responses/input-admission.ts:128`). | Paths cited. | +| Provider quota bars | **Live provider-account state, separate from request usage.** `GET /api/provider-quotas` calls provider quota endpoints through `fetchProviderQuotaReports`; native ChatGPT quota uses WHAM. | `src/server/management/provider-routes.ts:342`; `src/providers/quota.ts:1014`; `src/providers/quota.ts:1964`. | +| Native Codex quota credential | **Reads `$CODEX_HOME/auth.json` only to authorize WHAM.** The quota values themselves come from `https://chatgpt.com/backend-api/wham/usage`. | `src/codex/auth-api.ts:681`, `src/codex/auth-api.ts:700`. | +| Quota cache | **OPENCODEX-local, not Codex-local.** Percent/reset-credit snapshots live at `$OPENCODEX_HOME/codex-quota-cache.json`. | `src/codex/quota.ts:24`; WHAM parsing at `src/codex/quota.ts:439`. | +| Codex quota management view | Reads the in-memory/disk quota cache, not rollouts. | `src/codex/auth-api.ts:1640` `GET /api/codex-auth/quota`. | + +## 4. `/v1/responses` inbound lifecycle + +### End-to-end path + +| Order | File and symbol | Behavior | +|---:|---|---| +| 1 | `src/server/index.ts:1187` HTTP route | Auth/origin/admission, request-log context, then `handleResponses`; final logging can be deferred until a native stream terminal. | +| 1b | `src/server/index.ts:1427`, `src/server/index.ts:1534` WS route | Converts each `response.create` frame into an internal POST body with `stream: true` and invokes the same handler. | +| 2 | `src/server/responses/core.ts:1572` `handleResponsesInner` | Reads/decompresses bounded JSON, handles combo dispatch, local continuation expansion, encrypted-task normalization, parses request. | +| 3 | `src/server/responses/core.ts:1035` `applyFinalRouteRequestNormalization` | Rewrites selected model/stream/store/service-tier/guidance/effort after final route selection. | +| 4 | `src/server/responses/core.ts:1942` input admission | Estimates full parsed input and may return 413; skips compaction turns. | +| 5 | `src/server/responses/core.ts:2242` passthrough or translated adapter branch | OpenAI Responses passthrough builds from `_rawBody`; other adapters build provider-specific requests from parsed context/options. | +| 6a | `src/server/responses/core.ts:2781` passthrough SSE | Tees native SSE for bounded inspection/logging/continuation recording while relaying to client. | +| 6b | `src/server/responses/core.ts:2852` passthrough JSON | Reads max 32 MiB, logs usage, optionally records continuation, repairs/reframes output. | +| 6c | `src/server/responses/core.ts:4087` translated stream; `src/server/responses/core.ts:4151` translated JSON | Parses adapter events, applies terminal/empty-completion guards, builds Responses SSE/JSON, captures usage and continuation state. | + +### Field-by-field inbound behavior + +The accepted top-level schema is `src/responses/schema.ts:139` `responsesRequestSchema`. Zod parses known fields into `data`, while `parseRequest` also preserves the original/expanded object as `_rawBody` (`src/responses/parser.ts:703`). Therefore a field can be ignored by translated adapters yet still survive an OpenAI Responses passthrough. + +| Field | Validation/read | Internal rewrite/use | Forward/drop behavior | +|---|---|---|---| +| `store` | Optional boolean at `src/responses/schema.ts:151`. `parseRequest` does not copy it into `OcxRequestOptions`; later code reads `_rawBody.store`. | Canonical ChatGPT forward route defaults an omitted value to `false`, preserving explicit values (`src/server/responses/core.ts:1083`). `rememberResponseState` ordinarily skips `store:false`, while forced passthrough/Kiro/Cursor paths may still retain bounded local continuation state (`src/responses/state.ts:1164`). | OpenAI Responses raw passthrough preserves explicit values. Stateless Responses providers force `store:false` (`src/adapters/openai-responses.ts:790`). When `store:false`, item `id` fields are stripped because upstream cannot resolve stored items (`src/adapters/openai-responses.ts:119`). Translated non-Responses adapters do not receive a `store` field. | +| `previous_response_id` | Optional string at `src/responses/schema.ts:152`; copied to `parsed.previousResponseId` at `src/responses/parser.ts:703`. | Before parsing, `expandPreviousResponseInput` looks up OPENCODEX continuation state and prepends stored items (`src/server/responses/core.ts:1609`; `src/responses/state.ts:986`). Cross-task scope mismatch removes the id (`src/responses/state.ts:1000`). Spill corruption/missing is a structured 400 (`src/server/responses/core.ts:1614`). Duplicate full client-carried history is detected and not prepended (`src/responses/state.ts:1010`). | Canonical forward always strips it after requiring local replay for safety; API-key Responses keeps an unexpanded id for real server-side continuation but strips it after local expansion; stateless providers always strip it (`src/adapters/openai-responses.ts:750`, `src/adapters/openai-responses.ts:790`). Combo children delete it after expansion to prevent double prepend (`src/server/responses/core.ts:1244`). Translated adapters consume the expanded context, not the id. | +| `prompt_cache_key` | Optional string at `src/responses/schema.ts:154`; copied to `options.promptCacheKey` at `src/responses/parser.ts:686`. | Used for provider transport/key affinity (`src/server/responses/core.ts:2063`) and as one candidate in Anthropic account session affinity (`src/server/responses/core.ts:2000`). | Raw OpenAI Responses passthrough preserves it. OpenAI Chat forwards it only when provider config opts into `promptCacheKey` (`src/adapters/openai-chat.ts:1298`). Other translated adapters do not receive the top-level field merely because it was present. | +| `instructions` | Optional string/null at `src/responses/schema.ts:142`. Nonempty string is read at `src/responses/parser.ts:330`. | Appended to `context.systemPrompt`; system-role input messages are also flattened into the same system prompt (`src/responses/parser.ts:420`). | Translated adapters receive it as system context, not a Responses top-level field. Raw OpenAI Responses passthrough retains the original field. Empty/null instructions do not add system context. | +| `input` string | Optional string or item array at `src/responses/schema.ts:141`. | String becomes one user message (`src/responses/parser.ts:334`). | Translated to provider message format; raw passthrough retains the string unless another sanitizer rewrites the body. | +| `input[]` message items | Item schema at `src/responses/schema.ts:36-103`; parsing loop begins `src/responses/parser.ts:337`. | User/developer → messages; system → system prompt; assistant → assistant content/phase. Images stay structured; inline file bytes become a marker, not prompt bytes (`src/responses/parser.ts:37`). | Unknown item types can pass schema via the loose catch-all and remain in `_rawBody`, but the translated parser ignores unhandled types. | +| `input[]` reasoning | `reasoning` item schema at `src/responses/schema.ts:52`. | Summary text preferred over content; OPENCODEX reasoning envelopes decoded; plaintext thinking attached to the proper assistant/tool-call turn; opaque native encrypted-only reasoning is not exposed (`src/responses/parser.ts:455`). | Passthrough sanitation can strip OPENCODEX envelopes/raw reasoning content and item ids depending on target/store (`src/adapters/openai-responses.ts:38`, `src/adapters/openai-responses.ts:119`). | +| `input[]` calls/results | Schemas at `src/responses/schema.ts:60-91`. | Function/custom/local-shell/tool-search calls and outputs are paired into assistant/tool-result history (`src/responses/parser.ts:501`, `src/responses/parser.ts:543`, `src/responses/parser.ts:566`, `src/responses/parser.ts:609`). Malformed call arguments default to `{}`. | OpenAI Responses passthrough has orphan repair, adjacency normalization, id sanitation, and tool compatibility rewrites (`src/adapters/openai-responses.ts:611`, `src/adapters/openai-responses.ts:660`, `src/adapters/openai-responses.ts:1363`). | +| `input[]` compaction | Loose item acceptance; explicit handling at `src/responses/parser.ts:356` and `src/responses/parser.ts:373`. | `compaction_trigger` flags a compaction request and is not a message. Stored `compaction`/`compaction_summary`/`context_compaction` becomes a summary user message; payload-less `context_compaction` is only a boundary marker. | Routed compaction replaces tools/structured output and adds the compaction prompt (`src/server/responses/core.ts:2215`). Passthrough sanitizer converts OPENCODEX `ocx1:` compaction items to plain messages but leaves genuine OpenAI opaque blobs (`src/adapters/openai-responses.ts:141`). | +| top-level `reasoning` | Optional nullable `{effort?, summary?}` at `src/responses/schema.ts:132`, `src/responses/schema.ts:150`. | `ultra`→`max`; only known effort strings retained; summary absent/`none` sets `hideThinkingSummary` (`src/responses/parser.ts:673`). Final routing can cap/clamp effort and mutate `_rawBody.reasoning.effort` (`src/server/responses/core.ts:1157`, `src/server/responses/core.ts:1173`); shadow calls force low (`src/server/responses/core.ts:1681`). | Raw passthrough preserves it subject to model capability/spark/summary sanitizers (`src/adapters/openai-responses.ts:167`, `src/adapters/openai-responses.ts:190`). Native `/responses/compact` explicitly drops top-level reasoning before forwarding (`src/server/responses/compact.ts:401`). Translated adapters map `options.reasoning` to their wire format. | +| `include[]` | Accepted as `unknown` at `src/responses/schema.ts:161`. | **Not read by `parseRequest` and no other top-level `include` consumer was found in `src/responses`, `src/server/responses`, or adapters.** | Preserved only through `_rawBody` on OpenAI Responses passthrough. Dropped implicitly on translated provider requests because no internal option carries it. No focused `include[]` test was found. | + +### Other observed top-level mutations + +| Mutation | Evidence | +|---|---| +| Routed model namespace/virtual model rewrites mutate `_rawBody.model`. | `src/server/responses/core.ts:1052`, `src/server/responses/core.ts:1099`. | +| Provider policy may force `_rawBody.stream = false`. | `src/server/responses/core.ts:1076`. | +| Service tier can be injected/deleted based on route capability. | `src/server/responses/core.ts:1114`, `src/server/responses/core.ts:1524`. | +| Multi-agent guidance inserts a developer item into parsed context and raw input. | `src/server/responses/core.ts:1137`; insertion implementation `src/server/responses/collaboration.ts:452`. | +| Plaintext task payloads mislabeled as encrypted content are rewritten in raw `input` before parsing. | `src/server/responses/core.ts:1624`. | +| Canonical forward rejects and therefore drops `max_output_tokens` and `metadata`. | `src/adapters/openai-responses.ts:801`. | +| Stateless Responses providers drop `previous_response_id`, `conversation`, `background`, `metadata`, and `prompt`, then force `store:false`. | `src/adapters/openai-responses.ts:767`. | + +## 5. Long inputs, compaction, truncation, and caching + +| Mechanism | Current behavior | Evidence | +|---|---|---| +| Request-body cap | Accepts identity/zstd/gzip/deflate JSON up to 256 MiB decompressed; rejects larger bodies with 413. This is byte admission, not semantic truncation. | `src/server/request-decompress.ts:15`, `src/server/request-decompress.ts:190`, `src/server/request-decompress.ts:237`. | +| Translation memory budget | Per request, retained translation buffers are capped at 32 MiB and one tool-call argument at 2 MiB; overflow maps to 413 `translation_buffer_limit`. | `src/lib/translator-budget.ts:1`; `src/server/responses/core.ts:1651`. | +| Context-size admission | Estimates all parsed system/messages/thinking/tool args/tool schemas/images and refuses only above `known ceiling × 2.5`; unknown ceiling fails open. Returns 413 telling the client to start a new session or choose a larger model. | `src/server/responses/input-admission.ts:81`, `src/server/responses/input-admission.ts:128`, `src/server/responses/input-admission.ts:159`; call site `src/server/responses/core.ts:1942`. | +| Automatic truncation on ordinary turns | **None found.** The admission gate explicitly says it is not a context manager or compaction trigger. Ordinary over-limit input is rejected, not shortened. | `src/server/responses/input-admission.ts:2-11`. | +| Parser-level content omission | Malformed blocks are ignored; remote image URLs remain structured; inline `input_file.file_data` bytes are replaced by a marker. This is protocol normalization, not old-turn truncation. | `src/responses/parser.ts:37-82`. | +| Remote compaction v2 | `compaction_trigger` makes a routed model summarize with `COMPACT_PROMPT`; bridge emits exactly one synthetic `compaction` item containing `ocx1:` + base64 summary. Later input decodes it to a summary message. | `src/responses/compaction.ts:1-53`; `src/server/responses/core.ts:2215`; bridge flags at `src/server/responses/core.ts:4121`. | +| Remote compaction v1 | `/v1/responses/compact` forwards to supported native compact upstreams; routed models are internally converted to v2 summarization, then return recent user messages plus summary. Retained user-message budget is 20k tokens approximated as 80k chars, newest-first with partial oldest tail. | Route `src/server/index.ts:1071`; handler `src/server/responses/compact.ts:267`; retention `src/responses/compaction.ts:56-123`; routed conversion `src/server/responses/compact.ts:656-716`. | +| Compaction input admission | Compaction turns bypass the context 413 gate so a full context can be shrunk. | `src/server/responses/core.ts:1944`. | +| Post-compaction continuation | Compaction turns are deliberately not recorded in OPENCODEX continuation state, because `_rawBody` contains pre-compaction history and replay would rehydrate it. | `src/server/responses/core.ts:2257`, `src/server/responses/core.ts:4130`, `src/server/responses/core.ts:4201`. | +| Local `previous_response_id` cache | Stores request input + response output, prepends it on continuation, detects a full client-carried duplicate, and binds scoped entries to client task ids. | `src/responses/state.ts:986`, `src/responses/state.ts:1010`, `src/responses/state.ts:1164`. | +| Continuation cache bounds | 1,000 ids, 1-hour TTL, 64 MiB resident cap; oversized resident entries spill durably; snapshot write retains max 2 MiB per resident entry and 24 MiB total; existing snapshot parse capped at 32 MiB. Dedicated spill payload cap is 256 MiB. | `src/responses/state.ts:17-30`; pruning `src/responses/state.ts:853`; spill cap `src/responses/spill-store.ts:61-67`. | +| Continuation persistence | Best-effort, debounced `$OPENCODEX_HOME/responses-state.json`; dedicated spills in `$OPENCODEX_HOME/responses-state-spill/`. Cache is not a source of truth. | `src/responses/state.ts:402`, `src/responses/state.ts:597`; `src/responses/spill-store.ts:151`. | +| Prompt caching | Anthropic adapters can add up to four ordered `cache_control` breakpoints based on `cacheRetention`; native Anthropic may also use top-level automatic caching. | `src/adapters/anthropic.ts:64-162`, `src/adapters/anthropic.ts:925`. | +| `prompt_cache_key` | Forwarded only on supported/opted-in wires and also used for affinity/transport selection; OPENCODEX does not maintain a prompt-content cache keyed by it. | `src/responses/parser.ts:686`, `src/adapters/openai-chat.ts:1298`, `src/server/responses/core.ts:2000`, `src/server/responses/core.ts:2063`. | +| Reasoning replay cache | Separate process-local, provider/account/model/thread-scoped bounded cache for pairing raw reasoning with tool calls; never logged or persisted. | `src/responses/reasoning-replay-cache.ts:1-18`, `src/responses/reasoning-replay-cache.ts:187`. | +| Management “request history” | Caches/indexes request telemetry only. It never supplies conversation text to models. | `src/server/management/request-history-routes.ts:1-11`, `src/routing/history/indexer.ts:1-9`. | + +## 6. Test coverage inventory + +### Responses ingress and lifecycle + +| Test file and anchors | Covered behavior | +|---|---| +| `tests/responses-parser.test.ts:5` | Core parser, tool schemas, phase, tool choice, `service_tier`, `prompt_cache_key`, structured output, files/images, compaction replay, additional tools, reasoning effort normalization. | +| `tests/responses-parser-agent-message.test.ts:4` | Agent-message/reasoning turn boundaries. | +| `tests/responses-parser-malformed-content.test.ts:22` | Malformed content blocks, null/object containers, image/file omission/markers, adapter build resilience. | +| `tests/responses-inbound-store-default.test.ts:37` | Omitted versus explicit `store` across canonical forward, key-auth Responses, and noncanonical forward gateways. | +| `tests/responses-state.test.ts:108` | `previous_response_id` storage/expansion/scope, `store:false`, forced continuation, spill/persistence/TTL/size bounds, corrupt/missing state, compaction-marker provenance, provider continuation state. | +| `tests/continuation-dedup.test.ts:58` | Exact client-carried replay deduplication, ordering, provider-authored identity requirement, bounded fingerprints. | +| `tests/issue-702-expired-replay-state.test.ts:246` | Fail-closed expired/corrupt forward replay, fresh local expansion, and API-key preservation of native `previous_response_id`. | +| `tests/openai-responses-passthrough.test.ts:241` | Passthrough sanitization: reasoning, ids/store, prompt cache key, previous-response stripping, stateless fields, tools, summaries, compaction compatibility. | +| `tests/openai-chat-hardening.test.ts:504` | `prompt_cache_key` opt-in forwarding/drop behavior. | +| `tests/responses-compaction.test.ts:43` | Compaction envelope encode/decode, parser trigger/summary behavior, v1 retained user-message output. | +| `tests/responses-compaction-routing.test.ts:139`, `tests/responses-compaction-routing.test.ts:437`, `tests/responses-compaction-routing.test.ts:557`, `tests/responses-compaction-routing.test.ts:639` | Native/routed compact routing, auth/pool/fallback/circuit behavior and compact request output contract. | +| `tests/input-admission.test.ts:34`, `tests/input-admission.test.ts:96`, `tests/input-admission.test.ts:154` | Context ceiling resolution, whole-input estimation, images/tool schemas/thinking, tolerance and fail-open behavior. | +| `tests/responses-tool-conformance.test.ts:39` | `additional_tools`, tool-kind discrimination, tool-search history, stream/JSON parity. | +| `tests/adapter-tool-conformance.test.ts:74` | Instructions entering translated adapter context along with tool behavior. | + +Field-specific coverage result: + +| Field | Focused coverage found? | +|---|---| +| `store` | Yes — `tests/responses-inbound-store-default.test.ts:37`; continuation effects in `tests/responses-state.test.ts:291`. | +| `previous_response_id` | Yes — `tests/responses-state.test.ts:108`, `tests/continuation-dedup.test.ts:58`, `tests/issue-702-expired-replay-state.test.ts:246`, `tests/openai-responses-passthrough.test.ts:843`. | +| `prompt_cache_key` | Yes — parser at `tests/responses-parser.test.ts:191`, raw passthrough at `tests/openai-responses-passthrough.test.ts:785`, chat opt-in at `tests/openai-chat-hardening.test.ts:504`. | +| `instructions` | Indirect translated-adapter coverage at `tests/adapter-tool-conformance.test.ts:74`; no dedicated parser-only instructions test found. | +| `input[]` | Broad parser/malformed/tool/compaction coverage in the parser and conformance files above. | +| top-level `reasoning` | Yes — `tests/responses-parser.test.ts:410`; passthrough sanitization beginning `tests/openai-responses-passthrough.test.ts:472`. | +| `include[]` | **No focused test found.** | + +### Rollout/session and Codex-state reads + +| Test file and anchors | Covered behavior | +|---|---| +| `tests/codex-history-provider.test.ts:98` | Latest session meta, append-not-rewrite, id mismatch, first-line provider patch, large first-line metadata, provider restore/eject, DB retry/no-op/migration. | +| `tests/codex-native-residue.test.ts:627` | DB↔rollout provider consistency, first/latest metadata, malformed/oversized/BOM/chunk-split/missing rollout, backup references, fail-closed classification. | +| `tests/storage-cleanup.test.ts:325` | Archived-only listing, active-session exclusion, `.jsonl`+`.zst` grouping/path normalization. | +| `tests/storage-cleanup.test.ts:1389` | Legacy quarantine restore reconstructs production thread fields from rollout JSONL. | +| `tests/storage-cleanup.test.ts:1475` | Bounded compressed-only rollout reconstruction. | +| `tests/storage-scanner.test.ts:105` | Session/archive byte inventory, immutable DB row counts, unreadable DB behavior, zero-write invariant. | +| `tests/codex-inject-integration.test.ts:366` | External provider config remains byte-identical so existing session history remains visible. | +| `tests/codex-history-job.test.ts`, `tests/codex-history-worker.test.ts`, `tests/codex-history-writer.test.ts`, `tests/codex-history-reachability.test.ts` | History job/worker serialization, writer/retry boundaries, and module reachability around the same provider migration. | +| `tests/api-storage.test.ts`, `tests/api-storage-cleanup.test.ts` | Management storage scan/cleanup API surfaces. | +| `tests/openai-provider-option-tooling.test.ts:38` | Validates the smoke/final-gate evidence artifact shape; the temporary rollout parsing itself runs in the script final gate rather than a unit test. | + +## Inventory conclusions + +| Question | Current-state answer | +|---|---| +| Does OPENCODEX parse Codex rollout JSONL? | Yes, for provider/source migration, native-residue verification, and legacy archived-session restoration. It extracts session metadata and at most the first user-message preview, not the full conversation for inference. The parsers are envelope/schema-sensitive. | +| Does usage/quota read rollout history? | No. Per-request usage comes from provider response frames or local estimates. Quotas come from provider quota endpoints (native ChatGPT via WHAM), with `$CODEX_HOME/auth.json` used only as credentials and `$OPENCODEX_HOME` used for caches/ledgers. | +| How are the named inbound fields handled? | `store` is raw-body policy; `previous_response_id` is locally expanded and target-conditionally stripped; `prompt_cache_key` is preserved plus used for affinity where supported; `instructions` becomes system context; `input[]` is deeply normalized; `reasoning` is mapped/capped/sanitized; `include[]` is passthrough-only and disappears on translated wires. | +| What happens to very long input? | Byte/translation/context gates reject oversized ordinary turns. OPENCODEX does not truncate old turns automatically. It implements explicit Codex remote compaction v1/v2 and bounded local continuation/spill caches. | +| What tests exist? | Strong coverage exists for parsing, continuation, passthrough sanitization, compaction, admission, rollout migration/residue, and archived restore. No focused top-level `include[]` test was found; instructions coverage is indirect. | diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/005_public_web_evidence.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/005_public_web_evidence.md new file mode 100644 index 0000000000..dff5c00c30 --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/005_public_web_evidence.md @@ -0,0 +1,61 @@ +# Lane E — public web evidence and upstream-to-OpenCodex roadmap + +Research cutoff: 2026-08-16 (Asia/Seoul). The two checkouts were inspected read-only. + +## Executive finding + +The public evidence verifies the GPT-5.6 family, `ultra`/multi-agent delegation, and +the official Daybreak Blue alias mapping. It does **not** verify the reported +`741`-turn / `27.6s -> 1.7s` / `~98% fewer requests` figures: no primary web page, +official changelog entry, or upstream commit message containing those exact figures +was found. Treat those numbers as **UNVERIFIED** pending an internal benchmark or +the missing upstream announcement. + +The upstream checkout itself contains the implementation evidence needed for the +roadmap. Its HEAD is `9dd22890f5ff47e4af128c20e32b9758a61d78d2`. + +## Public claim ledger + +| Claim | URL | Publication date | Source type | Status | +|---|---|---:|---|---| +| GPT-5.6 launches Sol, Terra, and Luna for general availability; `ultra` coordinates multiple agents across parallel workstreams. | [OpenAI GPT-5.6 release](https://openai.com/index/gpt-5-6/) | 2026-07-09; page also records a 2026-07-30 pricing update | primary, official OpenAI release | verified | +| The official API model guidance maps `gpt-5.6` to `gpt-5.6-sol`, and names Terra for balance and Luna for efficient high-volume work. | [OpenAI model guidance](https://developers.openai.com/api/docs/guides/latest-model) | Not stated on page; retrieved 2026-08-16 | primary, official developer docs | verified | +| The same guidance documents multi-agent beta in Responses API: a GPT-5.6 instance coordinates parallel subagents and synthesizes results. | [OpenAI model guidance](https://developers.openai.com/api/docs/guides/latest-model) | Not stated on page; retrieved 2026-08-16 | primary, official developer docs | verified | +| The same guidance documents persisted reasoning across turns and recommends `previous_response_id`/history replay when using `all_turns`. | [OpenAI model guidance](https://developers.openai.com/api/docs/guides/latest-model) | Not stated on page; retrieved 2026-08-16 | primary, official developer docs | verified | +| Daybreak Blue API alias `gpt-daybreak-blue` maps to model ID `gpt-5.6-sol`; Red maps to `gpt-5.6-cyber`. | [OpenAI Help: Enterprise Daybreak onboarding](https://help.openai.com/en/articles/20001261-enterprise-daybreak-onboarding) | Updated 3 days before retrieval; exact calendar date not rendered | primary, official Help Center | verified | +| OpenAI’s Daybreak page reports GPT-5.6 Sol completing 7/10 of a 32-step simulation versus 2/10 for GPT-5.5. | [OpenAI Daybreak](https://openai.com/daybreak/) | Page lists latest Daybreak item 2026-08-10; claim page date not separately rendered | primary, official OpenAI product page | verified | +| Codex GitHub releases show August 2026 prereleases, including `0.148.0-alpha.14` on Aug 14 and `0.148.0-alpha.17` on Aug 14. | [openai/codex releases](https://github.com/openai/codex/releases) | 2026-08-14 | primary, official repository release page | verified | +| Upstream issue reports a catalog-visible `gpt-5.6-luna` rejected by the native `spawn_agent` allowlist while Sol/Terra worked. | [openai/codex#34399](https://github.com/openai/codex/issues/34399) | Opened 2026-07-20; now closed | primary, official repository issue | verified as historical issue evidence, not a current support guarantee | +| Upstream issue reports a later Agent V2 behavioral regression where repeated Sol delegation emitted `exec`/`wait` instead of `spawn_agent`. | [openai/codex#35620](https://github.com/openai/codex/issues/35620) | Opened 2026-07-27; still open at retrieval | primary, official repository issue | verified as reported behavior; root cause unverified | +| The exact `741` turns, `27.6s -> 1.7s`, `~98% fewer requests`, and memory-use claims are publicly documented. | [Google/web search result set](https://github.com/openai/codex) | N/A | No primary source found | **candidate — unverified snippet / empty primary result** | + +Search coverage: 26 distinct queries across OpenAI developer docs, Help Center, +OpenAI product/release pages, GitHub releases/issues/source, and secondary coverage; +the exact performance-number searches returned no primary source. + +## Upstream implementation evidence + +All paths below are relative to `/Users/jun/Developer/codex/121_openai-codex` and +line numbers are from the inspected HEAD `9dd22890f`. + +### Delegation to leaf and supported models + +- `6d4d9442c7142c08ac5c5098dfd6e82d8cd9f65a` (2026-08-04, “Support leaf models in multi-agent v2”) changes `codex-rs/core/src/tools/handlers/multi_agents_common.rs:model_supports_multi_agent_backend` (line 36) so V2 accepts every model except one explicitly marked `Disabled`. The same commit changes `codex-rs/core/src/agent/control/spawn.rs:AgentControl` (stored model restoration around lines 268–324), preserving the selected worker model on reload. +- `92938d880eccbad1242a86a63f819f67780f68c0` (2026-07-13) adds backend-aware filtering and validation in `codex-rs/core/src/tools/handlers/multi_agents_common.rs:find_spawn_agent_model_name` (line 431) and `model_supports_multi_agent_backend` (line 36), plus `codex-rs/core/src/tools/handlers/multi_agents_spec.rs:spawn_agent_models_description` (line 781). The advertised override list is picker-visible and backend-compatible, capped at five. +- `ea1545628404e448347bae336771eaf649614105` (2026-07-13) exposes optional `model` and `reasoning_effort` controls in `codex-rs/core/src/tools/handlers/multi_agents_spec.rs:create_spawn_agent_tool_v2` (line 102), guarded by `SpawnAgentToolOptions` (line 26). +- `b00c9b2e16ccdbf2c7c8d58a590e0fc2ca97573b` (2026-07-20) marks Multi-Agent V2 stable in the feature configuration. `6d4d9442c` is the important “all visible models unless disabled” semantic change; it is not a hard-coded Sol/Terra/Luna list. +- `51e36d2ec23c0eff710053d28c400d447500a41a` (2026-08-07) exposes nullable `multiAgentVersion` (`disabled`, `v1`, `v2`) through `codex-rs/app-server/src/models.rs:model_from_preset` (line 27), sourced from `codex-rs/protocol/src/openai_models.rs:ModelPreset` (line 206). + +### Conversation-history loading and request reduction + +- `161748a68eb4a4aba5420c3a6f1739f098513178` (2026-07-17) adds `codex-rs/message-history/src/batch.rs:lookup_batch` (line 111): cursor-based newest-first reads, max 128 rows/64 KiB, byte anchors for unchanged append-only files, and safe offset fallback after rewrites. +- `8bfa49e350edb065889332c72854d06f0e7ce50f` (2026-08-04) adds `codex-rs/tui/src/app/history_pagination.rs:App::handle_older_history_page` (line 57) and bounded initial hydration in `codex-rs/tui/src/app_server_session/history.rs:AppServerSession::hydrate_initial_thread_history` (line 198). The initial view loads a bounded page; older pages load as the user navigates upward. +- `3b8d22ec2c75bf8fcd6048c34039344795ff7a0a` (2026-08-04) hardens `codex-rs/tui/src/app_server_session/history.rs:thread_items_page` (line 116), `merge_thread_item_page` (line 154), cursor-repeat protection (`advancing_cursor`, line 51), and initial-vs-complete hydration. This is the closest code-level explanation for fewer history-load requests, but the exact 98%/27.6s/1.7s benchmark is **UNVERIFIED**. +- `7ed19a97580e65a55feed9074f21decb2d720e9b` (2026-07-17) batches persistent history reads during reverse search; `codex-rs/message-history/src/batch.rs` is the reusable primitive. +- `63002bdb26c939925f3fa59b9575cc0a3564cb45` (2026-08-10) extracts persisted history types into `codex-rs/history/src/lib.rs`, separating history contracts from protocol/runtime consumers. + + +--- + +Evidence only (LEXICO-SPLIT-01). The prescriptive roadmap that once ended this file +lives in [000_plan.md](000_plan.md) and the decade docs. diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/006_audit_round1.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/006_audit_round1.md new file mode 100644 index 0000000000..7696ffa63a --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/006_audit_round1.md @@ -0,0 +1,35 @@ +# 006 — A-phase audit round 1: blockers and disposition + +Reviewer: independent `explorer` subagent (gpt-5.6-sol, high effort), read-only, dispatched +2026-08-16 against revision 1 of this unit. Verdict: **FAIL**, 11 blockers. + +Every blocker was re-verified by the main agent against real code before disposition — +none were accepted on the reviewer's word, and none were rejected without evidence. + +| # | Sev | Blocker | Disposition | +| --- | --- | --- | --- | +| 1 | High | Verifier commands exit 1 in this checkout (`zod/v4` missing, `bun-types` missing); no exit-code receipts recorded | **FOLDED.** Root cause confirmed: `ls node_modules` → absent. Every decade doc now carries a receipts table and a `bun install` precondition; `000_plan.md` has a global environment precondition. | +| 2 | Medium | Four citations miss their content: `sync.ts:627/1087` reversed; `package.json` `test` is `bun scripts/test.ts` not `bun test`; `compact.rs:35` is `fn path()` not `CompactRequest::new`; `index.ts:1071` is a comment, code at `:1074` | **FOLDED.** All four verified wrong and corrected in place. | +| 3 | High | Phase 1 left `isEligibleV2SubagentEntry` (`sync.ts:105-108`) implementing the old equality rule, so Luna stays out of the roster | **FOLDED — the most important finding.** Confirmed: the predicate returns true only for `v2`/null/undefined. Phase 1 rewritten around it; `000_plan.md` now names both sites. | +| 4 | High | `CatalogModel.multiAgentVersion` is a ghost field — no creation path; `provider-fetch.ts:999` discards `multi_agent_version`; native management rows (`model-rows.ts:67`) and GUI type (`models-shared.ts:28`) omit it | **FOLDED.** Confirmed by reading each site. Phase 1 Change 3 now specifies the full chain with an `N/A + reason` for user config. | +| 5 | High | Phase 2's fail-closed contract is unimplementable: sync/restore use bulk updates (`history-provider.ts:750`, `:801`); `rememberOriginal` (`:340`) and the result contract (`:165-175`) missing from the chain | **FOLDED.** Confirmed. Phase 2 now specifies explicit row partitioning with id-scoped SQL, a `skippedUnknownMode` result field, and its consumers. | +| 6 | High | G5's impact is unreachable: preview output is consumed only by `cleanup.ts:2305`, and `cleanup.ts:673` refuses paginated threads first. Fixture recipe used `rg -rn` (`-r` = replace) | **FOLDED.** Confirmed both. G5 downgraded to **conditional** behind an explicit caller-reachability gate that may legitimately end in removing G5; commands corrected. | +| 7 | High | Phase 4 changed the writer to the canonical key while readers (`features.ts:243`) recognize only `max_threads`; conflict detector had no consumer; GUI (`Models.tsx:1341`) and locales keep the false boot-refusal copy | **FOLDED.** Confirmed `getLogicalMaxThreads` (`:1305`), the `:1496` postcondition, and `agent-settings-routes.ts:232`. Phase 4 rewritten with the full chain. | +| 8 | High | Fallback wrongly declared "orthogonal and correct": `selectAvailableSubagentModel` (`subagent-model-fallback.ts:269`) ignores capability and rewrites the model after the child tool surface is built (`core.ts:1768`). Also `collaboration.ts:349` fork guidance is stale | **FOLDED as new gaps G12 (compat-break) and G13.** Both confirmed. Now Phase 1 Changes 4 and 5. | +| 9 | Medium | Stated inter-phase dependencies are not real | **FOLDED.** Confirmed: Phase 4's link to Phase 1 was a shared doc file, and Phase 3 does not consume Phase 2's resolver. All phases now marked independent, with a *recommended* risk-first order stated separately. | +| 10 | High | LEXICO-SPLIT-01: research docs 001, 002, 005 contain implementation roadmaps | **FOLDED.** Prescriptive sections removed from the research docs, which now point to the decade docs. | +| 11 | High | Several decade docs are not independently executable: unresolved "export or duplicate", "audit then conditionally fix", "two possible outcomes", "likely no-op" | **FOLDED.** Phase 2's helper question settled (move to `src/codex/sqlite-columns.ts`, re-export). Phase 3's audit is now a bounded lock-and-regress with an explicit gate. Phase 5's decision rule is fixed in advance and its Guardian item is a concrete regression. | + +## Rebuttals + +None. Every blocker was verified as accurate. + +## What the audit changed about the conclusion + +Revision 1 would have shipped a Phase 1 that renamed the symptom and left the defect: +Luna would still have been excluded from the roster after "fixing" leaf semantics, +because the exclusion lives in the roster predicate, not the catalog stamp. The audit +also surfaced two gaps the five-lane research swarm missed entirely (G12, G13), one of +them a compat-break. That is the argument for the A gate being a real dispatch rather +than a self-review. + diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/007_audit_round2.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/007_audit_round2.md new file mode 100644 index 0000000000..1c773fd584 --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/007_audit_round2.md @@ -0,0 +1,30 @@ +# 007 — A-phase audit round 2: blockers and disposition + +Same reviewer, re-dispatched against revision 2 (AUDIT-LOOP-01: blocker-closure rounds +reuse the same reviewer). Verdict: **FAIL**, 7 blockers. Round-1 folds for phase ordering +and conditional-path reachability were confirmed sound; the rest were incomplete. + +All 7 were re-verified by the main agent against real code before folding. No rebuttals. + +| # | Sev | Blocker | Disposition | +| --- | --- | --- | --- | +| 1 | High | Phase 1's provider chain still broken: `multi_agent_version` is TOP-LEVEL (`openai_models.rs:459-460`), not under `metadata`; and nothing wrote `OCX_MULTI_AGENT_FIELD`, so `applyMultiAgentMode` would never see it. Real seam is `applyCatalogModelMetadata` (`effort.ts:113`, called from `sync.ts:322`). GUI type without a renderer is not a consumer | **FOLDED (R2-1).** Both confirmed by reading the source. Bridge specified end-to-end; GUI marked `N/A + reason`. | +| 2 | High | G12 non-executable: no class type, no lookup, no signature; `selectAvailableSubagentModel` (`:269`) takes no catalog/capability; `applySubagentModelFallback` has TWO call sites (`core.ts:1768` **and `:1856`**), only one named; `tests/subagent-model-fallback.test.ts` absent from the verifier | **FOLDED (R2-2).** `SubagentCapabilityClass` + `subagentCapabilityClass()` defined; additive optional parameter; both call sites named; three fallback suites added. | +| 3 | High | Phase 2 missed a third mutation path `ejectRemainingOpencodexHistory` (`:586`, writes `:599`, bulk-updates `:608-618`, reachable from `:786`); `rememberOriginal` backs up rows before mutation (`:731`); restore clears the whole manifest (`:815`); the backup `historyMode` field was self-contradictory | **FOLDED (R2-1..R2-3 of 020).** Third path partitioned; backup scoped to mutated rows; manifest clears only restored entries; the contradictory field is **dropped**, not added. | +| 4 | High | `skippedUnknownMode` has no transport: worker DTO (`history-worker.ts:69,184`), message validation (`history-job.ts:152`), `CodexHistoryJobOutcome` (`:107`), classification (`:282`) all omit it; `history-transition.ts:28` would still classify partial work as `converged` | **FOLDED (R2-4 of 020).** Full 8-stage chain enumerated; partial work must not be recorded as converged. | +| 5 | High | Phase 4 missed live consumers: PUT DTO (`agent-settings-routes.ts:373`), CLI import (`cli/v2.ts:15`) and its **false boot-refusal warning** (`cli/v2.ts:127`), comment preservation (`features.ts:1342`), dotted-key rejection (`:1398`), duplicate detection (`:1416`), GUI `v2.enabled` gating, and `structure/05_gui-and-management-api.md:105` | **FOLDED (R2-2, R2-3 of 040).** All added. The CLI warning is the most user-visible instance of G7 — it ships a false claim today. GUI visibility settled: report the conflict regardless of `enabled`. | +| 6 | High | PLAN-VERIFIER-REAL-01 still violated: `tests/api-v2.test.ts` does not exist and Bun ignores it silently; Phase 1's command omitted the G12 suite; receipts recorded single-file output against multi-file commands; `lint:gui` receipts missing (real exit **127**, `oxlint: command not found`) | **FOLDED (R2-3 of 010, R2-5 of 020, R2-1 of 040).** Phantom test replaced with `tests/codex-v2-gate.test.ts`; commands corrected; receipts now record the exact named command; `lint:gui` exit 127 recorded with its `gui/` install precondition. | +| 7 | Medium | Three stale citations: `002:251` still said `compact.rs:35` is `CompactRequest::new`; `010:178` said upstream "discourages" overrides when `config/mod.rs:253` says they "do not accept" them; `history-provider.ts:165-175` excluded `failureReason` at `:176` | **FOLDED.** `002:251` corrected in place; the fork-guidance claim softened and Change 5 downgraded to optional; the range corrected to `:165-176`. | + +## What round 2 changed about the conclusion + +Round 1 fixed *what* to do; round 2 fixed *whether it could be done*. Three folds were +still unexecutable — a capability field with no producer, a fallback change with no +signature, and a result counter that could not cross a Worker boundary. It also caught a +false warning already shipping in `ocx v2 status` (`src/cli/v2.ts:127`), which is a live +user-facing defect rather than a roadmap flaw. + +The receipts lesson generalizes: recording a single-file test result beside a multi-file +command is not a receipt, and `bun test` silently ignoring a nonexistent path means a +command can "pass through" while observing nothing. + diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/008_origin_main_reverification.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/008_origin_main_reverification.md new file mode 100644 index 0000000000..26282b0b5f --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/008_origin_main_reverification.md @@ -0,0 +1,68 @@ +# 008 — origin/main 재검증 (9dd22890f → 49db349ff, +181 커밋) + +사용자가 "myth는 OpenAI 공식 컨펌"이라고 지적하여, 로컬 체크아웃이 4일 뒤처져 있던 것을 +확인하고 `git fetch` 후 181개 신규 커밋을 재조사했다. + +## 로컬 체크아웃이 실제로 오래되어 있었다 + +| | 커밋 | 날짜 | +| --- | --- | --- | +| 재검증 전 로컬 HEAD | `9dd22890f` | 2026-08-12 | +| `origin/main` | `49db349ff` | 2026-08-15 | +| 격차 | **181 커밋** | | + +005의 웹 조사가 `27.6s → 1.7s` / `741턴` / `~98%` 수치의 1차 출처를 찾지 못한 것은 +사실이지만, **그것이 수치가 틀렸다는 뜻은 아니다**. 공식 발표(사용자 확인)를 신뢰하고, +아래는 그 발표를 뒷받침하는 런타임 패치를 커밋으로 특정한 결과다. + +## 런타임 자체를 바꾼 패치 (성능 계열, 신규 181커밋 내) + +| 커밋 | PR | 무엇을 바꿨나 | +| --- | --- | --- | +| `1bb6384c1` | #38604 | **resume 왕복 제거.** 레거시 rollout을 `excludeTurns`로 resume할 때 app-server가 페이지네이션 로딩을 거부 → TUI가 재시도하던 구조. 세션 피커가 알아낸 history mode를 resume 흐름으로 전달해 재시도 자체를 없앰 | +| `c4941302c` | #38774 | `codex exec` 영속 스레드도 페이지네이션 히스토리 사용 | +| `80ceab7aa` | #38358 | `context_manager/normalize.rs` orphan 출력 정규화 최적화 — 단일 패스 수집, orphan이 있을 때만 압축 | +| `8d4d57387` | #38244 | 히스토리 materialization/lineage/paging/turn lookup을 **불변 rollout ID 기준**으로 재키잉 | +| `42bb50d50` | #38413 | 메타데이터 업데이트가 스레드를 materialize하지 않아도 되게 — 불필요한 읽기 제거 | +| `7093e8c48` | #38217 | 서브에이전트용 캐시 MCP 서버 lazy 시작 | +| `42b5f05ce` | #38623 | MCP 툴 카탈로그 캐시에 네임스페이스 설명 보존 | +| `3d7bb2dd2` | #38242 | TUI active-cell 레이아웃 측정 캐싱 | +| `1ba9ce891`, `91d6f4899`, `49db349ff`, `5186e2ccc` | #38822 등 | TUI 렌더링 할당/클론 제거 | + +`thread-store` + `history` + `context_manager` + `app-server/request_processors` 누적 diff: +**51파일, +3102 / −727**. 즉 런타임 패치는 실재하며, 규모도 크다. + +## 그럼에도 이 유닛의 두 결론은 그대로 유효하다 + +`origin/main` 최신 소스로 직접 재확인: + +1. **G1 (Luna 위임).** `multi_agents_common.rs:36-42` `model_supports_multi_agent_backend`는 + 여전히 `Disabled`만 배제한다. `models.json`도 그대로 sol/terra=`v2`, **luna=`v1`**, + gpt-5.5/5.4/5.4-mini/5.2=`null`. → opencodex의 `isEligibleV2SubagentEntry`가 `v1`을 + 배제하는 한 Luna는 계속 로스터에 없다. **변경 없음.** +2. **G3 (ordinal).** `thread_history_materialization.rs`의 + "paginated rollout line for {thread_id} is missing an ordinal" 하드 에러가 그대로 존재. + `update_thread_metadata.rs`도 여전히 `paginated` 분기 후 레거시 `SessionMeta` 경로를 + 건너뛴다(`:87`, `:99`, `:121-123`). **변경 없음.** + +## 002의 "PROXY-VISIBLE vs LOCAL-ONLY" 판정도 유지된다 + +신규 성능 패치는 TUI 렌더링, app-server resume 프로토콜, SQLite 히스토리 프로젝션, +MCP lazy 시작에 집중되어 있다. `ResponsesApiRequest`(`codex-rs/codex-api/src/common.rs`) +필드는 이 계열에서 바뀌지 않았다. 프록시가 보는 와이어는 여전히 `/v1/responses/compact` +계열이 유일한 예외다. + +## 정정 사항 + +`000_plan.md`와 `002`가 "`27.6s → 1.7s`는 UNVERIFIED"라고 적은 것은 **웹 1차 출처를 찾지 +못했다**는 의미로 한정되어야 하며, 공식 발표를 부정하는 근거로 읽혀서는 안 된다. 수치를 +뒷받침하는 런타임 패치는 위 표대로 실재한다. 다만 `332eac4b8`의 N+1 제거가 로컬 SQLite +쿼리 수 감소라는 **메커니즘 분석 자체는** 소스로 확인된 그대로다 — "요청"이 프로바이더 +HTTP 요청이 아니라는 점이 opencodex 관점에서 중요한 부분이고, 그 판단은 바뀌지 않는다. + +## 후속 (신규 커밋에서 발견, 로드맵 영향) + +- `395723b23` (#38619) "Source multi-agent instructions from the model catalog" — 멀티에이전트 + role instruction이 **모델 카탈로그에서** 공급되도록 바뀌었다. opencodex는 카탈로그를 + 합성하므로 Phase 1의 후속 작업으로 조사 필요. 새 work-phase 후보 (**G14**). + diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/009_gh_pr_review.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/009_gh_pr_review.md new file mode 100644 index 0000000000..5ef9c9141e --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/009_gh_pr_review.md @@ -0,0 +1,95 @@ +# 009 — gh PR 직접 리뷰 (로컬 ff → 49db349ff) + +008은 `git log`만으로 재검증했다. 이 문서는 사용자 요청에 따라 **로컬을 fast-forward한 뒤 +`gh`로 PR 본문을 직접 읽어** 성능 주장을 커밋이 아니라 **PR 서술 기준으로** 검증한 결과다. + +## Fast-forward + +`git merge --ff-only origin/main` → `9dd22890f` (08-12) → **`49db349ff` (08-15)**, 181 커밋. +`git rev-list --left-right --count HEAD...origin/main` = `0 0`. 충돌 없음, 로컬 변경 없음 +(`.codexclaw/` untracked만 존재). + +## 검색으로 확인한 것: 수치 자체는 PR에 없다 + +`gh api search/issues` 로 리포지토리 전체를 조회한 결과: + +| 쿼리 | total_count | +| --- | --- | +| `repo:openai/codex is:pr 27.6s` | **0** | +| `repo:openai/codex is:pr "98% fewer"` | **0** | +| `741` (PR) | 무관한 2025년 릴리스 PR 1건뿐 | + +즉 `27.6s → 1.7s / 741턴 / ~98%` 라는 **문구**는 공개 PR 본문에 존재하지 않는다. 005의 +"1차 웹 출처 없음"과 일치한다. 공식 발표를 신뢰한다면, 그 수치는 사내 벤치마크이고 +**공개된 것은 그 수치를 만든 코드**다. 아래가 그 코드다. + +## PR 본문이 직접 서술한 성능 메커니즘 + +`gh pr view` 로 본문을 읽어 확인: + +| PR | 병합 | PR이 **직접 서술한** 문제/해결 | +| --- | --- | --- | +| **#36384** | 07-31 | Why: "Loading the summary view **issued a separate item query for every returned turn**." → 요약 뷰 쿼리에 first-user/final-agent 아이템을 조인. **N+1 제거를 PR이 명시** | +| **#32234** | 07-10 | 페이지네이션 히스토리 전용 DB `thread_history_1.sqlite` 신설 — "avoid adding lock contention to the main state store" | +| **#33364** | 07-15 | app-server가 `historyMode: "paginated"` 지원. resume이 **전체 히스토리 로딩 대신** `excludeTurns: true` 요구, `turnsBackwardsCursor`/`itemsBackwardsCursor` 반환 | +| **#36948** | 08-04 | Why: "Paginated threads **should not require the TUI to load and render their entire history** when a session is resumed." → 경계 있는 초기 페이지만 하이드레이션 | +| **#36949** | 08-04 | 렌더 행 예산까지만 스캔, 반복 커서에서 페이지네이션 중단 | +| **#36950** | 08-04 | 스크롤 시 페이지 단위 로드, 레거시 서버 폴백 | +| **#36951** | 08-04 | 페이지네이션 resume/transcript/fork를 bounded 요청으로 유지 | +| **#38604** | 08-14 | Why: 레거시 rollout을 `excludeTurns`로 resume하면 app-server가 거부 → **TUI가 재시도**. 피커의 history mode를 resume에 전달해 **왕복 자체를 제거** | +| **#34563** | 07-21 | 상속된 fork 계보를 세그먼트 단위로 페이지네이션 | +| **#34361** | 07-20 | 토큰 사용량 replay에서 **전체 히스토리 clone 회피** | +| **#38774** | 08-15 | `codex exec` 영속 스레드도 페이지네이션 사용 | + +"긴 대화 로딩이 느리고 메모리를 먹는다"는 증상에 대해, PR들이 스스로 밝힌 원인은 +**턴마다 쿼리 1회(N+1) + 전체 트랜스크립트 로드/렌더 + 전체 히스토리 clone + 불필요한 +resume 재시도**다. 발표된 수치의 방향과 정확히 일치한다. + +## 그래도 프록시 와이어는 그대로다 (`49db349ff` 기준 재확인) + +`ResponsesApiRequest` (`codex-rs/codex-api/src/common.rs:252`) 필드 전량: +`model, instructions, input, tools, tool_choice, parallel_tool_calls, reasoning, store, +stream, stream_options, include, service_tier, prompt_cache_key, text, client_metadata` — +**이 계열에서 바뀐 것 없음**. 해당 파일의 최근 커밋은 `12c115d55` "Reduce cloning when +building Responses requests" 로 clone 최적화이지 필드 변경이 아니다. + +`CompactionInput` (`common.rs:28-42`)도 9개 필드 그대로. + +결론: 002의 **PROXY-VISIBLE vs LOCAL-ONLY 판정 유지**. 성능 작업은 TUI 렌더링 / app-server +resume 프로토콜 / SQLite 프로젝션 / MCP lazy 시작에 있고, opencodex가 보는 HTTP 와이어에는 +없다. opencodex가 해야 할 일은 여전히 "새 포맷을 깨뜨리지 않기"다. + +## 두 P0도 최신 HEAD에서 그대로 (`49db349ff`) + +1. `multi_agents_common.rs:36-42` — `Disabled`만 배제. `models.json` luna = `"v1"` 유지. +2. `thread_history_materialization.rs` — ordinal 누락 시 하드 에러 유지. + `update_thread_metadata.rs:87,99,121` — `paginated` 분기 유지. + +## G14 확정: 카탈로그가 멀티에이전트 지시문을 공급한다 (#38619) + +`gh pr view 38619` 본문: "Add model-catalog messages for root and subagent roles, explicit +delegation, and delegation hints. Resolve role instructions in **config, catalog, then +bundled-default order**." + +신규 타입 (`codex-rs/protocol/src/openai_models.rs:577-592`): + +```rust +pub struct MultiAgentMessages { pub role: Option, pub mode: Option } +pub struct MultiAgentRoleMessages { pub root: Option, pub subagent: Option } +pub struct MultiAgentModeMessages { pub explicit: Option, pub hint_text: Option } +``` + +`ModelMessages.multi_agent` (`:534`) 아래로 들어간다. **opencodex는 `model_messages` 를 +이미 합성/변형한다** — `src/codex/catalog/metadata.ts:300-308` 이 `instructions_template` 을 +`identifyRoutedModel` 로 재작성하고, `src/codex/data/upstream-models.json` 이 8개 네이티브 +행에 `model_messages` 를 담고 있다. 즉 이 스냅샷은 `multi_agent` 서브트리를 모르는 상태이며, +라우팅된 모델에 대해 role/mode 지시문이 누락되거나 낡은 채로 공급될 수 있다. + +**분류: missed-opportunity → 잠재적 silent-degradation.** Phase 1의 후속 work-phase 후보 +(신규 decade doc `060`)로 등록 권고. 이번 문서 사이클에서는 구현하지 않는다. + +## 정정 + +008이 "008 시점에 `fetch` 했다"고만 적었으므로, 실제 **ff는 이 문서 시점에 수행**됐다. +로컬 `main` 은 이제 `49db349ff` 이며 `origin/main` 과 동일하다. + diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/010_phase1_catalog_capability_contract.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/010_phase1_catalog_capability_contract.md new file mode 100644 index 0000000000..3722c2c15c --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/010_phase1_catalog_capability_contract.md @@ -0,0 +1,243 @@ +# Phase 1 — catalog multi-agent capability contract + +Closes G1, G2, G9, G12, G13. One PABCD cycle. Independent of every other phase. + +Audit history: `006_audit_round1.md`, `007_audit_round2.md`. This document is canonical — +all audit corrections are integrated below, not appended. + +## Why + +`6d4d9442c` changed `multi_agent_version` from an eligibility gate into a **child +capability declaration**. Verified at upstream HEAD `9dd22890f`: + +- `codex-rs/core/src/tools/handlers/multi_agents_common.rs:36-42` — + `model_supports_multi_agent_backend` returns true unless the model's value is + `Some(Disabled)`. Eligibility no longer requires `v2`. +- `codex-rs/core/src/tools/spec_plan.rs:599-610` — `collab_tools_enabled` grants a + **child** (`session_source.get_agent_path().is_some()`) collaboration tools only when + its own catalog value is exactly `Some(MultiAgentVersion::V2)`. +- `codex-rs/models-manager/models.json` — sol/terra `"v2"` (lines 21, 136), luna `"v1"` + (line 249), gpt-5.5 and others `null` (lines 358, 465, 570, 670, 767). + +| Value | Offered to a v2 parent | Child gets collab tools | Meaning | +| --- | --- | --- | --- | +| `"v2"` | yes | yes | recursive delegator | +| `"v1"` | yes | no | **leaf worker** | +| absent/null | yes | no | **leaf worker** | +| `"disabled"` | no | no | ineligible | + +## Current state (verified this session) + +| Location | Fact | +| --- | --- | +| `src/codex/catalog/sync.ts:105-108` | `isEligibleV2SubagentEntry` true for `v2`/null/undefined — **excludes `v1`**, so Luna is excluded | +| `src/codex/catalog/sync.ts:157` | roster filter uses that predicate | +| `src/codex/catalog/sync.ts:190-193` | exclusion reasons reuse it (`surface_incompatible`) | +| `src/codex/catalog/parsing.ts:382-388` | `default` mode + v2 feature stamps unpinned rows `"v2"` | +| `src/codex/catalog/sync.ts:627` | `applyMultiAgentMode` call in `buildCatalogEntries` | +| `src/codex/catalog/sync.ts:1087` | `applyMultiAgentMode` call in the merge/sync path | +| `src/codex/catalog/provider-fetch.ts:999` | `catalogHintsFromModelsApiItem` does not read `multi_agent_version` | +| `src/codex/catalog/effort.ts:113` | `applyCatalogModelMetadata` — the RawEntry stamping seam, called from `sync.ts:322` | +| `src/server/management/model-rows.ts:67` | native management rows built field-by-field | +| `gui/src/pages/models-shared.ts:28` | GUI model type omits the field | +| `src/codex/subagent-model-fallback.ts:269` | `selectAvailableSubagentModel` — quota/health only | +| `src/codex/subagent-model-fallback.ts:510` | `applySubagentModelFallback` | +| `src/server/responses/core.ts:1768`, `:1856` | **both** fallback call sites | +| `src/server/responses/collaboration.ts:349` | fork-override guidance text | + +## Change 1 — eligibility follows upstream (`src/codex/catalog/sync.ts:105-108`) + +```ts +export function isEligibleV2SubagentEntry(entry: RawEntry): boolean { + // Since 6d4d9442c a v2 parent may spawn ANY model except explicit "disabled" + // (multi_agents_common.rs:36-42). A "v1" pin means "eligible LEAF": the child simply + // receives no collaboration tools (spec_plan.rs:599-610). + return entry.multi_agent_version !== "disabled"; +} +``` + +Rewrite the doc comment at `sync.ts:90-104` (it cites the superseded `92938d880` equality +rule). Keep the three-way distinction — it now maps to eligible-recursive / +eligible-leaf / excluded. `surface_incompatible` (`:190-193`) now fires only for an +explicit `disabled` pin; keep the reason code and update its message. + +## Change 2 — drop the blanket v2 stamp (`src/codex/catalog/parsing.ts:382-388`) + +```ts + if (typeof upstreamPin === "string") { + entry.multi_agent_version = upstreamPin; + } else if (typeof entry[OCX_MULTI_AGENT_FIELD] === "string") { + entry.multi_agent_version = entry[OCX_MULTI_AGENT_FIELD]; // Change 3 provenance + } else { + // Absent means LEAF, not "refused". Stamping "v2" would claim every routed + // third-party model is a recursive delegator. + delete entry.multi_agent_version; + } +``` + +Rewrite the `@param v2FeatureEnabled` comment at `parsing.ts:345-360` (states the +superseded "clean refusal at spawn time" rationale). Mark the parameter `@deprecated` and +stop reading it; leave it in the signature so the two call sites need no change this cycle. +Explicit `mode === "v1"` / `"v2"` force-all behavior is UNCHANGED. + +## Change 3 — the capability's creation path and bridge + +**NEW** constant in `src/codex/catalog/parsing.ts` beside `SPAWN_PRIORITY_FIELD` +(`sync.ts:80` precedent): `export const OCX_MULTI_AGENT_FIELD = "opencodex_multi_agent_version";` + +**MODIFY** `src/codex/catalog/provider-fetch.ts:999` `catalogHintsFromModelsApiItem`: + +```ts + // CANONICAL: multi_agent_version is a TOP-LEVEL ModelInfo field (openai_models.rs:459-460). + // The metadata-nested form is only a tolerated fallback for providers that mirror it there. + const declared = + (typeof item.multi_agent_version === "string" ? item.multi_agent_version : undefined) + ?? (typeof metadata?.multi_agent_version === "string" ? metadata.multi_agent_version : undefined); + // Only "v2" | "v1" | "disabled" are meaningful; anything else is treated as absent, + // matching upstream deserialize_optional_model_selector (openai_models.rs:322-331). +``` + +**MODIFY** `src/codex/catalog/effort.ts:113` `applyCatalogModelMetadata` — without this +stamp the marker never exists and `applyMultiAgentMode` can never see it: + +```ts + // Private marker: survives strict normalization so applyMultiAgentMode can serialize the + // declared capability without inferring it. Same pattern as the owned_by combo marker. + if (model.multiAgentVersion) entry[OCX_MULTI_AGENT_FIELD] = model.multiAgentVersion; +``` + +**MODIFY** `CatalogModel` (`src/codex/catalog/parsing.ts:94`): add +`multiAgentVersion?: "disabled" | "v1" | "v2"`. + +Field chain (PLAN-FIELD-CHAIN-01): + +| Stage | Exact location | +| --- | --- | +| creation — provider `/models` | `src/codex/catalog/provider-fetch.ts:999` (top-level field) | +| creation — native pins | `src/codex/catalog/metadata.ts:174` `nativeMultiAgentVersion` (exists) | +| creation — user config | **N/A** — no per-model user control this cycle; `ocx v2 mode` is global | +| type | `CatalogModel` `parsing.ts:94` | +| **bridge** | `applyCatalogModelMetadata` `effort.ts:113` (called `sync.ts:322`) stamps `entry[OCX_MULTI_AGENT_FIELD]` | +| serialization | `applyMultiAgentMode` writes snake_case `multi_agent_version` (`openai_models.rs:372-461`) | +| deserialization | `readCatalog`; unknown strings preserved verbatim, never coerced | +| consumer — roster | `isEligibleV2SubagentEntry`, `effectiveSubagentRoster` `sync.ts:142` | +| consumer — management | `src/server/management/model-rows.ts:67` — native rows must carry it explicitly | +| consumer — fallback | Change 4 | +| GUI — **type projection, N/A as a consumer** | `gui/src/pages/models-shared.ts:28` gets the optional field for type fidelity only; no GUI surface renders it this cycle, and a type without a renderer is not a consumer. Rendering is deliberately deferred. | + +Do **not** emit camelCase `multiAgentVersion` on the `/models` wire; camelCase belongs only +to app-server `model/list` (`v2/model.rs:110`), which opencodex does not serve. + +## Change 4 — fallback preserves the capability class (G12) + +`applySubagentModelFallback` rewrites a child's model *after* Codex built that child's +tool surface, and `selectAvailableSubagentModel` (`:269`) never checks capability. A `v2` +child quota-swapped to a `v1`/`disabled` model keeps collaboration tools it cannot honor. + +ADD to `src/codex/catalog/sync.ts`, beside the predicate: + +```ts +export type SubagentCapabilityClass = "recursive" | "leaf" | "excluded"; + +export function subagentCapabilityClass(entry: RawEntry | undefined): SubagentCapabilityClass { + if (!entry) return "leaf"; // unknown model: assume leaf, never recursive + if (entry.multi_agent_version === "disabled") return "excluded"; + return entry.multi_agent_version === "v2" ? "recursive" : "leaf"; +} +``` + +`selectAvailableSubagentModel` gains an **additive optional** `requiredClass?: SubagentCapabilityClass` +and resolves each candidate's class from the active catalog via `configuredCatalogEntry` +(`sync.ts:127-130`). A candidate weaker than `requiredClass` is skipped; `excluded` is never +selected. Omitting the parameter preserves today's behavior exactly. When no same-class +candidate remains, prefer failing the fallback over silently downgrading. + +**Both** call sites must pass it: `src/server/responses/core.ts:1768` and `:1856`. + +## Change 5 — fork-override guidance (G13, OPTIONAL) + +Upstream's `DEFAULT_MULTI_AGENT_V2_MODEL_OVERRIDE_USAGE_HINT_TEXT` +(`codex-rs/core/src/config/mod.rs:253`) states full-history forks "do not accept +overrides", while the implementation honors them (`multi_agents_v2/spawn.rs:39-99`; test +`subagent_notifications.rs:1040-1087`). opencodex's `collaboration.ts:349` text therefore +**mirrors upstream guidance and is not wrong**. + +Settled outcome: keep the wording aligned with upstream's hint. Do NOT assert a runtime +rejection, and do NOT claim upstream merely "discourages" it. This change is optional +polish, not a defect fix. + +## Change 6 — SoT doc sync + +MODIFY `structure/03_catalog-and-subagents.md:133-153`: replace the `"default"` row's +"codex feature flag decides" with the three-state semantics table; cite `6d4d9442c` and +`spec_plan.rs:599-610`. + +## Tests + +`tests/multi-agent-compat.test.ts` (roster anchors `:243, :293, :408, :471`): + +1. **Luna is in the roster** — `effectiveSubagentRoster(["gpt-5.6-luna"], "v2")` lists it in + `advertised`. Fails today with `surface_incompatible` — activation evidence. +2. `disabled` is the ONLY capability-based exclusion. +3. `candidates` still capped at `MAX_SPAWN_AGENT_MODEL_OVERRIDES` (5). + +`tests/native-model-toggle.test.ts` (anchor `:324`): + +4. Four-row matrix (`v2`/`v1`/absent/`disabled`) with the v2 flag ON and OFF. The + routed-row-with-flag-ON case asserts `Object.hasOwn(entry, "multi_agent_version") === false` + — currently `"v2"`. + +`tests/codex-catalog.test.ts` / `tests/codex-v2-gate.test.ts` (anchor `:1899`): + +5. A provider `/models` item declaring a top-level `multi_agent_version` reaches the wire. +6. An unknown declared value is treated as absent. +6b. **Management projection** — `listManagementModelRows` (`src/server/management/model-rows.ts:67`) emits the capability on a NATIVE row (routed rows inherit it by spreading `CatalogModel`; native rows are built field-by-field and would silently drop it). Assert via the `/api/models` route suite. + +`tests/subagent-model-fallback.test.ts`, `tests/subagent-model-fallback-api.test.ts`, +`tests/subagent-fallback-handle-responses.test.ts`: + +7. A `v2` child quota-swapped to a `v1` candidate does not silently downgrade, on both + runtime paths (`core.ts:1768` and `:1856`). + +## Verification + +```bash +bun install # REQUIRED: this worktree has no node_modules +cd gui && bun install && cd .. # REQUIRED for lint:gui (oxlint) +bun test tests/multi-agent-compat.test.ts tests/native-model-toggle.test.ts tests/codex-catalog.test.ts tests/codex-v2-gate.test.ts tests/subagent-model-fallback.test.ts tests/subagent-model-fallback-api.test.ts tests/subagent-fallback-handle-responses.test.ts +bun x tsc --noEmit +bun run lint:gui # Change 3 touches gui/src/pages/models-shared.ts +``` + +**Receipts (PLAN-VERIFIER-REAL-01, measured 2026-08-16 in this dependency-less worktree).** +Each row is the exact command named above: + +| Command | Exit | Observed | +| --- | --- | --- | +| the 7-file `bun test` command | 1 | `0 pass, 7 fail, 7 errors; Ran 7 tests across 7 files` — `Cannot find module 'zod/v4'` | +| `bun x tsc --noEmit` | 1 | `TS2688: Cannot find type definition file for 'bun-types'` | +| `bun run lint:gui` | **127** | `oxlint: command not found` (missing binary, not a lint failure) | + +All three are environmental: `ls node_modules` → absent. B runs both installs first and +re-records real exit codes. + +Target observation: `tests/native-model-toggle.test.ts:23` imports `applyMultiAgentMode` +from `../src/codex/catalog/parsing`; `tests/multi-agent-compat.test.ts:13` imports +`effectiveSubagentRoster` from `../src/codex/catalog`. Both gates read this phase's targets. +The `structure/` prose edit is read by no gate — **human review**. + +## Accept criteria + +1. `effectiveSubagentRoster(["gpt-5.6-luna"], "v2")` advertises Luna. +2. Only an explicit `disabled` pin excludes a model for capability reasons. +3. `applyMultiAgentMode(rows, "default", true)` leaves an unpinned row's key absent. +4. A provider-declared top-level capability survives creation → bridge → serialization → + roster → management. +5. Quota fallback never downgrades a `v2` child, on both call sites. +6. Fork guidance stays aligned with upstream's hint text (optional polish). +7. `structure/03_catalog-and-subagents.md` states the three-state semantics. + +## Out of scope + +`ocx v2 mode` CLI semantics; a per-model user-facing capability control; GUI rendering of +the capability; the quota/health logic itself (only its capability awareness changes). diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/020_phase2_history_mode_awareness.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/020_phase2_history_mode_awareness.md new file mode 100644 index 0000000000..f691f562ab --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/020_phase2_history_mode_awareness.md @@ -0,0 +1,306 @@ +# Phase 2 — history-mode awareness in the Codex history integration + +Closes G3, G4. One PABCD cycle. Independent of every other phase. + +Audit history: `006_audit_round1.md`, `007_audit_round2.md`. This document is canonical — +all audit corrections are integrated below, not appended. + +## Why this is a compat-break + +Paginated rollouts (`6bb6e9045`, `4bb7ee347`) give every JSONL line a monotonically +increasing `ordinal` and set `SessionMeta.history_mode = "paginated"` (`ThreadHistoryMode`, +`codex-rs/protocol/src/protocol.rs:700`). Verified rejection — +`codex-rs/thread-store/src/local/thread_history_materialization.rs:170-186`: + +```rust + None => { + return Err(ThreadStoreError::Internal { + message: format!( + "paginated rollout line for {thread_id} is missing an ordinal" + ), + }); + } +``` + +opencodex appends an ordinal-less `session_meta` line unconditionally. Upstream's +equivalent branches on mode (`codex-rs/thread-store/src/local/update_thread_metadata.rs:74`) and skips legacy `SessionMeta` +persistence when paginated. + +## Current state (verified this session) + +| Location | Fact | +| --- | --- | +| `src/codex/history-provider.ts:518` | `parseThreadFieldsFromRolloutText` ALREADY reads `payload.history_mode` | +| `src/codex/history-provider.ts:401` | `historyMode?: string` on the parsed-fields type | +| `src/codex/history-provider.ts:179` | `interface ThreadRow` — no `history_mode` | +| `src/codex/history-provider.ts:165-176` | `CodexHistorySyncResult`: `rows`, `files`, `ejectedRows?`, `failed?`, `failureReason` (`:176`) | +| `src/codex/history-provider.ts:340` | `rememberOriginal` — backup entry serializer | +| `src/codex/history-provider.ts:523+` | `updateSessionMeta` — always appends | +| **three mutation paths** | sync `:705` (SELECT `:714`/`:722`, backup `:731`, bulk update `:750`); restore `:780` (loop `:801`, manifest clear `:815`); **ejection `:586`** (SELECT `:588`, writes `:599`, bulk update `:608-618`, reachable from `:786`) | +| `src/storage/cleanup.ts:631-655` | precedent: `columnExists` + conditional select | +| `src/storage/cleanup.ts:673` | precedent: refuses cleanup when any thread is `paginated` | + +## Change 1 — shared `columnExists` + +Three SQLite helpers are currently **private** to `src/storage/cleanup.ts` and are all +needed by the history provider: + +| Helper | Current location | Currently exported? | +| --- | --- | --- | +| `columnExists` | `src/storage/cleanup.ts` | no | +| `SQLITE_ID_CHUNK` (= 200) | `src/storage/cleanup.ts:112` | no | +| `chunkIds` | `src/storage/cleanup.ts:123` | no | + +Importing `cleanup.ts` into the history provider would drag unrelated heavy imports. +**Move all three** to `src/codex/sqlite-columns.ts` (NEW, ~20 lines), export them there, and +import them back into `cleanup.ts` so its existing call sites (`:631`, `:635`, `:2318`, and +every `chunkIds`/`SQLITE_ID_CHUNK` use) keep working unchanged. Cleanup's own exports and +tests are preserved. One definition each, no new dependency edge. + +## Change 2 — carry `history_mode` on thread rows + +`interface ThreadRow` (`:179`) gains `history_mode?: string | null`. + +There are **three existing SELECTs**: `:588` (ejection), `:712` (openai/resumable rows), +and `:720` (opencodex exec rows). Each becomes column-guarded: + +```ts +const hasHistoryMode = columnExists(db, "threads", "history_mode"); +const cols = ["id", "rollout_path", "model_provider", "source", "has_user_event"]; +if (hasHistoryMode) cols.push("history_mode"); +``` + +The guard is required: an older `state_5.sqlite` predates the column and an unguarded +SELECT throws. + +**A fourth, NEW SELECT is required for restore.** `restoreCodexHistoryProvider` (`:780`) +restores directly from manifest entries and issues **no row query at all**, so it has no +`ThreadRow` to resolve a mode from. Without one, a thread whose live DB mode is unknown +(e.g. `"sharded"`) while its rollout file lacks that mode would be treated as legacy — +violating fail-closed. Add a column-guarded batch lookup keyed by the manifest's thread ids +before partitioning: + +```ts +// Restore has no live rows of its own; the manifest is a snapshot that may predate a +// migration. Resolve the CURRENT mode per id before deciding how to restore. +const liveRows = new Map(); +for (const chunk of chunkIds(entries.map(e => e.id), SQLITE_ID_CHUNK)) { + const placeholders = chunk.map(() => "?").join(","); + for (const row of db.query( + \`SELECT \${cols.join(", ")} FROM threads WHERE id IN (\${placeholders})\` + ).all(...chunk)) liveRows.set(row.id, row); +} +``` + +**An entry with no live row (thread deleted since backup) is terminally resolved, not +skipped:** there is nothing left to restore, so its manifest entry is REMOVED and it is not +counted as skipped work. This keeps the partial-work invariant honest — only +`skippedUnknownMode` (work we refused to do) blocks `converged`, while a deleted thread is +genuinely done. Do not conflate the two. + +## Change 3 — resolve the mode + +```ts +type HistoryMode = "legacy" | "paginated" | "unknown"; + +function resolveHistoryMode(row: ThreadRow, rolloutPath: string): HistoryMode { + const fromRow = (row.history_mode ?? "").toLowerCase(); + if (fromRow === "paginated") return "paginated"; + if (fromRow === "legacy") return "legacy"; + if (fromRow) return "unknown"; // future mode: fail closed + const fromFile = (readThreadFieldsFromRollout(rolloutPath)?.historyMode ?? "").toLowerCase(); + if (fromFile === "paginated") return "paginated"; + if (fromFile === "legacy" || fromFile === "") return "legacy"; // absent == legacy (serde default) + return "unknown"; +} +``` + +Absent means legacy: `ThreadHistoryMode` derives `#[default] Legacy` (`protocol.rs:700`). + +## Change 4 — partition ALL THREE mutation paths + +Sync and ejection currently select rows and then bulk-update them; restore has no query of +its own and obtains its rows from the new Change 2 lookup. All three must partition BEFORE +mutating, with **id-scoped statements only** — never a table-wide UPDATE: + +```ts +const partitioned = { legacy: [] as ThreadRow[], paginated: [] as ThreadRow[], unknown: [] as ThreadRow[] }; +for (const row of rows) partitioned[resolveHistoryMode(row, row.rollout_path)].push(row); +``` + +| Path | Legacy rows | Paginated rows | Unknown rows | +| --- | --- | --- | --- | +| sync `syncCodexHistoryProviderUnsafe` `:705` | append `session_meta` + line-1 patch; `UPDATE ... WHERE id = ?` | **no rollout write**; DB update only | excluded from every statement | +| restore `restoreCodexHistoryProvider` `:780` | current restore behavior | DB update only | skipped, backup entry retained | +| **ejection `ejectRemainingOpencodexHistory` `:586`** | current write at `:599` | **no rollout write**; DB update only | excluded from `:608-618` | + +The `unknown` partition must never appear in any **mutating** `WHERE id IN (...)` list (it is +necessarily READ first — that is how its mode is discovered) — that is the +difference between the claimed contract and today's bulk update. + +`updateSessionMeta` gains a mode parameter and returns false for anything but `legacy`, so +no caller (including ejection) can reach a paginated file through it. + +## Change 5 — backup discipline + +`rememberOriginal` (`:340`) currently runs for **every** selected row before mutation +(`:731`). Call it ONLY for rows about to be mutated — otherwise restore would later +"restore" a row that was never touched. + +**Do not add a `historyMode` field to the backup entry.** Restore must resolve the live +mode regardless (a thread can migrate after the backup was taken), so a stored mode would +be a ghost field with no trustworthy consumer. The manifest shape is unchanged. + +Restore clears the whole manifest at `:815`. It must clear exactly the entries that are +**done** — either successfully restored, or terminally resolved because no live row exists +(deleted thread) — and **retain** entries refused for unknown mode. Otherwise an +unknown-mode thread silently loses its backup and can never be restored. + +## Change 6 — `skippedUnknownMode` and its full transport chain + +The count crosses a Worker boundary, so a result field alone is invisible: + +| Stage | Location | Change | +| --- | --- | --- | +| producer | `src/codex/history-provider.ts:165-176` `CodexHistorySyncResult` | add `skippedUnknownMode?: number` | +| worker DTO | `src/codex/history-worker.ts:69` `HistoryWorkerResult` | add to the `done` variant | +| worker construction | `src/codex/history-worker.ts:184` | populate it | +| message validation | `src/codex/history-job.ts:152` | accept/validate it | +| job outcome | `src/codex/history-job.ts:107` `CodexHistoryJobOutcome` | add to `converged` | +| classification | `src/codex/history-job.ts:282` | carry through | +| consumers | `src/codex/inject.ts:1023`, `:1515` (restore), `src/codex/history-migration-guardian.ts:49` | surface it | +| durable state | `src/codex/history-transition.ts:28` `classify` | see below | + +**Decision on `classify` (SETTLED — do not redesign).** Reusing `skipped` is wrong: +`history-transition.ts:42` maps `skipped` to `status: "converged"` precisely because a user +opting out is a completed decision. Unknown-mode rows are the opposite — work we refused to +do and must retry after a Codex upgrade teaches us the mode. + +`CodexHistoryState.status` (`src/codex/convergence-types.ts:36`) has no `partial` member and +its `reason` union has no fitting value. The settled design adds **one reason, not a new +status**: + +| Element | Change | +| --- | --- | +| `CodexHistoryState.reason` (`convergence-types.ts:36+`) | add `"unknown-history-mode"` to the union | +| `classify` (`history-transition.ts:28`) | for a `converged` outcome carrying `skippedUnknownMode > 0`, return `{ status: "pending", reason: "unknown-history-mode", attempts: 1, nextRetryAt: }` instead of `converged` | +| retry semantics | `pending` already carries the durable retry schedule — a later run re-attempts those rows, which is exactly right: a Codex upgrade may make the mode known | +| `skipped` | UNCHANGED — still `converged` (user opt-out) | + +`pending` is chosen over `blocked` because nothing is broken: the rows are simply not +understood yet, and the existing retry path will revisit them. Do NOT add a `partial` +status: it would require every `status` consumer to learn a new state, whereas a new +`reason` on the existing `pending` status is inert for consumers that do not inspect it. + +**Durable persistence requires a schema migration — the type change alone is rejected.** +`src/codex/transition-state.ts` enforces the reason vocabulary in THREE places, and a +literal implementation of the type change would make the transition write fail with +`unavailable/database`: + +| Layer | Location | Content | +| --- | --- | --- | +| runtime allowlist | `transition-state.ts:42` `DURABLE_HISTORY_REASONS` | `db-busy, permission, unreadable, schema, timeout, shutdown-cancelled, worker-died, overtaken, record-write-failed` | +| write validation | `transition-state.ts:249` `validateHistoryWrite` | rejects any reason outside that set | +| **SQLite CHECK constraint** | `transition-state.ts:70-72` | the same list embedded in the table DDL (line 69 is the *status* CHECK) | + +Updating only the table-creation SQL is insufficient: an **existing** database already +carries the old CHECK, and `COORDINATOR_SCHEMA_VERSION` is `1` (`:40`) with +`PRAGMA user_version` gating at `:284-285` — a mismatched version is refused outright. + +Settled migration plan: + +1. Add `"unknown-history-mode"` to `DURABLE_HISTORY_REASONS` (`:42`) and to the CHECK list + in the DDL (`:70-72`). +2. Bump `COORDINATOR_SCHEMA_VERSION` to `2` (`:40`). +3. Add a v1→v2 migration. SQLite cannot alter a CHECK constraint in place, so the migration + is the standard table rebuild: create the new table under a temporary name with the + widened CHECK, `INSERT INTO ... SELECT` the existing single row, drop the old table, + rename, then set `PRAGMA user_version = 2` — all inside the existing `BEGIN IMMEDIATE` + transaction discipline (`:424`, `:578`). +4. The version gate at `:284-285` must accept a v1 database and migrate it rather than + refusing it. + +**Alternative if the migration is judged too heavy for this phase:** reuse the existing +`"schema"` reason with `status: "pending"` and no vocabulary change at all. It is a +defensible fit (we do not understand the on-disk shape) and costs zero migration. Decide at +P and record which was chosen — but do NOT implement the new reason without steps 1-4. + +Affected files/tests: `src/codex/convergence-types.ts`, `src/codex/history-transition.ts`, +`src/codex/transition-state.ts`, `tests/codex-transition-state.test.ts` (real, anchor `:61`), +plus the suite covering `classify` (confirm its filename during P and record it). + +## Tests (`tests/codex-history-provider.test.ts`, anchor `:98`) + +1. **Paginated rollout byte-identical after sync** — hash before/after; assert unchanged + AND that the `threads` row provider changed. Today the file grows by one ordinal-less + line — activation evidence. +2. **Ordinals stay contiguous** after a sync. +3. **Legacy path unchanged** — existing append/line-1 assertions still pass. +4. **Missing `history_mode` column** — syncs as legacy, does not throw. +5. **Unknown mode** (`"sharded"`) — the row's provider value is unchanged in the DB (not + merely "no file written") and `skippedUnknownMode === 1`. Activation scenario for the + fail-closed branch. +6. **Mixed batch** — legacy + paginated + unknown in one sync. +7. **Restore after migration** — backup captured as legacy, thread now paginated; DB only. +8. **Ejection path** — `ejectRemainingOpencodexHistory` with a mixed set: paginated files + byte-identical, unknown rows untouched in the DB. +9. **Backup scoping** — an unknown-mode row is not written to the backup manifest. +10. **Partial restore** — a manifest with one restorable and one unknown entry keeps the + unknown entry after restore. +10b. **Backup legacy, live row unknown** — the manifest entry says legacy but the current + `threads` row is `"sharded"`: restore must skip it (fail-closed) and retain its backup + entry. This is the activation scenario for the new restore SELECT; without that query + the row would be restored as legacy. +10c. **Manifest entry with no live row** — thread deleted since backup: its manifest entry + is REMOVED, it is NOT counted in `skippedUnknownMode`, and the job outcome may still be + `converged` (nothing remained to restore). +11. **Transport** — `skippedUnknownMode` survives the Worker round-trip into the job + outcome, and a run with skips is not classified `converged`. +11b. **Durable persistence** — the chosen reason actually WRITES: exercise a real transition + state write and read it back. If the new reason was chosen, also assert that an + EXISTING v1 database migrates and accepts it (the old CHECK constraint would reject it). + +Must stay green: `tests/codex-history-job.test.ts`, `tests/codex-history-worker.test.ts`, +`tests/codex-history-writer.test.ts`, `tests/codex-native-residue.test.ts`, +`tests/storage-cleanup.test.ts` (re-exported `columnExists`). + +## Verification + +```bash +bun install # REQUIRED: this worktree has no node_modules +bun test tests/codex-history-provider.test.ts tests/codex-history-job.test.ts tests/codex-history-worker.test.ts tests/codex-history-writer.test.ts tests/codex-native-residue.test.ts tests/codex-transition-state.test.ts tests/storage-cleanup.test.ts +bun x tsc --noEmit +``` + +**Receipts (measured 2026-08-16, dependency-less worktree).** Each row is the exact command +named above: + +| Command | Exit | Observed | +| --- | --- | --- | +| the 7-file `bun test` command | 1 | `0 pass, 7 fail; Ran 7 tests across 7 files` — `Cannot find module 'zod/v4'` (the 5-file form reproduced 5/5) | +| `bun x tsc --noEmit` | 1 | `TS2688: Cannot find type definition file for 'bun-types'` | + +Environmental — `ls node_modules` → absent. B runs `bun install` and re-records. +`package.json:41` defines `"test": "bun scripts/test.ts"`; use `bun run test` for a full run. + +Target observation: `tests/codex-history-provider.test.ts` imports +`src/codex/history-provider.ts` directly. + +## Accept criteria + +1. A paginated rollout is byte-identical before and after sync, restore, AND ejection. +2. Its `threads` row still receives the provider/source change. +3. An unknown-mode row is absent from every **mutation** statement (it is necessarily READ, to discover its mode) and counted in `skippedUnknownMode`. +4. Legacy rollouts keep current append + line-1 behavior exactly. +5. A `threads` table without `history_mode` does not throw. +6. Backups cover only mutated rows; restore clears entries that were restored OR terminally + resolved (no live row), and retains only entries refused for unknown mode. +6b. Restore resolves each thread's mode from a LIVE row query, never from the manifest. +7. `skippedUnknownMode` reaches the job outcome, and partial work is not `converged`. +8. `columnExists`, `chunkIds`, and `SQLITE_ID_CHUNK` each have exactly one definition, in `src/codex/sqlite-columns.ts`. +9. A deleted thread's manifest entry is removed and does not block `converged`; only refused work does. + +## Out of scope + +Writing ordinals; maintaining `thread_history_1.sqlite`; changing `src/codex/paths.ts`; +relaxing the `src/storage/cleanup.ts:673` paginated refusal (G10 — that refusal stays). diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/030_phase3_rollout_identity_and_previews.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/030_phase3_rollout_identity_and_previews.md new file mode 100644 index 0000000000..96f778c81d --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/030_phase3_rollout_identity_and_previews.md @@ -0,0 +1,126 @@ +# Phase 3 — rollout identity hardening (G6). G5 downgraded to conditional. + +**Independent of every other phase.** (Round 1 claimed a Phase 2 dependency; the +reviewer proved neither preview parsing nor identity handling consumes Phase 2's +resolver. Corrected.) + +> **A-phase amendment (round 1).** The reviewer proved G5's stated impact is NOT +> reachable: `extractUserMessagePreview` output is consumed only by +> `reconstructThreadRowFromRollout` (`src/storage/cleanup.ts:2305`), and paginated +> threads are refused before cleanup ever runs (`cleanup.ts:673`). Sync/restore use the +> DB's existing `has_user_event` column, not the reconstructed preview. So a migrated +> thread is NOT misclassified by this code path. G5 is therefore **conditional** — see +> the gate below. The round-1 fixture recipe was also wrong (`rg -rn` parses `-r n` as a +> replacement); corrected. + +## G6 — the filename no longer implies the thread id (unconditional) + +`4ef836f88` ("Distinguish rollout IDs from thread IDs", 2026-08-12) permits several +rollout ids/files per thread id; `codex-rs/rollout/src/rollout_file_name.rs` owns +`RolloutFileName`, where reverted/alternate rollouts carry a distinct rollout id. +`4496ba3fd` makes upstream validate a path by its first `SessionMeta.id` rather than by +its name. + +opencodex reads `threads.rollout_path` from the DB — correct today. This phase's job is +to **prove** that and prevent regression, since the failure mode is silent +cross-thread corruption. + +### Audit + lock + +| Location | Requirement | +| --- | --- | +| `src/codex/history-provider.ts:459` `readThreadFieldsFromRollout` | id from parsed `session_meta.payload.id` only | +| `src/codex/history-provider.ts:530` `updateSessionMeta` | already compares `payload.id !== expectedId` — keep; add a comment citing `4496ba3fd` | +| `src/storage/cleanup.ts:222` `normalizeArchivedRolloutPath` | must not derive a thread id from the filename stem | +| `src/storage/cleanup.ts:2297-2312` `reconstructThreadRowFromRollout` | binds `id` from `entry.threadId` (manifest), not the filename — verify and lock | + +Expected outcome: **no behavior change**, only regressions. If the audit finds a +filename-derived id, that becomes a real fix in this cycle. + +### Tests (MODIFY `tests/codex-history-provider.test.ts`, `tests/storage-cleanup.test.ts`) + +1. **Alternate rollout id filename** — a file named + `rollout--_.jsonl` whose `session_meta.payload.id` is the + thread id resolves correctly through sync, backup, and restore. +2. **Mismatched id still refuses** — regression-lock the existing `:530` guard. +3. **Archived cleanup with an alternate-id filename** — listing/normalization behave. + +## G5 — canonical `ItemCompleted` previews (CONDITIONAL) + +`src/codex/history-provider.ts:420` `extractUserMessagePreview` recognizes legacy +`event_msg` and raw `response_item` user messages, but not the canonical +`EventMsg::ItemCompleted { item: TurnItem::UserMessage(..) }` records migration produces. + +**Gate — run this BEFORE writing any parsing code.** G5 is worth implementing only if a +production caller can reach a paginated rollout AND consume the reconstructed preview. +Enumerate every `extractUserMessagePreview` / `readThreadFieldsFromRollout` caller and +answer, per caller, whether a paginated file can reach it: + +```bash +cd /Users/jun/.codex/worktrees/e80c/opencodex +rg -n 'extractUserMessagePreview|readThreadFieldsFromRollout|parseThreadFieldsFromRolloutText' src/ scripts/ +``` + +Known callers and their current verdict: + +| Caller | Paginated reachable? | +| --- | --- | +| `src/storage/cleanup.ts:2305` `reconstructThreadRowFromRollout` | **No** — `cleanup.ts:673` refuses paginated threads first | +| `src/codex/history-provider.ts` sync/restore | Uses the DB's `has_user_event`, not the preview | +| `scripts/openai-provider-option-runtime-smoke.ts:343` | Diagnostic script, its own parsing | + +If every row stays "no": **record the finding, delete G5 from the gap matrix, and close +this phase with G6 only.** That is an honest NOOP for G5, not a skipped item. + +If a reachable caller IS found, implement the third branch — ordered after the existing +two so legacy parsing is untouched — with these upstream anchors: + +| Symbol | Location | +| --- | --- | +| `ItemCompletedEvent` | `codex-rs/protocol/src/protocol.rs:1850` | +| `TurnItem::UserMessage` | `codex-rs/protocol/src/items.rs:44` | +| `UserMessageItem` | `codex-rs/protocol/src/items.rs:78` | +| `UserInput` (snake_case-tagged) | `codex-rs/protocol/src/user_input.rs:15` | + +**Do not guess the serialized tag.** `TurnItem` carries no `rename_all`, so the variant +key must be read from generated output. Correct commands (note `-n`, not `-rn` — `rg -r` +means *replace*): + +```bash +cd /Users/jun/Developer/codex/121_openai-codex +rg -n 'enum TurnItem' -A 30 codex-rs/protocol/src/items.rs +rg -n 'serde' -B 3 -A 3 codex-rs/protocol/src/items.rs | head -40 +``` + +If no serialized fixture exists upstream, generate one through upstream serde (a throwaway +Rust test that serializes the variant and prints it) rather than hand-authoring JSON — +a hand-written fixture matching a wrong spelling would pass the test while production fails. + +## Verification + +```bash +bun install # REQUIRED FIRST: this worktree has no node_modules +bun test tests/codex-history-provider.test.ts tests/storage-cleanup.test.ts tests/codex-native-residue.test.ts +bun x tsc --noEmit +``` + +**Verifier receipts (recorded 2026-08-16).** In this worktree the test command exits `1` +with `0 pass, 3 fail` — `Cannot find module 'zod/v4'` — and `bun x tsc --noEmit` exits `1` +with `TS2688: Cannot find type definition file for 'bun-types'`. Environmental: +`node_modules/` is absent. B runs `bun install` and re-records. `package.json:41` defines +`"test": "bun scripts/test.ts"`. + +Target observation: both suites import the changed modules directly. + +## Accept criteria + +1. No code path derives a thread id from a rollout filename (proven by audit + regression). +2. Alternate-rollout-id files survive sync/restore/cleanup unchanged. +3. The mismatched-id refusal is regression-locked. +4. G5 is either implemented with a named reachable consumer and an activation test, or + formally removed from the gap matrix with the caller table as evidence. + +## Out of scope + +Writing canonical records; ordinal generation; `thread_history_1.sqlite` reads. + diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/040_phase4_config_alias_and_docs.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/040_phase4_config_alias_and_docs.md new file mode 100644 index 0000000000..530e37c873 --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/040_phase4_config_alias_and_docs.md @@ -0,0 +1,169 @@ +# Phase 4 — config alias truth and its full consumer chain + +Closes G7. One PABCD cycle. Independent of every other phase. + +Audit history: `006_audit_round1.md`, `007_audit_round2.md`. This document is canonical — +all audit corrections are integrated below, not appended. + +## Why: opencodex enforces a rule upstream reverted 4 months ago + +`src/codex/features.ts:226-240` claims codex-rs "REFUSES to boot" on `[agents] max_threads` +with multi_agent_v2 enabled. Git archaeology in the upstream checkout: + +| Commit | Date | Subject | +| --- | --- | --- | +| `d3b044938` | 2026-04-23 | Reject agents.max_threads with multi_agent_v2 (#19129) | +| `1f304dd1f` | 2026-04-26 | **Allow** agents.max_threads to work with multi_agent_v2 (#19733) | +| `03bb3b123` | 2026-07-16 | Unify multi-agent settings under `agents` (#33550) | + +At HEAD `9dd22890f`: + +```bash +$ rg -n 'cannot be set when|multi_agent_v2 is enabled' codex-rs/ --type rust +(no matches) +``` + +`max_threads` is a first-class alias — `codex-rs/config/src/key_aliases.rs:17-21` and +`codex-rs/config/src/config_toml.rs:668` (`#[serde(alias = "max_threads")]`). +`normalize_key_aliases` uses `.entry(canonical).or_insert(value)`: with BOTH keys present +the canonical key wins and the legacy value is silently discarded. + +## Semantics that must survive + +- V1 `[agents] max_threads = N` → N **child** threads (root not counted); default 6. +- V2 `max_concurrent_threads_per_session = N` → N **total** including root → N-1 children; + default 4. +- `[agents] N` used as the V2 fallback resolves to N+1 total. + +Already modeled by `isTranslatableV1ChildLimit` / `v1ChildLimitToV2TotalLimit` +(`src/codex/features.ts:1305-1310`). Preserve it. + +## Change 1 — readers accept both keys + +`getAgentsMaxThreads` (`:242-258`) matches only `max_threads`. Replace with: + +```ts +/** Current [agents] thread limit: canonical max_concurrent_threads_per_session, else the + * max_threads legacy alias (key_aliases.rs:17-21 — canonical wins on conflict). */ +export function getAgentsThreadLimit(configPath?: string): number | null +``` + +Keep `getAgentsMaxThreads` as a deprecated alias for one release; move every consumer. + +## Change 2 — correct the claim + +`hasAgentsMaxThreads` (`:226-240`) keeps its detection but changes its meaning: the key is +an accepted legacy alias, not a boot blocker. Rename to `hasLegacyAgentsMaxThreadsAlias` +(deprecated re-export for one release) and rewrite the doc comment citing `1f304dd1f` and +`key_aliases.rs:17-21`. + +## Change 3 — detect the case that matters, and consume it + +```ts +/** TRUE when [agents] defines BOTH keys with DIFFERENT values. Upstream + * normalize_key_aliases uses or_insert, so the canonical key wins and the legacy value is + * silently dropped — worth warning about, unlike the alias itself. */ +export function hasConflictingAgentsThreadKeys(configPath?: string): boolean +``` + +Reuse the existing TOML readers (`parsedTomlTable`, `tomlTableBody`) at `:236-253` / +`:294-304`; no new dependency. + +**GUI visibility (settled):** a two-key conflict is a config defect regardless of whether +V2 is on — the canonical key wins and the legacy value is discarded either way +(`key_aliases.rs:24-36`). So the conflict is surfaced **independently of `enabled`**. The +DTO key `agentsMaxThreadsConflict` keeps its name for contract stability; its value becomes +`hasConflictingAgentsThreadKeys()` with **no `enabled &&` guard**, and the GUI condition +drops `v2.enabled`. + +## Change 4 — writer prefers the canonical key + +`src/codex/features.ts:1215-1234`: new insertions write +`max_concurrent_threads_per_session`. An existing user-authored `max_threads` line keeps its +spelling and trailing comment (`mergeTrailingComments` discipline). The `:1496` +postconditions at **`:1486` (v2) and `:1496` (v1)** must be updated in the same commit or +they will assert on the old key and fail. + +## Change 5 — the complete consumer chain + +Every consumer of the old predicate or the `max_threads` spelling, verified this session: + +| Consumer | Location | Required change | +| --- | --- | --- | +| logical display | `src/codex/features.ts:1305-1310` `getLogicalMaxThreads` | use `getAgentsThreadLimit` | +| migration discovery | `discoverStoredThreadLimit` (same file) | both spellings | +| **v2 migration postcondition** | `src/codex/features.ts:1486` | calls `hasAgentsMaxThreads(path)`, which sees only the legacy spelling — a residual **canonical** `[agents]` key would escape the check. Must assert that NEITHER `[agents]` spelling remains | +| v1 migration postcondition | `src/codex/features.ts:1496` | uses `getAgentsMaxThreads(path) !== threadLimit`; move to `getAgentsThreadLimit` so a canonical-spelled target validates | +| comment preservation | `src/codex/features.ts:1342` `activeThreadComment` | regex matches `max_threads` only; must also match the canonical key or comments are dropped on migration | +| dotted-key rejection | `src/codex/features.ts:1398` | pattern names `agents.max_threads`; add the canonical dotted form | +| duplicate detection | `src/codex/features.ts:1416` | counts duplicate `max_threads` only; must also count canonical duplicates and cross-key duplicates | +| management GET DTO | `src/server/management/agent-settings-routes.ts:232` | `hasConflictingAgentsThreadKeys()`, no `enabled &&` | +| management PUT DTO | `src/server/management/agent-settings-routes.ts:373` | same fix — repeats the stale predicate | +| CLI import | `src/cli/v2.ts:15` | move to the renamed predicate | +| **CLI warning text** | `src/cli/v2.ts:127` | **ships a false claim today:** "codex refuses to start while multi_agent_v2 is enabled". The most user-visible instance of G7 | +| GUI | `gui/src/pages/Models.tsx:1341` + locale strings | drop `v2.enabled` gating; rewrite the boot-refusal copy | +| SoT doc 1 | `structure/03_catalog-and-subagents.md` | child-vs-total table + alias fact with SHAs | +| SoT doc 2 | `structure/05_gui-and-management-api.md:105` | the `/api/v2` row describes migrating `[agents] max_threads` | + +## Tests (`tests/codex-features-cache.test.ts`, `tests/codex-features-residual.test.ts`, `tests/codex-v2-gate.test.ts`) + +1. `[agents] max_threads = 6` alone → read correctly, NOT a boot blocker. +2. `max_concurrent_threads_per_session = 6` alone → read correctly. +3. Both keys, same value → no conflict. +4. Both keys, different values → conflict reported AND surfaced in `GET /api/v2`. +5. **PUT `/api/v2` reports the conflict identically to GET.** +6. **The conflict is reported with V2 disabled** — activation evidence for Change 3. +7. `ocx v2 status` emits no boot-refusal claim for a lone `max_threads`. +8. New writes emit the canonical key; an existing `max_threads` line is preserved verbatim. +9. **A trailing comment on a canonical-key line survives migration** (`:1342`). +10. **Duplicate canonical keys, and one-of-each duplicates, are both rejected** (`:1416`). +11. Child/total conversion: `[agents] 6` ⇒ v2 total 7; v2 total 4 ⇒ 3 children. +12. Both migration postconditions (`:1486` v2, `:1496` v1) pass against the new writer. +13. **Transition starting from the canonical `[agents] max_concurrent_threads_per_session` + spelling** — the v2 postcondition at `:1486` must catch a residual canonical key, which + it cannot today. Activation evidence for that row. + +## Verification + +```bash +bun install # REQUIRED: this worktree has no node_modules +cd gui && bun install && cd .. # REQUIRED for lint:gui (oxlint) +bun test tests/codex-features-cache.test.ts tests/codex-features-residual.test.ts tests/codex-v2-gate.test.ts +bun x tsc --noEmit +bun run lint:gui # Change 3/5 touch gui/src/pages/Models.tsx +``` + +The `/api/v2` route is covered by `tests/codex-v2-gate.test.ts` (verified). There is no +`tests/api-v2.test.ts`, and `bun test` ignores a missing path **silently** — a phantom +filename leaves the gate observing nothing. + +**Receipts (measured 2026-08-16, dependency-less worktree).** Each row is the exact command +named above: + +| Command | Exit | Observed | +| --- | --- | --- | +| the 3-file `bun test` command | 1 | `0 pass, 3 fail; Ran 3 tests across 3 files` — `Cannot find module 'zod/v4'` | +| `bun x tsc --noEmit` | 1 | `TS2688: Cannot find type definition file for 'bun-types'` | +| `bun run lint:gui` | **127** | `oxlint: command not found` (missing binary, not a lint failure) | + +Environmental — `node_modules/` absent. B runs both installs and re-records. + +Target observation: the features suites import `src/codex/features.ts` directly; +`tests/codex-v2-gate.test.ts` exercises the `/api/v2` route. Both `structure/` prose edits +are read by no gate — **human review**. + +## Accept criteria + +1. No opencodex surface claims codex-rs refuses to boot on `[agents] max_threads` — + including `src/cli/v2.ts:127`, the GUI, and every locale. +2. Both key spellings are read, with canonical winning on conflict. +3. The conflict detector is consumed by GET **and** PUT `/api/v2` and rendered by the GUI + regardless of `enabled`. +4. New writes use the canonical key; user-authored legacy keys are preserved verbatim. +5. Comment preservation, dotted-key rejection, and duplicate detection all handle both keys. +6. Both migration postconditions (`:1486`, `:1496`) pass, and neither `[agents]` spelling can escape them. +7. Both `structure/` docs are updated. + +## Out of scope + +Auto-migrating a user's config; changing default thread counts. diff --git a/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/050_phase5_compact_wire_verification.md b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/050_phase5_compact_wire_verification.md new file mode 100644 index 0000000000..b529031a9e --- /dev/null +++ b/devlog/_plan/260816_codexrs_multiagent_v2_and_history_perf/050_phase5_compact_wire_verification.md @@ -0,0 +1,139 @@ +# Phase 5 — /v1/responses/compact wire-fidelity verification + +Closes G8. Independent of every other phase. + +> **A-phase amendment (round 1).** The reviewer flagged that this doc deferred its +> central decision ("two possible outcomes") and labelled Guardian handling +> "investigate, likely no-op" — asking B to design the phase. Both decisions are now +> settled in the doc, with the investigation moved to a P-phase precondition whose +> answer is written down before B starts. Upstream anchors corrected: +> `compact.rs:35` is `fn path()`, not `CompactRequest::new`. + +## Why + +`002_upstream_history_perf_evidence.md` established the history rework is local except +for a narrow proxy-visible surface. `b9ba969f3` ("Enable remote compaction for Amazon +Bedrock") widens the set of configurations issuing `POST /v1/responses/compact`, so more +traffic reaches an endpoint opencodex already implements. + +Upstream request contract — `CompactionInput`, `codex-rs/codex-api/src/common.rs:26`: +`model`, `input`, `instructions`, `tools`, `parallel_tool_calls`, `reasoning`, +`service_tier`, `prompt_cache_key`, `text`. + +Endpoint anchors (corrected): `fn path() -> "responses/compact"` at +`codex-rs/codex-api/src/endpoint/compact.rs:35`; the sending function is +`compact_input` at `compact.rs:71`, which serializes `&CompactionInput` into the body. + +`4bd5b9fd0` changes which items stay adjacent inside the existing `input` array — no new +field, but observable body content. + +## Current state (verified this session) + +| Location | Fact | +| --- | --- | +| `src/server/index.ts:1074` | `/v1/responses/compact` POST branch (the `:1071` line is its comment) | +| `src/server/responses/compact.ts:267` | `handleResponsesCompact` — native forward or routed conversion | +| `src/server/responses/compact.ts:401` | explicitly DROPS top-level `reasoning` before forwarding | +| `src/server/responses/compact.ts:656-716` | routed models converted to v2 summarization | +| `src/responses/compaction.ts:56-123` | v1 retention: ~20k tokens ≈ 80k chars, newest-first | +| `tests/responses-compaction-routing.test.ts:139,437,557,639` | existing routing/auth/fallback coverage | + +## P-phase precondition (settle BEFORE building) + +One question must be answered and its answer written into this doc as an amendment: +**is the `compact.ts:401` reasoning drop deliberate?** + +```bash +cd /Users/jun/.codex/worktrees/e80c/opencodex +sed -n '390,410p' src/server/responses/compact.ts # read the surrounding rationale +git log -S'reasoning' --oneline -- src/server/responses/compact.ts | head +rg -n 'reasoning' devlog/_fin --glob '*compact*' | head # prior devlog rationale (-n, never -r: -r means replace) +``` + +The decision rule is fixed in advance, so B is not designing: + +- **Evidence of a deliberate reason** (a comment, commit message, or devlog explaining a + provider rejecting `reasoning` on compact) → **keep the drop**, add a citing comment at + `:401`, and assert the drop in the test so it is intentional rather than incidental. +- **No such evidence** → treat it as incidental: forward `reasoning` on the native path + where `CompactionInput` carries it, keep dropping it for targets that reject it, and + cover both branches. + +Record the finding and which branch was taken. Do not change behavior before it is +recorded — compaction runs on the user's full context and a wrong change is expensive. + +## Change 1 — field-fidelity regression (ADD to `tests/responses-compaction-routing.test.ts`) + +Send a compact request carrying every `CompactionInput` field with a distinguishable +value; assert on the captured upstream body: + +```ts +const body = { + model: "gpt-5.6-sol", + input: [/* ... incl. an image item followed by its resize notice ... */], + instructions: "SENTINEL_INSTRUCTIONS", + tools: [{ type: "function", name: "sentinel_tool", parameters: {} }], + parallel_tool_calls: true, + reasoning: { effort: "medium" }, + service_tier: "priority", + prompt_cache_key: "SENTINEL_CACHE_KEY", + text: { format: { type: "text" } }, +}; +``` + +Each field is asserted as forwarded verbatim OR deliberately transformed with the +transformation asserted explicitly. No field may vanish without a named reason. The +resulting table is this phase's deliverable and gets appended to this doc. + +## Change 2 — resize-notice adjacency (ADD test) + +Per `4bd5b9fd0`, an image item and its resize notice must stay adjacent through +opencodex's compact handling. Assert the relative order of those two items end-to-end. + +## Change 3 — Guardian opaque-blob passthrough (settled: regression only) + +`c2bcb9a26` lets a Guardian review session start from the parent's encrypted compaction +response item, which then traverses opencodex as ordinary `input[]` content. The relevant +opencodex behavior already exists: `src/adapters/openai-responses.ts:141` converts +opencodex `ocx1:` compaction items to plain messages while leaving genuine OpenAI opaque +blobs alone. + +Decision (not an investigation): **add a regression asserting a genuine OpenAI opaque +compaction blob survives the passthrough sanitizer byte-identical**, in +`tests/openai-responses-passthrough.test.ts`. Expected production change: none. If the +regression fails, that is a real defect and this phase fixes it. + +## Verification + +```bash +bun install # REQUIRED FIRST: this worktree has no node_modules +bun test tests/responses-compaction-routing.test.ts tests/responses-compaction.test.ts tests/openai-responses-passthrough.test.ts +bun x tsc --noEmit +``` + +**Verifier receipts (recorded 2026-08-16).** In this worktree the test command exits `1` +(`0 pass, 3 fail`, `Cannot find module 'zod/v4'`) and `bun x tsc --noEmit` exits `1` +(`TS2688: bun-types`). Environmental — `node_modules/` absent. B runs `bun install` and +re-records real exit codes. `package.json:41` defines `"test": "bun scripts/test.ts"`. + +Target observation: these suites exercise `src/server/responses/compact.ts` and +`src/adapters/openai-responses.ts` directly. + +## Accept criteria + +1. Every `CompactionInput` field is forwarded verbatim or has a documented, asserted + transformation — recorded as a table in this doc. +2. Image/resize-notice adjacency is preserved. +3. The `reasoning` drop is recorded as deliberate-and-cited or fixed, per the + precondition's decision rule. +4. A genuine OpenAI opaque compaction blob survives passthrough byte-identical. + +## Out of scope + +Implementing native compaction ourselves; changing the v1 retention budget +(`compaction.ts:56-123`); Bedrock-specific provider work. + +## Terminal note + +If every assertion passes with no production change, the honest outcome is **NOOP** — +recorded with the evidence table that proves it, not skipped.