[Bug] Routing profile `optimize.latency` has no latency score term and only becomes the residual `priorityWeight`

[trungtaottn]

Client or integration

Codex CLI (ocx) — routing policy profiles, and the config set / config import commands.

Area

CLI

Summary

Two independent config-shells prompts reported separately, now combined. Both share the theme "an option I set is not honored / silently loses my data," but they are two distinct defects:

1. Routing optimize.latency does not optimize latency. Setting optimize.latency on a routing profile does not make the router prefer faster providers. On the live policy evaluator (evaluatePolicyProfile), optimize.latency produces no measured-latency score term; with optimize: { latency: 1, health: 0, cost: 0, quota: 0 } selection is a pure argmax over declaration order — reversing the provider list flips the winner to the 40 s-latency provider. Measured latency only ever rides inside the health score (scaled by optimize.health), never optimize.latency. The reference docs call latency a "residual priority split," so this is a design/docs gap rather than a crash: the option name misleads operators, and the residual 1 - health - quota - cost formula is not honored when quota/cost evidence is absent.

2. ocx config import / set silently deletes hand edits. A hand-edit committed to ~/.opencodex/config.json is destroyed, deterministically, by ocx config import file.json --yes when the import file lacks the edited field — import reads only the file passed (src/cli/config-command.ts:178, loadInput at :67), never the current config, and writes the whole thing. ocx config set/unset additionally have a lost-update race (snapshot read at config-command.ts:133, whole-object write at :145); reproduced 34/51 stale reverts under a concurrent writer.

I'm filing both in one issue as the parent asked. I'm not asserting a confirmed root cause from logic alone; the Evidence section lists what was actually run.

Reproduction

Routing latency (run the real evaluator)

Config: profile "lat" with candidates [a/m1, b/m1], optimize { latency:1, health:0, cost:0, quota:0 }, evidence a/m1 recentLatencyMs 800 (fast), b/m1 40_000 (slow).

Order [fast, slow]  -> wins a (fast):  a total=1.000 comps={"configuredPriority":1}  b total=0.500
Order [slow, fast]  -> wins b (slow):  b total=1.000 comps={"configuredPriority":1}  a total=0.500

Reversing the candidate list flips the winner to the 40 s provider; the only score component is configuredPriority. No latency term ever appears.

Config import clobber (deterministic, no concurrency)

ocx config set emptyCompletionRetry true        # writes ~/.opencodex/config.json
# hand-edit: add  "streamMode":"eager-relay", "customFlag":"x" ; set emptyCompletionRetry=false
ocx config import ./fresh.json --yes            # fresh.json omits those two fields
ocx config get streamMode    # -> error: config path not found: streamMode   (edit destroyed)
ocx config get customFlag    # -> error: config path not found: customFlag
ocx config get emptyCompletionRetry             # -> true   (hand-toggle reverted)

Config set race (with concurrent writer)

ocx config set port 10100 in a loop while a separate process rewrites an unrelated field; live result: 34 of 51 rounds the concurrent counter reverted to a stale lower value.

Version

2.22.0 (main d9de895, @bitkyc08/opencodex)

Operating system

Linux

Provider and model

OpenAI Codex Responses API (routing case); config case is provider-independent.

Logs or error output

No crash/error is raised — both defects are silent (wrong selection; field disappears). The routing case is visible in the evaluator output above; the config case has no explicit error.

Redacted configuration

// routing profile
{ "routingProfiles": { "profiles": [{ "name": "lat", "optimize": { "latency": 1 } }] } }
// config edit before import (the part that is destroyed)
{ "streamMode": "eager-relay", "customFlag": "x", "emptyCompletionRetry": true }

Evidence

• Routing: live bun test tests/latency-scope-live.test.ts against real evaluatePolicyProfile → 5 pass / 0 fail (fast-vs-slow order flip, no latency component ever emitted). grep -c "latency" src/routing/evaluator.ts → 0; optimize.latency has no consumer outside profile.ts (src/routing/evaluator.ts:398-418). Measured p50 latency folds only into healthScore (src/routing/health.ts:386-389), scaled by optimize.health: (evaluator.ts:408), not optimize.latency.
• Config: live CLI in throwaway copy (Bun 1.3.14, source v2.22.0 d9de895): hand-edit → config import fresh.json --yes (schema-valid, omitting the fields) → exit 0, edits gone; config set port 10100 under concurrent writer → 34/51 counter reverts. Source: src/cli/config-command.ts:133,145,178; src/config.ts:2766 (saveConfig does no general disk rebase), :3185 (saveConfigPreservingClaudeCode — the rebasing writer, used by server routes, not these CLI commands); src/server/index.ts:562 (baseline armed at server start).
• import is not a blind overwrite: it validates the file against the schema and aborts on invalid input (a file missing providers wrote nothing). Data loss occurs with a schema-valid-but-partial file, which is the common case.

Expected behavior

• optimize.latency (if intended as a latency knob) should consult a real latency signal and prefer the lower-p50 provider regardless of order; otherwise the docs/edit schema should make clear it is a residual priority split and the residual should match the documented 1 - health - quota - cost even when evidence is missing.
• ocx config set/unset should merge onto the current on-disk config before writing (like the server-side rebasing writer), and ocx config import should diff its candidate against the current on-disk config and warn about keys it is about to remove.

Checks

• I searched existing issues and documentation (incl. routing reference docs, [Bug] ocx sync overwrites hand-edited config.json from stale server memory (GLM-5.3 tune lost); 2.21.0 save path fixed it but sync path question remains #1802, the earlier [Bug] Routing profile optimize.latency has no latency score term and only becomes the residual priorityWeight #1834/[Bug] ocx config set / ocx config import bypass the protected 3-way reconcile and can clobber concurrent disk / hand edits #1835/[Bug] Routing profile optimize.latency has no latency score term and only becomes the residual priorityWeight #1837/[Bug] ocx config set / ocx config import bypass the protected 3-way reconcile and can clobber concurrent disk / hand edits #1838 records).
• I removed secrets, tokens, account details, request credentials, and personal data.
• Reproduction run live (routing evaluator + CLI config commands) on source d9de895 (v2.22.0).

[coderabbitai]

⚠️ Possible Duplicate Issue(s)

• [Bug] Routing optimize.latency never scores latency + ocx config import/set silently delete hand edits #1839

🔗 Related PRs

#770 - fix: research-batch reliability and security-boundary issues [closed]
#886 - fix(server,config): extension CORS origin identity + symlink-safe atomic writes [open]
#892 - fix: bound the six remaining retained-state leaks (#820 wave 2) [merged]
#1003 - feat(routing): record durable route decision traces [merged]
#1012 - feat(routing): execute capability-aware policy profiles [merged]
#1018 - feat(gui): add routing profiles and why-this-route views [merged]
#1288 - fix(routing): keep unbound account quota unknown (#1195) [merged]
#1606 - docs(devlog): routed tool-discovery profile plan with landing verification [open]
#1705 - feat(lab): CL-10 public evidence trust core [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[github-actions]

Issue reopened

The report now contains the information required by the automated check. Thanks for updating it.

[lidge-jun]

Fixed on dev by #1848 (merged as 73440d4bb).

optimize.latency is now actually spent. It defaults to 0.55 — the largest default weight — and the normalizer included it in the weight sum, but the evaluator read only health, quota and cost. Whatever was allocated to latency fell into priorityWeight = 1 - spentHealth - spentQuota - spentCost and was multiplied by configuredPriorityScore(index, total), so a latency-optimized profile silently became a declaration-order profile. components.latency was declared in the trace type but never populated.

Real latency was already measured, but only inside healthScore() scaled by optimize.health. That computation is now extracted as latencyScoreFromEvidence() and shared, so the health composite and the standalone term cannot drift apart.

An unmeasured candidate takes the neutral midpoint rather than 0. Scoring it 0 would make selection depend on which candidate happened to be exercised first — the same order-dependence under a different name.

Regressions: a later-declared faster candidate now wins under latency: 1 (the exact inversion reported); an unmeasured candidate is not ranked below a measured-but-slow one; and latency: 0 leaves declaration-order behavior and the trace unchanged, so existing profiles see no behavior change. Two existing tests asserted the exact composite total: 0.685 and were updated to assert components instead.

Closing as completed. The change is on dev and ships with the next release.