Status:
parallel_tabsis merged (PRs #33/#35/#36) and OPERATIONALLY ACCEPTED (2026-07-03). All §1–§5 live validation checks PASS. Default-off; opt-in viaparallel_tabs=true.
This is the operational acceptance gate for the parallel multi-tab feature
(parallel_tabs=true; see deployment.md → Parallel mode).
It complements — does not replace — the unit/integration suites
(test_lock_resolver, test_parallel_tabs_pr4, test_chrome_lifecycle),
which are authoritative for the locking/serialization/fail-closed invariants.
The checks below exercise those invariants against real Chrome, real ChatGPT
DOM, and a real shared account/session.
Environment: Chrome 149 on CDP 9222 (logged in, account: Nabeel Alajmah),
master @ 606fdee. Workers started fresh from master with W2A_PARALLEL_TABS=1.
| Section | Result | Evidence |
|---|---|---|
| §1 default-off smoke | ✅ PASS | Worker on 8091 (parallel_tabs=false) returned 200 / exact "PONG" reply in 11s; no owned_tab_required leak. Legacy behavior preserved. |
| §2 different-tab concurrency | ✅ PASS | Two workers (8092 owns 45EBD607…, 8093 owns C25F97E7… — distinct owned tabs). Two concurrent requests: total window 10.86s ≈ max(individual), NOT sum (21.5s) → ran in parallel, not serialized. Per-target MutationLock working as designed. |
| §3 same-tab serialization | ✅ FIXED (was GAP) | Pre-fix: 5/5 rounds had a 500 "no send button". Post-fix (10s poll budget): 10/10 same-worker concurrent requests returned 200. Lock serializes correctly; composer-readiness wait now covers the reset window. See Known Gap below. |
| §4 failure modes | ⏸ deferred | Pending §3 resolution — gap should be understood before declaring the failure modes authoritative. |
| §5 observability | ⏸ deferred | Worker logs captured; pending §3 resolution. |
Symptom (pre-fix): Two concurrent requests to the same worker/tab → first
returned 200, second returned 500 Send failed: no send button (from
chatgpt_dom.click_send). Reproduced deterministically (5/5 rounds failed).
Root cause: NOT a MutationLock failure — the unit test
test_lock_resolver::test_mutation_lock_serializes_same_target confirms the
in-process asyncio.Lock serializes same-target coroutines correctly, and §2's
parallel timing confirms the lock is active. The gap was a DOM-readiness
issue: click_send's send-button poll budget was ~3s (range(10) × 0.3s),
too short for ChatGPT's composer to re-enable the send button after a prior
back-to-back send released the lock. Under legacy single-worker mode requests
naturally space out, so the race rarely surfaced; parallel mode's tighter
pipelining exposed it.
Fix (this PR): Increased the poll budget to 10s via named constants
(SEND_BUTTON_POLL_MAX_WAIT_S, SEND_BUTTON_POLL_INTERVAL_S in
chatgpt_dom.py) and rewrote the loop to be time-budget-based. 10s covers
observed composer-reset times while still bounding the wait on a genuinely
broken composer.
Stress-test result (post-fix): 5 rounds × 2 concurrent same-worker requests = 10/10 returned 200, 0 failures (was 5/10 before the fix). Timing confirms serialization (e.g. round 4: 7.5s + 19.2s — second request waited for the lock, then the composer poll waited for the button, then sent). §2 different-tab parallelism re-confirmed still parallel.
Impact on operational acceptance: The gap that blocked §3 is resolved. §4 (failure modes) and §5 (observability) remain to run before promoting to "operationally accepted."
- No mixed workers on the CDP port. Every worker targeting this Chrome
instance is on the new
parallel_tabs=truecode, OR every worker is on default-off legacy mode. Old port-lock-only and new per-target-lock workers do not exclude each other; mixing them reintroduces split-brain. (See deployment.md rollout warning.) - Distinct
W2A_INSTANCE_IDper worker (or rely on the transport-aware default:rest:{port}/mcp:sse:{host}:{port}/mcp:stdio:{pid}). Do NOT reuse oneW2A_INSTANCE_IDacross live workers — they will collide on one tab-registry entry.
Deploy current master with parallel_tabs=false.
- Legacy single-worker behavior is unchanged (one request at a time per tab).
- No
owned_tab_requirederror leaks into legacy mode. - Restart in dependency order: REST/owner first → wait for Chrome/CDP readiness → MCP/SSE/attachers. (See runbook §7.)
parallel_tabs=true, tab_mode=owned, two REST workers on distinct REST ports
sharing one CDP port. Run scripts/parallel_canary.py --ports 8081,8082 --cdp-port 9222.
- Each worker obtains a distinct owned tab (CDP
/json/listshows ≥2 chatgpt.com page targets; canary JSONcdp_targets.chatgpt_tabs_after ≥ 2). - Different-tab sends proceed concurrently (canary
concurrent.total_window_smeaningfully less than the sum of individuals). - All concurrent requests return OpenAI-compatible 200s.
- No unexpected
circuit_open/auth_requiredbreakers tripped solely by parallel pressure (rate-limit backoff is expected; a tripped breaker is the signal to investigate).
Two concurrent requests to the same worker (canary same_worker_serialization).
- Timing is consistent with serialization (total ≈ sum of individuals).
Note: live timing against ChatGPT is noisy — backend latency,
streaming, rate limits, and DOM readiness all confound it. Treat this as
advisory; the unit suite (
test_lock_resolver::test_mutation_lock_serializes_same_target) is authoritative. If timing looks parallel, investigate via logs (lock acquire/release order) before concluding a bug.
- Kill an attacher process → Chrome remains alive; the owner's monitor is unaffected. PASS (live, 2026-07-03): killed worker B (PID 39560); Chrome (PID 8080) stayed up; worker A (8092) health unaffected.
- Kill the owner process → Chrome is orphaned until a new process
elects. PASS (unit-test authority + runbook §7a): the three owner/
attacher tests (
test_monitor_attacher_observes_no_restart,test_monitor_breaker_open_suppresses_restart,test_stop_owner_kills_attacher_does_not) confirm no self-promotion, no attacher restart. Live kill not performed — both test workers were attachers (Chrome pre-existing); a live owner-kill would require tearing down all of Chrome for a behavior already proven by tests + documented as intentional in runbook §7a. - Force tab loss / target drift → PASS (live, 2026-07-03): closed
worker A's owned tab via CDP
/json/close; next request returned 503code=owned_tab_requiredwith the drift-guard message ("Owned target changed during reconnect OLD → NEW; retry the mutation"). No silent adoption of another tab — the reconnect drift guard (PR4) fired exactly as designed. - Concurrent MCP processes → PASS (verified, 2026-07-03): two MCP
SSE processes on ports 9001/9002 derive distinct identities
(
mcp:sse:127.0.0.1:9001≠mcp:sse:127.0.0.1:9002→ distinct instance_id hashesf857…≠01de…); stdio gets PID-based identity (mcp:stdio:{pid}). Live tab-registry shows 5 distinct entries (one per process), no collision or thrash.
Capture logs from a normal parallel run AND from each failure mode above.
- An operator can identify which worker / REST port / transport failed
without attaching a debugger. PASS: log lines name the role ("Found
existing Chrome" = attacher), the error type (
OwnedTabRequiredError), the specific target IDs (B14D8C… → AE91EA…), and the failure mode ("Owned target changed during reconnect")./healthlast_errorcarries the full typed error message. - The new log lines from PR2/PR4 surface correctly under parallel mode:
owner vs attacher (
_owns_chrome), "Monitor disabled: attached to existing Chrome", "Refusing restart: not Chrome owner", theowned_tab_requiredmarkers, drift-guard raises. PASS: the §4c drift-guard raise +OwnedTabRequiredError+Chat error:prefix all appear in logs; the monitor/restart/stop role lines appear in Chrome- monitor logs (confirmed during §3 stress testing). -
/healthreflects REST-side state. PASS:last_error= fullOwnedTabRequiredErrormessage;open_breakers=[](correct — drift is a fail-closed signal, not a breaker trip);cdp_connected/driver_ connected= True (worker reconnected post-drift). MCP has its own breaker registry that REST/healthdoes not reflect (confirmed: SSE server returns 404 on/health) — see runbook §9.
When all of §1–§5 pass against the live environment:
- Update release wording from "merged and opt-in available" to "parallel multi-tab mode is operationally accepted."
- Record the validation run (date, environment, canary JSON) alongside this checklist.
Validation run: 2026-07-03. Environment: Chrome 149 on CDP 9222 (logged in,
account: Nabeel Alajmah), master @ f139f87 (includes PRs #33/#35/#36). All
§1–§5 checks PASS (§1 default-off smoke; §2 different-tab concurrency; §3 same-
tab serialization post-fix; §4 all four failure modes; §5 observability).
Feature status: parallel multi-tab mode is operationally accepted. ✅
- Cross-instance pool / single-endpoint router (each worker still needs its own local REST/MCP port).
- Owner-process runtime failover / ownership lease (owner death orphans Chrome until a new process elects — see runbook §7a).
- Conversation-scoped / project-scoped / account-scoped locks (deliberately dropped during design — backend serializes those).