Repository navigation
Run Full journeys in parallel and trim their runtime - #1345
Merged
Merged
Conversation
Run all four agent x OS Full jobs at once instead of serializing Linux and Windows per agent. Cases share no workspace state, only inference capacity. Exclude smoke cases from Full; the smoke job already runs them on the same OS matrix. A contract test checks smoke + Full still cover every blocking live case exactly once. Launch the Codex smart-router toggle journey's parent with low reasoning effort; parent turns dominated its runtime and routing never reads it. Co-authored-by: Isaac <no-reply@databricks.com>
rohita5l
force-pushed
the
ci-parallel-full-journeys
branch
from
October 28, 2026 18:39
36acd1a to
c101d55
Compare
Smoke had the same per-agent concurrency group as Full, so each agent's Linux job waited for its Windows job. Drop it for the same reason: cases share no workspace state, only inference capacity. Co-authored-by: Isaac <no-reply@databricks.com>
lilly-luo
approved these changes
Oct 28, 2026
With `-c model_reasoning_effort="low"` forwarded to the parent, Codex still spawned the child but the Smart Router subagent banner never appeared, so all 18 Codex TUI journeys timed out in Managed config and Full. main passes them without the flag. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Full journeys currently take ~20 min of the ~23 min integration run (e.g. run 40687468164): Codex · Linux (~11 min) then Codex · Windows (~9 min), because a per-agent
concurrency:group serializes Linux and Windows. This PR:live and <agent> and not soft_fail and not smoke. The smoke job already runs those cases on the same Linux/Windows matrix, so every blocking case still runs once per OS (Claude 3 + 34 = 37, Codex 3 + 46 = 49).Expected: Full stage ~20 min → ~11 min (bounded by Codex · Linux); integration ~23 min → ~13 min. To be confirmed by this PR's CI.
Tests
smokenor thefulljob has aconcurrency:group.tests/test_integration_contract.py: 92 passed locally.tests/integration/README.md(parallel lanes, new marker) and thetests/README.mdcoverage matrix.Dropped
An earlier revision launched the Codex Smart Router TUI journeys' parent with
-c model_reasoning_effort="low". In CI the parent still spawned the child, but the Smart Router subagent banner never appeared, so all 18 Codex TUI journeys timed out. That change is reverted; those tests now matchmain.Risk
Up to 2 jobs per agent now hit the shared e2e workspace at once, and tests have no retries, so a 429 would fail the run. Worth watching this PR's CI for 429/503s across a couple of runs before merging.
This pull request and its description were written by Isaac.