Skip to content

refactor: every variant ends on the one abort path - #1896

Merged
ZhengGong-amd merged 13 commits into
mainfrom
fix/chenyi/grid-one-abort-path
Oct 21, 2026
Merged

ZhengGong-amd merged 13 commits into
mainfrom
fix/chenyi/grid-one-abort-path

Conversation

@chennyiiis

Copy link
Copy Markdown
Contributor
  • Description: what and why

Follow-up to review feedback on #1882 (finding 1, mechanism-added-not-removed). The reviewer is right and the title of #1882 was not: unifying eleven pre-benchmark aborts left _record_round_stop and two inline rc-gated tails owning the same concern, so "record the result and decide whether the grid stops" had three implementations instead of one.

All sixteen endings now raise _VariantAborted, and the halt is data on it:

_Halt meaning who raises it
CONTINUE the grid always walks past this the two post-benchmark tails when rc == 0
ON_FAILURE keep_going_on_failure decides the eleven pre-benchmark aborts, a stopped round, the tails when rc != 0
STOP the batch is over; trailing carries the rows for the variants it will not reach a round stop that ends the batch, and the AgentX preflight abort

The rc asymmetry that TestKeepGoingAsymmetry documents - an rc == 0 failure always continues, an rc != 0 failure does not - was buried in two if rc != 0 and not keep_going_on_failure tails. It is now the difference between two enum members.

_record_round_stop becomes _abort_on_round_stop and only builds the result: it no longer appends to results, no longer extends the grid with not-run rows, and no longer answers a question its caller then re-asked.

Why not the requested _run_variant extraction. I measured both ways it can be written, and both have a hard cost:

  • As a nested async def, which is the only way it avoids plumbing: the gate's function_line_findings de-folds nested definitions - I probed it, a 103-line nested function reports as its own outer.inner 103 entry - so the extraction creates a new ~900-line function-lines finding. CI refuses any baseline entry added relative to the base, so that needs a second baseline-raise. And ruff's mccabe folds nested definitions into the enclosing function, so run_grid's cyclomatic complexity would not drop at all.
  • As a module-level def: roughly thirty parameters, which is the signature rule the extraction was meant to serve.

So the extraction trades one oversized unit for another, needs a waiver, and moves no complexity number. This PR instead makes "one abort path" true at a cost of 111 changed lines and no re-indent. If you still want the extraction after the above, I will open it separately - but I would rather it be the split of run_grid into named phases than a single function holding the same body.

Finding 2 (exception-control-flow) is not addressed here and I have not argued it away: the three try: ... except Exception blocks in the loop body are still there, and an abort added inside one of their bodies in future would be caught and re-recorded under the wrong class. Today every raise sits in a handler or at loop level - I re-ran the AST check and all sixteen are safe - but that is a property of the current code, not a guarantee. Closing it properly means removing the exception, which is the extraction above.

  • Linked issue(s): close/fix refs

Review feedback on #1882.

  • Tests: added/updated? commands run?

TestVariantAbort is kept, with the reviewer's two tidies: the setenv/base-yaml setup moves into _run, and the c0 marker assertion that TestKeepGoingAsymmetry already pins is dropped, leaving the assertion on the aborted variant's own marker.

Run locally: pytest over test_grid_runner.py, test_grid_runner_behavior_lock.py, test_grid_runner_kill_and_branches_unit.py, test_conc_sweep.py - 322 passed; plus test_baseline_warmup_double_run.py, test_agentx_preflight.py and src/hyperloom/orchestrator/actions/executors/tests - 910 passed, the 2 failures being a missing optional TraceLens import in the local environment that reproduces on main.

  • Size/complexity triggers crossed: split, or why the shape is right:
unit cyclomatic cognitive function-lines
run_grid 61 to 55 196 to 177 1040 to 1026

_grid_runner.py goes from 2247 lines to 2246. No new unit above the ceiling: _Halt and the extended _VariantAborted carry no branches. run_grid is still far over and this PR does not claim otherwise - what it claims is that the sixteen endings now share one path.

  • If this simplifies or refactors: which mechanism was removed, which contract was preserved, and the tests that show it:

Removed: two of the three implementations of "record the result and decide whether the grid stops", the boolean _record_round_stop returned for its callers to re-test, and the two inline rc tails. Preserved: the same rows in the same order (the progress note reads results[idx], so moving the trailing rows after the report changes nothing), the same abort markers, and the same decision at every ending - pinned by TestKeepGoingAsymmetry, TestVariantAbort and the kill/branches suite.

  • Observable effect:

None - refactor with nothing observable.

  • Breaking changes: no

  • PR addresses single concern: yes

  • Root cause is upstream: no

Sixteen places end a variant, and three different mechanisms decided what that
meant for the grid: the abort handler, `_record_round_stop`, and two inline
tails after a benchmark that returned nothing usable. Each read
`keep_going_on_failure` or returned a flag of its own, and two of them filled
the rest of the grid with not-run rows.

All sixteen now raise `_VariantAborted`, and the halt is data on it: CONTINUE
for an outcome the grid always walks past, ON_FAILURE for one the policy
decides, STOP for one that ends the batch and carries the rows for the
variants it will not reach. The rc asymmetry that was buried in two
`if rc != 0 and not keep_going_on_failure` tails is now the difference between
two enum members.

`_record_round_stop` becomes `_abort_on_round_stop` and only builds the
result: it no longer appends, no longer extends the grid, and no longer
answers a question its caller then re-asked.

Cyclomatic complexity of run_grid drops from 61 to 55, cognitive from 196 to
177, and its length from 1040 lines to 1026.

Refactor with nothing observable: the same rows in the same order, the same
abort markers, and the same decision at every one of the sixteen endings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis
chennyiiis requested a review from a team as a code owner October 15, 2026 02:28
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Code metrics gate: PASSED

Judged: the 3 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension Limit New Worse Baseline out of date Baseline entries vs base
Cyclomatic complexity fails > 20 0 0 0 55 +0
Cognitive complexity fails > 30 0 0 0 203 +0
Function length (lines) fails > 80 0 0 0 346 +0
Nested block depth fails > 5 0 0 0 20 +0
Parameter count fails > 5 0 0 0 94 -1
Module length (lines) warns > 800, fails > 1200 0 0 0 49 +0
Duplicated lines per file fails > 0 0 0 0 96 +0
Dead code (vulture) fails > 0 0 0 0 2 +0
Check Findings Effect
Comments 0
English only 0
Production code imports test code 0
Repeated literals 0
A1 Private name of another package 0
A8 Flag argument 0
A9 Tuple return 0
A10 Raw vocabulary comparison 0
A11 Hidden global write 0
A12 Pure forwarder 0
A12 Single-implementation abstraction 0

Already over at the base (backlog, not this change's): 2

Unit Dimension Base Now
src/hyperloom/orchestrator/actions/executors/_grid_runner.py:768 run_grid Cognitive complexity 196 180
src/hyperloom/orchestrator/actions/executors/_grid_runner.py:768 run_grid Cyclomatic complexity 61 56
108 findings in files this change does not touch

Outside this change (informational): 108

Unit Dimension Baseline Now
src/hyperloom/orchestrator/state/shared_state.py resolve_graded_comparison Cognitive complexity absent 34
src/hyperloom_kb/collect/mapping.py _experience Cognitive complexity absent 31
src/hyperloom_kb/schema.py FieldDeclaration.__post_init__ Cognitive complexity absent 32
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Cognitive complexity absent 36
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Cognitive complexity absent 40
src/kernelforge/cli/forge_loop_command.py forge_loop Cognitive complexity absent 31
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Cognitive complexity absent 44
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Cognitive complexity absent 38
src/hyperloom/inference_optimizer/cli/__init__.py _reexport_session_environment Function length (lines) absent 111
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_agent_lineup Function length (lines) absent 89
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_run_topology Function length (lines) absent 92
src/hyperloom/inference_optimizer/cli/__init__.py _resume_session Function length (lines) absent 170
src/hyperloom/inference_optimizer/cli/__init__.py _seed_fresh_session Function length (lines) absent 164
src/hyperloom/inference_optimizer/multi_node/cli.py _restart_multinode Function length (lines) absent 107
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _build_parser Function length (lines) absent 84
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _spawn_rank_actors Function length (lines) absent 84
src/hyperloom_kb/sync.py GlobalSync.pull Function length (lines) absent 84
src/kernelforge/cli/forge_loop_command.py _RunReporter.build_result Function length (lines) absent 92
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Function length (lines) absent 136
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Function length (lines) absent 149
src/kernelforge/loop/runner.py IterationLoop._close_iteration Function length (lines) absent 117
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Function length (lines) absent 185
src/kernelforge/loop/runner.py IterationLoop._implementer_history Function length (lines) absent 112
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Function length (lines) absent 102
src/kernelforge/loop/runner.py IterationLoop._source_attempt Function length (lines) absent 175

...and 83 more; the job log lists every row.

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, ruff 0.16.2, vulture 2.16.

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1896 -- refactor: every variant ends on the one abort path

What it does: In run_grid, the decision "record this variant's result and decide whether the grid stops" had three implementations: the _VariantAborted handler, _record_round_stop's boolean that three call sites re-tested, and two inline rc != 0 and not keep_going_on_failure tails. This PR moves every ending onto the one _VariantAborted handler. The stop decision now travels with the exception as a _Halt enum (CONTINUE / ON_FAILURE / STOP), and the not-run rows for a batch-ending stop travel as trailing. The halt mapping is the same as on base at all five rewritten endings, and nothing an operator can see changes.

Blocking issues: 1

  1. [V5] The PR conflicts with main in scripts/code_metrics_baseline.txt [verified]
    Problem: meta.txt reports mergeable: CONFLICTING. git merge-tree --write-tree origin/main 9fc336e shows the only conflicted file is scripts/code_metrics_baseline.txt; _grid_runner.py and the test file merge cleanly.
    Impact: The PR cannot merge until the baseline file is reconciled, and the baseline numbers this PR lowers were measured against a tree main has since moved past.
    Action: Rebase onto main and regenerate the run_grid and module-length entries in scripts/code_metrics_baseline.txt from the rebased tree (python scripts/code_metrics.py --update-baseline).

SKIPPED: V4 -- "# PARTIAL: #1869 changes 336 files, only its first 100 were compared", and the same for #1675, #1276, #1154 and #1032. Overlap with those PRs was not checked.
SKIPPED: V2 -- pytest and ruff are not installed on the review host. Behaviour checks rely on the CI test shards and code-metrics, which are green at head 9fc336e.

Checked: _grid_runner.py (_Halt, _VariantAborted, _abort_on_round_stop, the three round-stop sites, the no-ws and invalid-report tails, the except handler, _variant_progress_note); an AST check that no abort raise sits inside a broad try body; test_grid_runner_behavior_lock.py (TestKeepGoingAsymmetry, TestVariantAbort); test_grid_runner.py (cancelled warmup tests); test_grid_runner_kill_and_branches_unit.py (AgentX preflight); main-side drift via git diff base..origin/main | Ran: git show/grep/diff, a Python AST walk; pytest SKIPPED | Base: 190604f | Head: 9fc336e

chennyiiis and others added 2 commits October 19, 2026 11:14
…bort-path

# Conflicts:
#	scripts/code_metrics_baseline.txt
main added a parameter-count limit after this branch was written;
_abort_on_round_stop took nine. The seven describing the variant's run are one
thing -- which variant, in which slot, what its process returned, when it
started, and the servers it left behind -- so they travel as _RoundAttempt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis chennyiiis added the baseline-raise Waives new, worse and baseline-growth code-metrics findings; reason required in the PR description label Oct 19, 2026
@chennyiiis

Copy link
Copy Markdown
Contributor Author

Rebased on main and fixed what moved underneath it.

main added a parameter-count limit (fails > 5) after this branch was written, and _abort_on_round_stop took nine arguments. The seven describing the variant's run are one thing — which variant, in which slot, what its process returned, when it started, and the servers it left behind — so they now travel as _RoundAttempt. All three call sites build the record; none passes the nine by name.

baseline-raise covers the one remaining entry: _grid_runner.py module length 2089 → 2102. Those 13 lines are the record's definition and docstring — the sentence about server_survivors moved there from the function that used to take it as an argument.

Verification: pytest across the 19 test files that reference run_grid or the grid runner — 1818 passed. The one failure, test_trace_analyze_handler_dry_run_returns_structured_result, is a missing TraceLens install locally and fails the same way on the base commit.

@chennyiiis

Copy link
Copy Markdown
Contributor Author

Rebased onto main; the one blocking issue is resolved, and a second thing moved underneath this branch while it waited.

The baseline conflict. scripts/code_metrics_baseline.txt reconciled against the rebased tree. run_grid cyclomatic reads 55 on the rebased tree, matching the entry and the 61 → 55 this PR claims.

A rule that did not exist when this was written. main added a parameter-count limit (fails > 5) and _abort_on_round_stop took nine arguments. The seven describing the variant's run — which variant, in which slot, what its process returned, when it started, and the servers it left behind — are one thing, so they travel as _RoundAttempt. All three call sites build the record; an AST check confirms none passes the nine by name.

Value
run_grid cyclomatic 55 (baseline 55)
_abort_on_round_stop parameters 9 → 3
_grid_runner.py module length 2089 → 2102
CI 33 pass / 0 fail

The module length is the one entry --update-baseline cannot regenerate, since it only lowers: the 13 lines are _RoundAttempt's definition and docstring — the sentence about server_survivors moved there from the function that used to take it as an argument. That is what baseline-raise covers.

Verification: pytest across the 19 test files that reference run_grid or the grid runner, including test_grid_runner_behavior_lock.py — 1818 passed. The one failure is the local TraceLens gap and fails the same way on the base commit.

@chaojhou ready for another look.

Test and others added 5 commits October 20, 2026 14:31
…bort-path

# Conflicts:
#	scripts/code_metrics_baseline.txt
…bort-path

# Conflicts:
#	scripts/code_metrics_baseline.txt
…the rows

The _VariantAborted carried a prebuilt `trailing` list of skipped VariantResults
alongside the halt. The loop that owns the grid already knows which variants come
after the one that aborted, so the list was a second source of the same state.
Replace it with the reason string and let the handler build the rows.

One mechanism fewer: the `trailing` sequence, its two construction sites and the
Sequence import all go; _not_run_skip_result now takes the error and class it
needs rather than a whole StoppedByTheRun.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…n state

_RoundAttempt carried idx, variant and slot, which every one of its three call
sites filled from the enclosing loop's i, variant and slot. That is a second
source of the same state; the helper reads them from the scope it already runs
in, and the tuple keeps only what the round itself produced.

Brings the module back under its recorded length after the merge with main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis chennyiiis removed the baseline-raise Waives new, worse and baseline-growth code-metrics findings; reason required in the PR description label Oct 20, 2026
Test and others added 3 commits October 20, 2026 19:49
…bort-path

Resolve scripts/code_metrics_baseline.txt with resolve_baseline.py: two hunks
retired by main's over-gate deletions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…bort-path

Conflict: scripts/code_metrics_baseline.txt only, resolved with
resolve_baseline.py (1 hunk: try_flydsl_kb_warmstart retired by main,
run_grid kept at this branch's lower 55).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
chennyiiis pushed a commit that referenced this pull request Oct 21, 2026
…grid-runner

#1896 lands first and this branch rebuilt the same abort path over module-level
phases, so _grid_runner.py keeps this branch's shape. Both of #1896's later
commits survive in it: the rows a halted grid never reaches still carry that
stop's own ``never_started`` and ``error_class`` (through the whole
``StoppedByTheRun`` on ``_VariantAborted.abandoned`` rather than a copy of the
two fields), and the round attempt still carries no copy of the loop's state
the slot already names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ZhengGong-amd
ZhengGong-amd merged commit 39c5ac1 into main Oct 21, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants