Repository navigation
refactor(writeback): bring every function in the writeback module under the complexity ceiling - #1956
Conversation
…e_explore _promote_explore carried the whole GEAK stack-revalidation path inline, so one method held the rebench reading, the grading that decides it, the overlay survival check, the material gate, and each of the four ways a ruling can be carried out. Give each of those its own name. _promote_explore: 493 -> 185 lines, cognitive 237 -> 79, cyclomatic 48 -> 24, nested depth 7 -> within the limit. Observable effect: none -- this is a pure refactor. No behaviour, interface, default, flag, or artifact changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…teback-promote # Conflicts: # scripts/code_metrics_baseline.txt # src/hyperloom/orchestrator/loop/writeback.py
… locals The GEAK ruling's steps each took the measurement, the hash, the current best and the candidate as separate arguments, which both repeated the derivation and pushed three signatures past the parameter limit. Read the rebench once into _GeakRebench and pass that: current_best and candidate are then captured before a promotion can move them, so every arm judges the state the rebench was measured against. _rule_on_geak_rebench: 109 -> 78 lines, 7 -> 3 parameters. Observable effect: none -- pure refactor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Code metrics gate: PASSEDJudged: the 4 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.
Module length warning: over 800 lines, within the 1200-line failure limit (not failing): 1
67 findings in files this change does not touchOutside this change (informational): 67
...and 42 more; the job log lists every row.
Thresholds and sources: |
…teback-promote # Conflicts: # scripts/code_metrics_baseline.txt # src/hyperloom/orchestrator/loop/writeback.py
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CodeQL flagged it on this PR because the extraction moved the line, but it is the same dead assignment main carries: the except branch binds geak_result and the refusal branch below rebinds it before any read, on every path that reaches either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Addressed the CodeQL finding at It is a real dead assignment, and it is the one main already carries — Removed. An AST check confirms the first remaining write (L3021) still precedes the first read (L3027). Verification: |
…eft stale Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…aunch identity their own modules The writeback module carried four jobs at once: settling a task result into SharedState, ruling on a GEAK revalidation rebench, replaying a GEAK win through GEAK's own harness, and describing the launch `current_best` was measured on. The last three now live beside it, each as one module reading as one job, and the settlement handlers it keeps name the steps they take rather than inlining them. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…launch descriptor Each of the three moved modules now states its work as a sequence of named steps: what an accepted GEAK result offers to deploy, the params that re-measure it, how a 2a replay is run and what its accuracy and overlay earn it, and which layers reconstruct the launch `current_best` was measured on. The rebench's engagement evidence travels as one value instead of four parallel keyword arguments. The "did not validate" summary drops the accuracy fields it could never reach: an accuracy failure returns before it. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… what they settle The per-kind promote handlers, the current_best lift and the unpromotable-result path each read as the sequence of decisions they make: what grading refuses, what a stack row is attributed to, which recipe delta a KEEP re-applies, what a failed baseline is charged to, and what the anchor run's facts are. The values that travel together — a lift request, the launch it resolves to, a crashed integrate window's marker, a scan of the event log — are named types rather than parallel locals and keyword arguments. Two duplicated decisions go with it: the profile and roofline handlers cleared the pending auto-roofline pointer twice under the same condition, and the GEAK 2a summary carried accuracy fields that an accuracy failure returns before it can ever reach. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`validated` and `fallback` are the 2b ruling's own vocabulary, read by the harness replay as well, so they are named once where the ruling lives. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A scripted edit on Windows rewrote them with CRLF, so every line of all four read as changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… produced it The resume alert for a KEEP the stack lost reported the stack family label, ``framework``, where it had always reported the producing kind, ``framework_agent``. An operator filtering the resume report on the kind no longer saw them. The eval-failure trigger helpers lost their dependency on the collaborator with it: the enablement contract tests bind `_persist_eval_failure` onto a stand-in that carries SharedState alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…riteback modules do not import each other in a cycle ``_PromoteOutcome`` is the vocabulary the promote handlers and the GEAK revalidation share, so it sits beside both rather than in one of them. The GEAK modules now name `Task` through a plain import as well, which is what it always was. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…teback-promote # Conflicts: # scripts/code_metrics_baseline.txt # src/hyperloom/orchestrator/loop/writeback.py
…teback-promote Only scripts/code_metrics_baseline.txt conflicted. Resolved with the ratchet rule: whichever side records the lower value wins, an entry the merge base had and a side dropped is retired, an entry absent from the base is kept. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…teback-promote Brings the branch up to origin/main; no path conflicted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…teback-promote Brings the branch up to origin/main; no path conflicted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…teback-promote # Conflicts: # scripts/code_metrics_baseline.txt
…teback-promote # Conflicts: # scripts/code_metrics_baseline.txt
…teback-promote # Conflicts: # scripts/code_metrics_baseline.txt # src/hyperloom/orchestrator/loop/writeback.py
xiaofei-zheng
left a comment
There was a problem hiding this comment.
What this PR does
loop/writeback.py had grown to 4717 lines with 21 functions above the repo's complexity ceiling of 20 (_promote_explore at 159, lift_to_current_best at 94), so no further change to it could clear the code-metrics gate. The GEAK-revalidation group — the 2b rebench ruling and the 2a replay through GEAK's own harness — moves into a new loop/_geak_revalidation.py as a GeakRevalidation mixin that WritebackCollaborator inherits; the rest is split in place into ~106 private helpers, with values that already travelled together as locals becoming NamedTuples; orchestrator/collaborator.py drops its TYPE_CHECKING import of Coordinator/CoordinatorState so the loop package stays acyclic.
The behaviour-preservation claim holds everywhere I checked. The depth of that check is at the bottom.
Blocking issues
1. The description describes a revision that no longer exists.
The "New module" table names three new modules; the diff has one. loop/_launch_identity.py and loop/_promote_outcome.py do not exist anywhere in the tree — _PromoteOutcome is at src/hyperloom/orchestrator/loop/writeback.py:384 and _handoff_launch_identity at writeback.py:3057 — and writeback.py:88 imports one mixin, not two, so "the first two are mixins" and the three-node DAG sentence both describe an earlier revision (the commit refactor(writeback): keep the launch descriptor and the promote outcome in writeback is where they went away).
The numbers went stale with it:
| Body says | At head |
|---|---|
_geak_revalidation.py is 1080 lines |
818 |
writeback.py 4783 on main → 4123 |
4717 → 4156 |
| "the four files together are shorter than the one they replace" | three files plus the metrics baseline |
Both counts are identical under wc -l and under the gate's own len(read_bytes().splitlines()), so no alternate measurement rescues them.
src/hyperloom/orchestrator/collaborator.py is in the diff and the description never mentions it. It is not cosmetic: CoordinatorCollaborator.__init__ now takes coordinator: Any instead of "Coordinator", and .state loses its CoordinatorState return annotation — the type contract every orchestrator collaborator inherits. Nothing executes differently, but the reason for it (breaking the package cycle) is exactly what the description is there to carry.
Action: rewrite the module table, the line counts and the "four files" sentence against head, and state the collaborator.py change and why it was needed.
2. scripts/code_metrics_baseline.txt records a length writeback.py does not have.
scripts/code_metrics_baseline.txt:467 records module-lines src/hyperloom/orchestrator/loop/writeback.py 3972. The file at head is 4156 lines under the collector's own metric (module_lines, scripts/code_metrics_collect.py:310) — the entry understates by 184.
Nothing in CI reports this. The finding at 4156 lands in worsened against the recorded 3972; excuse_base_backlog then measures the same unit at the merge base (4717), concludes the change did not make it worse, and moves it to base_backlog, which is not part of failed. growth passes too, because 3972 against the base baseline's 4717 is a lowering. So the gate is green on a baseline entry 184 lines short of the tree, and the PR cites that bookkeeping as verified ("32 entries retired, 1 lowered"). It matches the same stale revision as the "4123" figure above.
Action: set the entry to 4156.
What was checked
- Complexity ceiling, the PR's whole point:
ruff check --select C901 --config "lint.mccabe.max-complexity=20"onwriteback.py,_geak_revalidation.pyandcollaborator.pyis clean at head. - Behaviour preservation, four groups compared line by line against the merge base:
lift_to_current_best/build_env_spec/geak_rebench_params/_keep_patch_sources;_promote_exploreplus all of_geak_revalidation.py; the five promote handlers plus_persist_eval_failureand_parse_baseline_workload_extra; the resume/recovery group. No undeclared difference found. Three traps verified by hand at head:maincomputesmeasured_okfromresult["output_throughput"]beforemeasuredis rebound tooutput_tput_of(rebench_measurement);_GeakRebenchpreserves that split viaround_tput(_geak_revalidation.py:402,421).main'sgeak_fallbackbranch returns unconditionally (the onlyreturnin that block), so_reconcile_revalidated_watermarkre-readingresult["output_throughput"]is reached only on the native path._promote_integrate_patchcalls_confirmed_integrate_completion(writeback.py:2838) before_record_authored_levers(2839), restoringmain's order.
fix(writeback): report an orphaned framework KEEP under the kind that produced itis a slip introduced and fixed inside this branch: head reportsorphan_kindfrom the payload's ownkind, and_orphan_stack_keyreproducesmain's(stack_action, variant)key including theFRAMEWORK_STACK_ACTIONremap — identical to the merge base.- Programmatic diff of the merge-base
writeback.pyagainst the two files at head: nodef/classname lost; every numeric literal present with the same value; every string literal present except five docstrings reduced to their summary line and the declaredaccuracy_failure=field drop in the 2a warning. The one module-level name that moved,_MIN_KERNEL_ENGAGED_GAIN_PCT, is still 2.0 with its one reader. GeakRevalidationandWritebackCollaboratorshare no method name, so the mixin cannot shadow anything; the suites that reachwb._PromoteOutcome,_promote_explore,_handoff_launch_identityand_BASELINE_FINGERPRINT_KEYSstill resolve them fromloop.writeback.- CI at head: all 12 test shards, both coverage jobs, ruff, pylint, mypy (advisory), import-linter, CodeQL and code-metrics green.
Base e34bb7afd760de8dd6a60759bb9bf7040b28a9e2, head c95b00ad976da4c4f66c374f8abaaf662cf4f592.
…t of date The entry read 3972 while the file is 4156; the gate excused it as base backlog because the merge base is 4717, so nothing reported the drift.
Both findings fixed: baseline entry corrected to 4156 in d51536f, PR description rewritten against head.
xiaofei-zheng
left a comment
There was a problem hiding this comment.
Both findings from the previous round are fixed in place rather than handed back:
scripts/code_metrics_baseline.txtrecordedmodule-lines .../writeback.py 3972while the file is 4156. Corrected in d51536f. The gate had been excusing the drift as base backlog (merge base 4717), so nothing would have reported it.- The description's module table, line counts and "four files" sentence described an earlier revision with three new modules. Rewritten against head: one new module,
_geak_revalidation.pyat 818 lines,writeback.py4717 → 4156, plus thecollaborator.pytype change and the cycle it breaks.
No blocking issues. LGTM.
What was checked:
- Complexity ceiling, the PR's whole point:
ruff check --select C901 --config "lint.mccabe.max-complexity=20"onwriteback.py,_geak_revalidation.pyandcollaborator.pyis clean at head. - Behaviour preservation, four groups compared line by line against the merge base:
lift_to_current_best/build_env_spec/geak_rebench_params/_keep_patch_sources;_promote_exploreplus all of_geak_revalidation.py; the five promote handlers plus_persist_eval_failureand_parse_baseline_workload_extra; the resume/recovery group. No undeclared difference. Three traps verified by hand at head:maincomputesmeasured_okfromresult["output_throughput"]beforemeasuredis rebound tooutput_tput_of(rebench_measurement);_GeakRebenchpreserves that split viaround_tput(_geak_revalidation.py:402,421).main'sgeak_fallbackbranch returns unconditionally (the onlyreturnin that block), so_reconcile_revalidated_watermarkre-readingresult["output_throughput"]is reached only on the native path._promote_integrate_patchcalls_confirmed_integrate_completion(writeback.py:2838) before_record_authored_levers(2839), restoringmain's order.
fix(writeback): report an orphaned framework KEEP under the kind that produced itis a slip introduced and fixed inside this branch: head reportsorphan_kindfrom the payload's ownkind, and_orphan_stack_keyreproducesmain's(stack_action, variant)key including theFRAMEWORK_STACK_ACTIONremap — identical to the merge base.- Programmatic diff of the merge-base
writeback.pyagainst the two files at head: nodef/classname lost; every numeric literal present with the same value; every string literal present except five docstrings reduced to their summary line and the declaredaccuracy_failure=field drop in the 2a warning. The one module-level name that moved,_MIN_KERNEL_ENGAGED_GAIN_PCT, is still 2.0 with its one reader. GeakRevalidationandWritebackCollaboratorshare no method name, so the mixin cannot shadow anything; the suites that reachwb._PromoteOutcome,_promote_explore,_handoff_launch_identityand_BASELINE_FINGERPRINT_KEYSstill resolve them fromloop.writeback.- CI at c95b00a: all 12 test shards, both coverage jobs, ruff, pylint, mypy (advisory), import-linter, CodeQL and code-metrics green. The baseline commit on top touches only
scripts/code_metrics_baseline.txt.
Base e34bb7afd760de8dd6a60759bb9bf7040b28a9e2, head d51536fb3.
…teback-promote Resolves the two conflicts #2001 left: the launch-control carry in _write_current_best now goes through kept_launch_controls(bv, previous) as on main, with graded_axes_of staying in the dict literal this branch moved it into; the metrics baseline keeps main's entries minus the writeback ones this branch retires, with module-lines at the 4155 the merged file has.
…teback-promote #2091's install_kept_source_layers call and #2098's _integrate_vocabulary / _patch_git import moves apply as on main; the resume consistency pass keeps this branch's _publish_recovered_keeps and _compensate_validation_watermark, which hold the block main only reworded the comments of. Baseline keeps main's entries minus the writeback ones this branch retires, module-lines at 4156.
|
The two red shards here ( Two PRs that landed on
Nothing else is red on this PR: lint, code-metrics, CodeQL, packaging, import-linter and the other 10 shards pass at 328e404, and the branch is MERGEABLE again after the two |
|
Confirmed and fixed on Your read is exact.
Once #2113 is in, I will merge |
…teback-promote # Conflicts: # scripts/code_metrics_baseline.txt
chaojhou
left a comment
There was a problem hiding this comment.
PR #1956 -- refactor(writeback): bring every function in the writeback module under the complexity ceiling
What it does: brings every function in loop/writeback.py under the complexity ceiling of 20. The GEAK 2a/2b revalidation ruling moves into loop/_geak_revalidation.py, a GeakRevalidation mixin on WritebackCollaborator. The rest is split in place into named helpers. collaborator.py drops its Coordinator type import so loop/ stays acyclic. This is a refactor: the only operator-visible change is one warning field, and the description states it.
Status at this head:
- All three findings from my review at 112a442 stay fixed.
- The author's three merges from main keep #2001, #2091 and #2098 in the helpers that now hold that code, in main's order.
- #1845, #2029 and #2075 do not touch the PR's files.
- A trial merge with current main is clean.
Blocking issues: 1
- [X2] The size-exception justification states numbers the diff contradicts [verified]
Problem: the "Whysize-exception" section says "The diff is ~10k production lines" and "the two modules together are shorter than the one they replace". At this head,src/hyperloom/orchestrator/loop/writeback.pyis 4156 lines andsrc/hyperloom/orchestrator/loop/_geak_revalidation.pyis 818, for 4974 in total. At the merge base the file was 4716 lines, so the split adds 258 lines rather than removing any. Production churn is 2408 insertions and 2147 deletions, about 4.5k lines rather than ~10k. The "Module length" section's "32 entries retired, 1 lowered" is 31 retired plus 1 lowered: the 32 removed baseline lines include the lowered entry.
Impact: this paragraph is the PR's answer to the open review objection about net production growth. Thebaseline-raise/ size exception and the release notes would rest on a size reduction that does not exist.
Action: rewrite the paragraph against this head: 4974 against 4716 (+258), about 4.5k lines of churn, 31 entries retired and 1 lowered. Then give the reason the net growth is acceptable, or reduce it; ZhengGong-amd asked for each remaining new type to be named with what it removes.
CI: all checks pass at 93e1905 (35 pass, 7 skipped), including all test shards, coverage and code-metrics. The PR is mergeable.
Checked: loop/writeback.py (the five originally over-ceiling functions: _promote_explore, lift_to_current_best, _promote_baseline, handle_unpromotable_result, _promote_integrate_patch; _write_current_best for #2001; the resume path for #2091; the #2098 import moves), loop/_geak_revalidation.py, collaborator.py, scripts/code_metrics_baseline.txt, the description; the three merges in d51536f..93e1905 against their first parents and main's changes; main commits #2001, #2091, #2098, #1845, #2029 and #2075 | Ran: git merge-tree against current main; line counts and numstat at base and head; imports of writeback and _geak_revalidation on an extracted merge tree; the lift and warm-replay tests on a Windows extract of the merge tree (the one failure there fails the same way on main); definition and type reference counts across src/; rule checklist and two independent refutation passes | Base: ac0c2dd | Head: 93e1905
|
Description updated — the last blocking point was in the Observable effect section, which contradicted itself: it opened with "No observable change: internal structure only. No behaviour, interface, default, flag or artifact changes" and then named a log line that does change. It now leads with the one thing an operator sees:
The framework-levers ordering is stated in the same section rather than buried under the merge notes, and stated as what it is — not a difference against Three figures in the description were also wrong against the current head and are corrected from the real diff:
Head is |
Blocker fixed in the description at 93e1905; superseded by the LGTM comment below.
PR #1956 -- refactor(writeback): bring every function in the writeback module under the complexity ceilingWhat it does: brings every function in Blocking issues: none The size-exception finding from my review at 93e1905 is fixed in the description. It now gives the net change as +258 lines (4156 + 818 = 4974 against 4716 on main), states the churn as 2,409 added and 2,179 deleted, records 31 baseline entries retired with
Checked: the live description against 93e1905 ( LGTM |
|
@ZhengGong-amd On the net-growth point: as maintainer I accept the +258 production lines here. The description now states the growth (4156 + 818 = 4974 against 4716 on |
chaojhou
left a comment
There was a problem hiding this comment.
PR #1956 -- refactor(writeback): bring every function in the writeback module under the complexity ceiling
What it does: moves the GEAK revalidation (the 2b rebench ruling and the 2a harness replay behind it) out of writeback.py into a GeakRevalidation mixin in loop/_geak_revalidation.py, and splits the rest of writeback.py into named helpers, so that every function in both modules scores under 20. Behaviour is unchanged apart from the one log field the description names.
Blocking issues: none
Re-review at 93e1905:
- Equivalence, branch by branch against main ac0c2dd. Covered the promote paths, the GEAK 2b/2a ruling,
build_env_specandgeak_rebench_params, the resume and recovery paths, and the agent KEEP outbox. In each, the same conditions lead to the same effects in the same order, including the levers registered after_confirmed_integrate_completion. - Merge with current main. It merges cleanly with 1a5bcbf; #2075, #2097 and #2119 touch none of these files. The merged tree imports.
- Description. Corrected in place: the module-length paragraph now reads 258 lines longer and 31 entries retired. The complexity table had six stale "After" values, measured before 2ed05d0 folded helpers back in. It is re-measured on the description's own scale, with Before at ac0c2dd and After at this head. The highest value at head is 19.
Ran: branch-by-branch comparison of main and head exports; git merge-tree --write-tree origin/main 93e1905a3 (clean); the targeted writeback test files on head and main (identical results); the complexity scorer on main and head. CI: 35 pass, 7 skipped.
Head: 93e1905
* Move profiler setup abd patching to common path * Fix shared profiler preparation for native AgentX * Simplify shared profiling setup and refresh patch status * Add default values for num_steps * Set detailed annotations to always true as Magpie check compatibility * chore(metrics): lower the _workload_envs module-lines baseline to 1664 * refactor(writeback): bring every function in the writeback module under the complexity ceiling (#1956) * refactor(writeback): lift the GEAK revalidation ruling out of _promote_explore _promote_explore carried the whole GEAK stack-revalidation path inline, so one method held the rebench reading, the grading that decides it, the overlay survival check, the material gate, and each of the four ways a ruling can be carried out. Give each of those its own name. _promote_explore: 493 -> 185 lines, cognitive 237 -> 79, cyclomatic 48 -> 24, nested depth 7 -> within the limit. Observable effect: none -- this is a pure refactor. No behaviour, interface, default, flag, or artifact changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): thread the rebench through one record, not eight locals The GEAK ruling's steps each took the measurement, the hash, the current best and the candidate as separate arguments, which both repeated the derivation and pushed three signatures past the parameter limit. Read the rebench once into _GeakRebench and pass that: current_best and candidate are then captured before a promotion can move them, so every arm judges the state the rebench was measured against. _rule_on_geak_rebench: 109 -> 78 lines, 7 -> 3 parameters. Observable effect: none -- pure refactor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): lower the baseline the merge left out of date Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): drop the geak_result assignment nothing reads CodeQL flagged it on this PR because the extraction moved the line, but it is the same dead assignment main carries: the except branch binds geak_result and the refusal branch below rebinds it before any read, on every path that reaches either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): lower the module-length entry the dead-line removal left stale Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): give the GEAK revalidation, harness replay and launch identity their own modules The writeback module carried four jobs at once: settling a task result into SharedState, ruling on a GEAK revalidation rebench, replaying a GEAK win through GEAK's own harness, and describing the launch `current_best` was measured on. The last three now live beside it, each as one module reading as one job, and the settlement handlers it keeps name the steps they take rather than inlining them. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): name the steps of the GEAK revalidation and the launch descriptor Each of the three moved modules now states its work as a sequence of named steps: what an accepted GEAK result offers to deploy, the params that re-measure it, how a 2a replay is run and what its accuracy and overlay earn it, and which layers reconstruct the launch `current_best` was measured on. The rebench's engagement evidence travels as one value instead of four parallel keyword arguments. The "did not validate" summary drops the accuracy fields it could never reach: an accuracy failure returns before it. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): settle each promote path through steps that name what they settle The per-kind promote handlers, the current_best lift and the unpromotable-result path each read as the sequence of decisions they make: what grading refuses, what a stack row is attributed to, which recipe delta a KEEP re-applies, what a failed baseline is charged to, and what the anchor run's facts are. The values that travel together — a lift request, the launch it resolves to, a crashed integrate window's marker, a scan of the event log — are named types rather than parallel locals and keyword arguments. Two duplicated decisions go with it: the profile and roofline handlers cleared the pending auto-roofline pointer twice under the same condition, and the GEAK 2a summary carried accuracy fields that an accuracy failure returns before it can ever reach. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): name the two rulings a GEAK revalidation can reach `validated` and `fallback` are the 2b ruling's own vocabulary, read by the harness replay as well, so they are named once where the ruling lives. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(writeback): store the writeback modules with LF endings A scripted edit on Windows rewrote them with CRLF, so every line of all four read as changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(writeback): report an orphaned framework KEEP under the kind that produced it The resume alert for a KEEP the stack lost reported the stack family label, ``framework``, where it had always reported the producing kind, ``framework_agent``. An operator filtering the resume report on the kind no longer saw them. The eval-failure trigger helpers lost their dependency on the collaborator with it: the enablement contract tests bind `_persist_eval_failure` onto a stand-in that carries SharedState alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): give the promote outcome its own module so the writeback modules do not import each other in a cycle ``_PromoteOutcome`` is the vocabulary the promote handlers and the GEAK revalidation share, so it sits beside both rather than in one of them. The GEAK modules now name `Task` through a plain import as well, which is what it always was. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): drop the function-length entry _resume_consistency_pass no longer earns Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(writeback): confirm integrate recovery before registering its levers An unconfirmed recovery sets the stop reason, saves the session and raises, so the levers a patch authored must not already be in that saved state: on resume the recovery can roll the patch back while explore keeps benching switches whose code is no longer on disk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): record the line writeback.py lost with the eval opt-out Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): rule on a GEAK win in one module instead of two that import each other The 2a harness replay is the 2b rebench's own fallback: it is reached from the rebench ruling, hands its verdict back as a _ReplayRuling, and shares that ruling's decision vocabulary. Splitting it off left the replay importing the revalidation at module level while the revalidation called into the replay through the mixin, which is a cycle with the evidence removed. One module, one mixin and the cross-module import go. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(loop): reduce the helpers' docstrings to their summary line The restructuring these files needed took writeback's function count from 76 to 176, and the per-function docstring convention took the docstring budget with it. The helpers added here keep a docstring, reduced to its summary line; the Args:/Returns: blocks go, and so does elaboration the signature already carries. The one sentence that states an invariant, a constraint from outside the file, or why a slower path is the correct one is kept where it exists. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(code-metrics): lower the module-length baseline the merge made stale Merging origin/main shortened the module, so its recorded module-lines entry no longer matches the tree and the gate reports the baseline as out of date. Lowered to the value CI measured; nothing else changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(orchestrator): let the collaborator base declare what it borrows `collaborator.py` is imported by every collaborator in the orchestrator, and it named `Coordinator` and `CoordinatorState` to annotate the object it borrows from. That pointed the package root back up into `loop.coordinator`, and it was the return leg of every import cycle CodeQL reports across the `loop` package: 321 cyclic import edges repo-wide collapse to 95 once it is gone. The borrowed surface is declared structurally instead, inside the `TYPE_CHECKING` block the file already had, so the contract is stated by the layer that consumes it and points down. `Coordinator` satisfies it unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(orchestrator): drop the protocol, widen the borrowed annotation The protocol restated nineteen members of the Coordinator in the module that borrows them -- a second declaration of a surface the Coordinator already owns, and it narrowed `_coord` for every subclass that reaches past those nineteen (245 new mypy findings). Widening the parameter instead deletes the same import with no second declaration: `cast` keeps the borrowed types the properties hand to callers, and mypy is back at its pre-change count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Lower the writeback module-lines baseline the merge made stale The merge hoisted the perf_metric imports to module scope, removing three lines from writeback.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): keep the launch descriptor and the promote outcome in writeback Fold `_launch_identity.py` and `_promote_outcome.py` back into `writeback.py` and restore every function they held to its merge-base body, except `build_env_spec`: its two near-identical `source_patch` appends become one comprehension, the recipe path is read once instead of three times, and the promoted-measurement fallback is read in place. `_geak_revalidation.py` no longer imports `_PromoteOutcome`: the GEAK ruling returns the verdict and `_promote_explore` records it, so the loop package's import graph points one way. `_integrate_keep_recipe_delta` and `_replayed_recipe_delta` built the same recipe delta from two different fallbacks; they are now one function that takes the fallback. `_server_arg_workload_fields` is back inside its one caller. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): drop the helper docstrings that restate their own names The one-line docstrings on the promote, keep-origin and GEAK-revalidation helpers repeat the name above them and nothing else; the GEAK module's own header restated the method list below it. Delete them, and fold the two branch-free record builders back into their single callers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(writeback): keep the revalidation-conclusion invariant with the method ``_record_geak_rebench_conclusion`` only moved module; its docstring states that the recorder declines silently once the kernel event has closed, which is an invariant the name does not carry. Restore it verbatim from the merge base. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): record the writeback module length the merges left out of date The entry read 3972 while the file is 4156; the gate excused it as base backlog because the merge base is 4717, so nothing reported the drift. --------- Co-authored-by: chennyiiis <chenyi@amd.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Test <test@test.com> Co-authored-by: xiaofei.zheng <xiaofei.zheng@amd.com> * feat(plugs): add the plug kernel, built-in profiles and the events socket (#2110) * feat(plugs): add the plug kernel, built-in profiles and the events socket Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * refactor(plugs): read the entry-point name once Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * Fix CodeQL findings in test_host activated test Co-Authored-By: Claude <noreply@anthropic.com> * fix(plugs): record a plugin name two distributions declare, and drop the test-only legacy groups iter_entry_point_objects skipped every name it had already yielded, so a second distribution declaring the same implementation name for an open socket never reached the registry: the first one won by metadata order and plugin_errors stayed empty. That dedup existed only for legacy_groups, which no caller passes. Both are gone; the registry's own duplicate check now records the conflict. load_manifest_file's docstring says JSON, which is all it reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Hou <chaojhou@amd.com> * feat(hosted): read and check the host-injected inputs in one place (#2114) * feat(hosted): read and check the host-injected inputs in one place Add hosted/host_info.py: read_host_info(env, require=...) checks the inputs a host injects into a hosted run, names every missing or malformed variable in a single error, and returns a frozen HostInfo. Credentials are held in a Secret that never prints. Document the variables. Co-Authored-By: Claude <noreply@anthropic.com> * fix(hosted): keep a URL query and a template key out of host-info errors and reprs A credential in a URL query or fragment would have been stored and printed in the HostInfo repr, and a malformed template entry's key was echoed in the error. Refuse query and fragment in the URL inputs and name no key in the template error. Co-Authored-By: Claude <noreply@anthropic.com> * test(hosted): derive the credential sentinels instead of spelling key-shaped literals The secret scanner flagged the fixture values. Sentinels derived from the variable's position are as distinctive for a substring search and carry no key-shaped literal. Co-Authored-By: Claude <noreply@anthropic.com> * feat(hosted): take_host_info removes every host-only input from the environment A scheduler that starts children without env= cannot strip inputs per launch, so the inputs must not stay in os.environ. NODE_NAME and CLAW_SESSION_ID stay because other code reads them. Co-Authored-By: Claude <noreply@anthropic.com> * feat(hosted): hold the archive region in HostInfo Co-Authored-By: Claude <noreply@anthropic.com> * feat(hosted): make the result archive optional; shared storage when unset HYPERLOOM_RESULT_ARCHIVE is unset (results stay on the shared storage) or s3://<bucket>/<prefix>. The AWS key pair is required only for s3://. Co-Authored-By: Claude <noreply@anthropic.com> * fix(hosted): refuse an archive prefix with an empty or '.' segment instead of rewriting it _archive_location split the prefix with PurePosixPath, which drops '.' segments and collapses repeated slashes before the check runs, so s3://b/a/./b, s3://b//a, s3://b/a// and s3://b/a/. were accepted and rewritten to another prefix. S3 keys are literal, so the results would land under a key the host did not name. The raw path is now split as written; one trailing slash still names the same prefix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Hou <chaojhou@amd.com> * [RSI] Block preflight unless the framework probe exits clean (#2099) * [RSI] Block preflight when the framework probe finds no usable GPU * Move preflight device-init detection to cli/device_init * Block preflight on any probe without a ROCm verdict * ci(code-metrics): tighten function length and nesting rules, require a reason on broad noqa (#2127) * ci(code-metrics): tighten function length and nesting, require a reason on broad noqa - Function length counts the lines of code in the body (no def lines, decorators, blank, comment or docstring lines); the limit drops from 80 to 50. - Nested block depth limit drops from 5 to 4. - Enable ruff B006 (mutable default argument) and B012 (jump in finally). - An added '# noqa: BLE001' must give a reason on its line. - Teach the gate a directory-size metric (entries directly in a directory); it stays pending until its threshold is added. - Re-seed the function-lines and nested-blocks baseline entries. * refactor(code-metrics): keep the A10 comparison walk within four nested blocks The gate scripts carry no baseline entries; split the match-case and compare branches of _comparisons into their own generators. * docs(agents): two review triggers remain once function length is gated --------- Co-authored-by: chaojhou <chaojhou@amd.com> * fix(prelude): commit a promoted warm replay's overlays on the framework tree (#2122) * fix(prelude): commit a promoted warm replay's overlays on the framework tree The overlays were left as uncommitted worktree changes, so every integrate_patch candidate stashed them before it applied and was benched on the stock tree. Checkout promotion now commits them on each git root; a commit that cannot be made fails the promotion, and a lift that refuses the replay moves HEAD back before the rollback restores the tree. Refs #2120 * fix(prelude): undo the promotion commit in every warm replay rollback The commit was undone only when the lift refused the replay. A commit failing on a later tree left the earlier trees committed, and a session killed after the commit resumed with a pending record still naming the pre-patch sha; both rollbacks then refused on head_mismatch. The pending record now says the overlays are being committed, persisted before the commit, and rollback_combined_warm moves a tree back to its pre_sha before it restores the files. * fix(prelude): undo the promotion commit on a tree the replay found already holding the overlays rollback_combined_warm skips a tree the replay did not write to, so a present_in_dirty_worktree overlay stayed committed after a refused replay. The undo now runs before that skip and resets the index too, leaving the overlays uncommitted as they were before the promotion. --------- Co-authored-by: BaoYunkai <269084791+BaoYunkai@users.noreply.github.com> * refactor(preflight): delete the framework-deps installer and preflight's unreachable branches (#2128) * refactor(framework-agent): drop unused __version__ constant Nothing reads the package-level version string. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): remove the per-framework dependency-manifest installer assets/framework_deps/ no longer exists, so the installer never had a manifest to act on. Drops framework_deps.py, the preflight framework_deps step, the install.sh pass and its tests and doc entries. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): drop the unreachable no-LLM-endpoint branch of _export_llm_endpoints _validate_credentials exits before endpoint export unless a base URL resolves, so resolved_urls is never None. Narrow it to tuple[str, str] across preflight and the model-resolution helpers and remove the dead None guards. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): make args required in _preflight and drop the recipe_kb_enabled copy _preflight and its helpers took an optional args only for tests; the sole production caller passes a Namespace, so the args-is-None branches and the IR-3 args_unavailable skip are gone. recipe_kb_enabled duplicated not args.degraded_kb and is derived at its one reader instead. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): share the rocm-smi GPU probe with detect_gpu_count device_init._rocm_smi_visible_gpus carried a line-for-line copy of detect_gpu_count's rocm-smi call and GPU[ dedup parse. visible_devices now owns the probe, keeping "could not probe" apart from "zero GPUs", and both use it. Co-Authored-By: Claude <noreply@anthropic.com> * chore(metrics): ratchet the code-metrics baseline for the preflight cleanup Lower the preflight.py `_preflight` function-length and module-length entries to match the tree after the cleanup. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): derive the diagnostics KB status from kb_degraded_reason The IR-3 step sets kb_degraded_reason before the diagnostics step runs, so the diagnostics no longer re-read degraded_kb. Also pin the skipped install-step contract of _check_gpu_visibility when rocm-smi cannot be probed, and restore a docstring in the auth-override fixture. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): replace the RocmSmiProbe result type with a raising rocm_smi_gpu_count The probe now returns a plain count and raises the standard subprocess/OS errors; device_init and detect_gpu_count map them themselves, so the dataclass, its Literal and their exports go. Drop the probe tests that duplicate detect_gpu_count and preflight coverage, the tautological degraded_kb asserts, the inaccurate "falls back to the other side" docstring, and the now-false "no LLM base URL resolved" diagnostics text. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Hou <chaojhou@amd.com> * Delete .github/workflows/auto-label.yml (#2137) * fix(grid): a profiled AgentX round that Magpie completed is a succeeded round of run_grid (#2139) run_grid counted a round a success only when its measurement was valid. Magpie marks a profiled AgentX replay diagnostic_only and not publishable, so such a round was a failed variant (benchmark_report_invalid_metric) even though Magpie reported the profile capture complete. run_grid now asks round_succeeded: for a native AgentX round it is Magpie's own verdict (success, Magpie-owned, valid or diagnostic), for any other round it is valid_measurement. ProfileExecutor already used that predicate inline; it now shares the one in benchmark_result. A diagnostic round stays diagnostic_only and not benchmark_valid, so it is no KEEP evidence. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: chaojhou <chaojhou@amd.com> * ci(code-metrics): gate directory size at 50 entries (#2136) * ci(code-metrics): gate directory size at 50 entries Directory size counts the production modules directly in a directory and its subdirectories that hold one. Baseline the two directories already over. * chore(code-metrics): drop function-args from PENDING Its threshold has been in pyproject.toml since the parameter-count gate landed, so the name should have left PENDING then; a config without it is now refused. * fix: parse every diff header shape, reject malformed HYPERLOOM_MN_* settings, restore kept patches on resume (#2130) * fix(common): decode quoted diff paths as UTF-8 and parse every header shape parse_patch_manifest read git's octal path escapes as Latin-1 characters ("b/caf\303\251.py" became "b/café.py"), rejected the header-only deletion of a binary or empty file (it has no --- line), rejected header-only binary and mode-only sections whose names are quoted or carry no a/ b/ prefix, and split non-git diffs at hunk lines such as "--- old comment", adding a write target the patch never makes. The no-git applier, the kernel snapshot apply and snapshot materialization all take their targets from this manifest, so each of these refused or misdirected a valid patch. Quoted names are now C-unquoted to bytes and read as strict UTF-8, the two diff --git names are split the way git apply splits them, a header-only deletion takes its path from that line, and non-git sections are walked by their hunk line counts. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): read HYPERLOOM_MN_POLL_TIMEOUT_S through one strict resolver Three readers applied three rules to the same variable: the multi-node CLI warned and fell back to 110s on a malformed value, optimize's rayjob adoption crashed on it (and on a whitespace-only value) with a bare int() ValueError, and the restart path read it with env_int and then took max() with the CLI's lenient read, which was never decisive. resolve_poll_timeout_s(default) is now the only reader: blank means unset, the value is floored at one second, and a malformed value raises EnvValueError naming the variable. The multi-node CLI reports that as a config error, not as a transient failure a re-run could fix. Observable effect: valid values behave as before. A malformed HYPERLOOM_MN_POLL_TIMEOUT_S now fails a multi-node CLI subcommand with exit 3 and optimize's cluster adoption with an EnvValueError naming the variable, instead of a warning plus a 110s budget or an unexplained int() error; a whitespace-only value counts as unset. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(executors): validate restart settings before the multi-node restart starts Every numeric HYPERLOOM_MN_* setting a per-round restart reads is now resolved once, up front, into RestartSettings. A malformed one used to surface wherever it was first read: the poll timeout and the health waits raised a bare EnvValueError past the restart's callers, which handle only ServerRestartFailed, and the health-poll, worker and probe settings were read inside the restart attempt's broad except, so each attempt killed and relaunched the server, swept its logs and reclaimed VRAM before failing with "post-launch /health wait raised". The restart now raises ServerRestartFailed("invalid multi-node restart setting: ...") before the TraceLens fan-out or any kill, which every caller already handles. A blank HYPERLOOM_MN_HEALTH_WAIT_S now counts as unset for the aiter wait widening, as it already did for the wait itself. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): read the serving-probe settings like the restart path The resume probe in restart-server and the post-restart wait read HYPERLOOM_MN_COMPLETION_PROBE_TOKENS and _MIN_TOKENS so that they agree on what "serving" means, but the probe used its own reader, which turned a malformed value into the default. It now reads both with env_int before its first request, outside the except that maps a ValueError to "not serving", so a malformed value raises EnvValueError naming the variable: the multi-node CLI exits 3, and a per-round restart has already rejected it up front. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kernel-agent): back every kernel-patch apply up under the session patches dir The apply backup root came from the payload when one was given, so a kernel patch's manifest could land anywhere. Every apply now backs up under <session>/patches/<kernel_id>/backup/, the one place the multi-node adoption looks for kept patches, and the policy gate no longer lists backup_root as a payload path. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): restore kept kernel patches on the pods when adopting a cluster A pod created after a KEEP starts from the image's sources. The old replay scanned kernel-agent-workspace for manifests with status=applied, a directory and a status no kept patch ever has, so it never ran. Adopting a cluster now sends every file a finalized patch changed, as the sandbox holds it, to every pod, carrying the AITER JIT invalidation once per build dir, and stops the run when a file cannot be restored. A cluster without server control (benchmark-only) is left as it is, since no restart would load the files. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(kernel): describe the kept-patch restore on cluster adoption Co-authored-by: Cursor <cursoragent@cursor.com> * chore(code-metrics): lower the duplicated-lines entries the removed replay test leaves behind test_multi_node_patch_replay_skip_and_failure_paths was a copy shared by test_common_utils.py and test_coverage_gap_units.py; deleting it drops their duplicated lines to 785 and 581, the values CI measured. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): restore kept kernel patches in the resume pass, after the kept source layers The restore ran when the optimize CLI adopted the cluster, which made hyperloom.inference_optimizer.cli import the orchestrator: an upward import the layer contract forbids. It now runs in the Coordinator's resume pass, right after the kept source layers are back on this host's tree, so the pods get the kept files from a tree that holds them. A kept file that cannot be read or sent halts the resume with patch_recovery_incomplete, as a source layer that cannot be reinstalled does. A run that is not multi-node, or a cluster without server control, sends nothing. external_has_server_control is exported from hyperloom.inference_optimizer.multi_node so the orchestrator reaches it without the package's _internal modules. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kernel): let a failed pod backup cleanup leave the resume running The restore's cleanup dispatch raises TimeoutExpired when one pod stalls, and nothing caught it, so a resume whose kept kernel files had all been sent died while deleting their pod-side backups. A raised cleanup is now treated like a reported one: a warning, and the resume continues. The kernel tool's docstring now names the resume pass, which replaced the adoption replay, as the reader of the backups. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kernel): skip a kept kernel file a later KEEP deleted on resume A file a later snapshot KEEP deleted is absent from this host, and the restore stopped the resume because a kept file was missing, so every resume of such a session ended as patch_recovery_incomplete. This host's tree is the reference and the pods keep their copy after that KEEP itself, so the file is now not sent. A JIT build dir is now carried only by a file that is sent, so another KEEP's file carries it when the first carrier was deleted. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(multi-node): read the published-service gate's settings in place The gate bound published_ready_s and health_poll_s to local aliases that are never rebound. Reading them from the settings, as the worker and /health waits do, keeps the function's body at its length on main under the body-line count the code-metrics gate now uses. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> * chore(code-metrics): record the executors directory at 84 entries (#2148) #2130 added _multi_node_restart_settings.py to src/hyperloom/orchestrator/actions/executors and was merged right after #2136 set the directory-size threshold, with CI that predated the gate. The directory now has 84 entries against a recorded 83, and directory-entries has no merge-base exemption, so every change touching a file under it fails code-metrics on a count it did not change. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * fix(grid): a round reaped before its server was ready names the server log its workspace holds (#2140) The server log of a round that the session or boot limit reaped sits in the round's benchmark_* directory under the variant slot. The lookup that fills server_log_path searched the slot's warmup round only, so such a round was recorded skipped with no server_log_path, and nothing could quote the line its log ends on. The lookup now searches the benchmark_* directories of the slot as well as those of its warmup round. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: chaojhou <chaojhou@amd.com> * refactor(critic, kb): remove dead Critic request kinds and unused KB plumbing (#2129) * refactor(critic): accept only coordinator_inbox requests and drop dead runtime paths CriticAgentBackend only ever writes coordinator_inbox requests. The four dialogue request kinds (critic_decision_request, kb_draft_request, kb_hint_request, objection_signal) have no producer anywhere in the repo, so remove them together with everything that existed only for them, plus unreachable branches and unread state in the runtime. Runtime changes (src/hyperloom/agents/critic/runtime/): request_models.py - Remove DECISION_REQUEST, KB_DRAFT_REQUEST, KB_HINT_REQUEST, OBJECTION_REQUEST and REQUEST_KINDS; parse_request accepts only coordinator_inbox and always requires a non-empty raw_prompt. - Remove the decision_id auto-generation for decision/objection kinds. - Remove CriticRequest.messages, .decision, .raw and CriticRequest.to_dict. - Add Proposal.from_payload, the single owner of the action_name and predicted_gain_pct extraction (previously copied in inbox_parser). decision_reviewer.py - Remove _commit_decision_request, _commit_kb_draft and _topic_for_decision; commit_review always commits a coordinator inbox. - _fetch_kb_priors looks up priors per proposal only; the trace mode is always per_proposal (an idle round records requests=[]). - Remove JudgeBundle.messages, .decision, .kb_priors_for_decision and CommitOutcome.decision_review, .notes. - DecisionReviewer.__init__ takes a required kb_writer and reuses its session_memory; the kb_client argument and in-memory fallback go. - _populate_inbox drops the kind/raw_prompt guards and the InboxParseError catch; _maybe_write_kb_for_verdict drops the unreachable RuntimeAdapterError catch. - ceiling_importance reads CRITIC_IMPORTANCE_CEILING instead of a literal 0.84. - decisions.jsonl inbox records no longer carry a duplicate inner ts. kb_writer.py - Remove write_kb_drafts and slug_for_kind. - Remove WriteContext.source_role, source_type, topic and extra_metadata; metadata writes "source_role": "critic" directly. - KBWriter requires session_memory. - Fold the three dead-letter except branches of _upsert_with_dead_letter into one: reason is validation_error for KBValidationError and transport_error otherwise; only KBTransportError trips the breaker. kb_client.py, in_memory_kb_client.py - Remove batch_insert and add_edges (Protocol, HTTP and in-memory). cli.py - replay-dead-letter replays only upsert records; batch_insert, edges/add and list branches go. - Reuse DEFAULT_TIMEOUT_MS/DEFAULT_RETRY_MAX; drop the duplicate KB_SERVICE_TOKEN and KB_DEAD_LETTER_DIR reads (owners read them). - _resolve_reviewer builds DecisionReviewer(kb_writer=...). inbox_parser.py - ParsedPrompt keeps only shared_state and proposals; drop agent_name, inbox, kb_hints_text, extras, raw_payload, both to_dict methods, _agent_from_inbox_title and the "Knowledge base hints" branch. - InboxRow becomes the private _InboxRow. intent_envelope.py - build_envelope drops its inline idle fallback (decision_reviewer owns the idle intent; an empty list now fails validation). - build_review_verdict_intent drops the verdict and target checks that _commit_coordinator_inbox already enforces; the source check stays. - DEFAULT_IDLE_BODY leaves __all__. session_memory.py - MergeResult keeps merged and missing_keys; merge_context always persists and drops its no-op second loop. - get_cached_priors drops the now= parameter. slugify.py - Remove the no-op repeated-dash fold and slugify_safe's fallback_prefix. errors.py, importance_mapping.py, category_mapping.py - Remove InboxParseError, importance_for_kb_draft and the whole category_mapping module. Callers and docs: - orchestrator/roles/critic_agent.py: drop kb_priors_for_decision from _build_kb_priors_trace and from the _reason bundle view. - agents/critic/SKILL.md: describe only the Coordinator inbox caller; drop the dialogue Mission item, the request-type table, KB drafts and needs_info mentions. - agents/critic/README.md: replace the A2A hosting guide with the CriticAgentBackend flow; drop the Bash allowlist; keep Exit codes. - Delete actions/review_decision.md, actions/draft_kb.md, actions/objection_signal.md and references/decision_review_schema.md; drop the KB draft sections of references/verdict_schema.md and references/risk_rules.md; update references/coordinator_protocol.md. - pyproject.toml: fix the critic package-data comment. - scripts/code_metrics_baseline.txt: drop the write_kb_drafts entry. Tests: delete the tests of removed behaviour; move the empty-target check to test_commit_review_empty_target_raises; pin the idle-round trace (configured=True, requests=[]); pin that a bare KBError dead-letters as transport_error without opening the breaker. Observable changes: - prepare-review/commit-review reject the four dialogue kinds (rc=2). - An idle round no longer looks up KB priors by decision topic; its kb_priors trace keeps configured=True with requests=[] and prior_count=0. - judge_bundle.json loses messages, decision and kb_priors_for_decision; the Critic prompt's bundle view loses kb_priors_for_decision and SKILL.md changes, so the prompt cache prefix is invalidated once. - emit.json loses notes; decisions.jsonl inbox records lose their duplicate inner ts. - The dead-letter queue no longer produces batch_insert.jsonl; a leftover one now fails replay as an unknown endpoint and stays queued. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(kb): drop dead framework-KB start-up code and unused KB client, Experience KB and recipe-identity surface Framework KB (src/hyperloom/agents/framework/kb.py): - Remove prepare_kb_environment, check_kb_configuration, migrate_legacy_partition_once, _copy_partition_atomically, _LEGACY_WORKSPACE_KB_DIRNAME and _REMOVED_KB_ROOT_ENV, with the logging/shutil/uuid imports they needed. The legacy <workspace>/kb migration only served pre-release alphas, and FRAMEWORK_AGENT_KB_DIR has not been read for several releases. - read_pr_ledger() takes no argument and resolves through framework_optimization_root(), the same owner the writer uses. Preflight (src/hyperloom/inference_optimizer/cli/preflight.py): - Remove the prepare_kb_environment install step and its kb_withdrawn_override/kb_enabled locals; IR-3 still owns recipe_kb_enabled and kb_degraded_reason. - .importlinter: drop the two preflight -> agents.framework.kb ignores that no longer match. Critic KB clients (src/hyperloom/agents/critic/runtime/): - in_memory_kb_client.py: remove the test-only surface (simulate_failure, _maybe_fail, _failure_queue, all_rows, reset), the never-firing scope_value_normalized warning, the metadata filter, the deleted flag on rows, and the local copies of the scope normaliser and scope key. - kb_client.py: remove KBListQuery.metadata_filter, sort_by and include_deleted; HTTPKBClient.list sends sort_by="updated_at_desc" and include_deleted=False as literals, so the request body is unchanged. - scope_builder.py: expose normalise_scope_value, scope_key and UNKNOWN_SCOPE_VALUES as the single owners; scope_cache_key builds on scope_key (keys are byte-identical). - session_memory.py: _is_missing reads UNKNOWN_SCOPE_VALUES instead of its own copy. Experience KB (src/hyperloom/inference_optimizer/experience_kb.py): - Remove _manifest_context and every manifest/architecture fallback in _context: manifest.json never carries task_config or metadata, and SharedState has no architecture, compute_partition_mode, partitions, current_best_tput or best_tput attribute. The read payload sent in production is unchanged; _current_best_throughput stays local (importing orchestrator from inference_optimizer would break the hl-top-layers contract). Remove the now-unused _first_value. - from_env relies on RemoteConfig.from_env returning None for a blank HYPERLOOM_KB_URL instead of repeating the check. - Remove ExperienceKBEvidence.warnings, which nothing reads; the reads-off state is still logged. Recipe identity: - recipe_snapshot_constants.py: remove 15 unreferenced wire constants (F_AUTHORITY, F_CONFIDENCE, F_EVIDENCE_REFS, F_PROVENANCE, F_CANONICAL_ID, F_VERSION, F_METRIC_MIN, F_METRIC_MAX, F_PV_DETAILS, AUTHORITY_EXPERIENTIAL, ORDER_BY_*, FOREGROUND_HTTP_TIMEOUT_SEC, DEFAULT_CONFIDENCE), the test-only canonical_labels with its seven F_LABEL_* keys, and the stale "single source of truth" docstring. - orchestrator/knowledge/recipe_kb/__init__.py and canonical_id.py: stop re-exporting names nobody imports through them (CANONICAL_ID_*, DEFAULT_*_SLUG, LOCK_FILENAME, canonical_labels, detect_framework_version). recipe_canonical_id stays exported, so recipe_journal and recipe_kb_t0 keep their imports. Docs and baselines: - .env.template and docs/reference/environment-variables.md: drop the FRAMEWORK_AGENT_KB_DIR and legacy-partition migration sentences. - scripts/code_metrics_baseline.txt: drop canonical_labels; lower preflight.py::_preflight to 193 and preflight.py module lines to 2637. Tests: delete the tests of removed behaviour (legacy migration, withdrawn-variable warning, start-up hook, canonical_labels, metadata filter, deleted rows, fault injection); observe the in-memory KB through list(); carry the validation-error dead-letter contract to a stub upsert; pin the HTTP list body including the sort_by and include_deleted literals; read the PR ledger through the env override; build the Experience KB read context from state fields only. Observable changes: - The SBD V6 install event no longer has a prepare_kb_environment step. Setting FRAMEWORK_AGENT_KB_DIR no longer logs a warning or marks the install event degraded; --degraded-kb is still recorded by IR-3, the manifest's kb_degraded_reason and the diagnostics kb_status. - A workspace that only ever ran pre-release alphas no longer has its <workspace>/kb/framework_optimization ledger copied to framework-kb; the FRAMEWORK phase cold-starts that ledger instead. - In-memory critic KB rows no longer carry "deleted": false, and ExperienceKBEvidence no longer carries warnings. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(experience-kb): drop the session_dir the integration no longer reads _manifest_context was the only reader of ExperienceKBIntegration.session_dir, and it went with the dead manifest fallbacks. Remove the parameter from ExperienceKBIntegration.__init__, ExperienceKBIntegration.from_env and integration_for, and from Coordinator.experience_kb and the tests that threaded it through. Nothing observable changes. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(critic): drop the request kind now that the Critic accepts one With only coordinator_inbox left, the request's "kind" carried no information but was still validated, stored and echoed. Remove it end to end: - orchestrator/roles/critic_agent.py: the request no longer carries "kind" and the judge-bundle view sent to the LLM drops it. - runtime/request_models.py: parse_request no longer reads or validates "kind" (a stray "kind" key is ignored like any other unknown key); drop CriticRequest.kind and COORDINATOR_INBOX. - runtime/decision_reviewer.py: drop JudgeBundle.kind and CommitOutcome.kind, so judge_bundle.json and emit.json no longer carry "kind". - SKILL.md: replace the "Request Type" section with a pointer to the inbox action; actions/review_coordinator_inbox.md and README.md stop describing a request kind. - scripts/code_metrics_baseline.txt: lower CriticAgentBackend._reason to 73. - Tests: drop "kind" from every request, bundle and emit fixture, the JudgeBundle construction and the view-key table; delete test_parse_non_inbox_kind_rejected. Observable changes: request.json, judge_bundle.json and emit.json no longer have a "kind" key, and the Critic prompt's judge-bundle view loses its "kind" entry, so the prompt cache prefix changes once more. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(recipe-identity): drop the stale remote-URL comment recipe_snapshot_constants no longer holds any remote wire constants, so the note about there being no default KB_STORE_URL no longer describes anything in the module. Comment-only change. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Hou <chaojhou@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): select plug implementations from optimize, share entry-point discovery (#2111) * feat(plugs): add the plug kernel, built-in profiles and the events socket Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * feat(cli): select plug implementations from optimize, share entry-point discovery Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * refactor(plugs): read the entry-point name once Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * refactor(cli): keep optimize's growth-gated units no larger than main's Move the no-target-GPU warning into bootstrap and the resume flags into their own parser helper, and add the plug flags in one loop, so _run_optimize, _build_parser and the cli module are not larger than at main. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * chore(metrics): tighten the baseline for two units this change shrank Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * Fix CodeQL findings in test_host activated test Co-Authored-By: Claude <noreply@anthropic.com> * Rename the optimize plug flag --profile to --plug-profile The flag no longer shadows the argparse abbreviation --profile -> --profile-osl, so existing invocations keep parsing as before. Adds regression tests for both. Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(cli): resolve and plan the plug manifest before any session state is written A refused plug flag, manifest variable or damaged plugs.resolved.json used to exit 2 only after the session was opened or seeded: each refused --resume-from ... --extend-hours grew the budget again, --force-resume cleared a terminal target_reached, and a fresh run left a half-seeded session. The manifest is now resolved and planned first (resolve_plug_manifest, no writes) and only bound to the session, recorded and announced afterwards (open_plug_host). Co-Authored-By: Claude <noreply@anthropic.com> * test(cli): assert the plug refusal itself, not only exit 2 Co-Authored-By: Claude <noreply@anthropic.com> * fix(kernelforge): log a failed provider load once The shared helper already logs it; the registry logged it again. Also pins that two entry points with one name are recorded as a plugin error. Co-Authored-By: Claude <noreply@anthropic.com> * docs: document the plug flags, the two manifest variables and the resume behaviour Co-Authored-By: Claude <noreply@anthropic.com> * refactor(cli): resolve_plug_manifest reads --resume-from itself; keep cli/__init__.py no longer than main Co-Authored-By: Claude <noreply@anthropic.com> * Lower code-metrics baseline entries that main improved Co-Authored-By: Claude <noreply@anthropic.com> --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: chaojhou <208445444+chaojhou@users.noreply.github.com> * refactor(writeback): give each promotion and recovery job its own module (#2132) `orchestrator/loop/writeback.py` had grown to 4156 lines: a hundred-method `WritebackCollaborator` plus the ~750 lines of helpers it read. The jobs it had collected -- staging a KEEP into the patch column, validating cumulative gain, charging a baseline failure, lifting a winner into current_best, promoting each of the five result kinds, pinning the launch a measurement ran under, replaying a resumed log, recovering the windows a crash left open, and auditing a resumed run -- each get a module, and the five-result dispatcher stays where it is. Every unit moves verbatim; the twelve stage modules are mixins the collaborator composes, so `coordinator.py` still constructs the same class. Co-authored-by: chenyi <test@test.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * set profiler config flags after probing the container. * Revert "set profiler config flags after probing the container." This reverts commit 3205af7. Magpie already probes the container for profiler support before it launches, so a second probe here is redundant, and it is the weaker of the two: it returns False for a server that lacks the fields, for a non-zero exit and for a probe that could not run at all, writes the same detailed_annotations: false in every case, records no degraded reason, and caches the verdict for the process. A probe that times out on a loaded node then costs a full profile round whose trace TraceLens cannot split, with nothing marking the round degraded. Back to the agreed shape: default true, explicit false preserved, Magpie checks runtime support. * fix(agentx): follow the patch outcome for the native annotation default Magpie defaults detailed_annotations to false and probes the server only when it is true, and that probe raises rather than degrading when the server lacks the fields. enable_profile claimed the field unconditionally, so a native profile round on a runtime the TraceLens patch did not reach ended before the server started, where the fixed-sequence path degrades instead. Both paths now read one predicate over the degraded reason patch_for_profile already writes; an explicit operator value is still preserved either way. --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Deval Shah <devashah@amd.com> Co-authored-by: Hao Li <18546749+haofrank@users.noreply.github.com> Co-authored-by: Tharun Adithya Srikrishnan <tsrikris@amd.com> Co-authored-by: Yang, Chenyi <chenyyan@amd.com> Co-authored-by: chennyiiis <chenyi@amd.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Test <test@test.com> Co-authored-by: xiaofei.zheng <xiaofei.zheng@amd.com> Co-authored-by: zoroyihan7 <Yihan.Wang@amd.com> Co-authored-by: Hou <chaojhou@amd.com> Co-authored-by: luochen <chenluo@amd.com> Co-authored-by: lishuoshuo-amd <shuoli@amd.com> Co-authored-by: BaoYunkai <ybao@amd.com> Co-authored-by: BaoYunkai <269084791+BaoYunkai@users.noreply.github.com> Co-authored-by: Zheng Gong <zgong@amd.com> Co-authored-by: fengshaoyi-amd <shafeng@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: meinali-566 <meili@amd.com> Co-authored-by: chaojhou <208445444+chaojhou@users.noreply.github.com>
What this does
writeback.pywas doing three jobs at once: settling a finished task result intoSharedState, ruling on a GEAK end-to-end win, and describing the launch
current_bestwas measured on. The GEAK ruling now lives in a sibling module ofits own, and every function in both measures under 20 on the Cmetrics scale.
loop/_geak_revalidation.pyIt is a mixin on
WritebackCollaborator, so every call site and every method nameis unchanged. The import graph is a DAG:
writebackimportsGeakRevalidationand nothing imports back. An earlier revision also split out
loop/_launch_identity.pyandloop/_promote_outcome.py; both were folded backinto
writeback.py, so the launch descriptor (_handoff_launch_identityandfriends) and
_PromoteOutcomestay there.src/hyperloom/orchestrator/collaborator.pychanges with them.CoordinatorCollaborator.__init__now takescoordinator: Anyand the borrowedproperties
cast(...)to their own types, because namingCoordinatorhere —in a module every collaborator in the orchestrator imports — points the package
root back up into
loop.coordinatorand makes the wholelooppackage cyclic..stateloses itsCoordinatorStateannotation for the same reason. Nothingexecutes differently;
castis a no-op at runtime.The functions that were over the ceiling
Measured on the Cmetrics scale —
ruff C901's statement branches plus eachand/oroperand past the first, each ternary and each comprehension clause."Before" is the value on
mainat the merge baseac0c2dd97, "After" the value at this head:lift_to_current_best_promote_explore_promote_baseline_promote_integrate_patchhandle_unpromotable_resultvalidate_geak_via_geak_harness_replay_keep_from_resultbuild_env_specgeak_rebench_params_recover_pending_integrate_from_task_drain_agent_keep_outbox_keep_patch_sources_resume_recover_orphaned_keeps_promote_profile_enqueue_agent_keep_outbox_resume_recover_pending_integrate_persist_eval_failure_resume_consistency_pass_parse_baseline_workload_extra_promote_rooflinerecord_integrate_keepThe 2b ruling this PR's first revision lifted out of
_promote_explorewas itselfover the ceiling, and is under it now too:
_geak_rebench_decision32 → 13,_rule_on_geak_rebench21 → 15.Each promote path reads as the sequence of decisions it makes — what grading
refuses, what a stack row is attributed to, which recipe delta a KEEP re-applies,
what a failed baseline is charged to, what the anchor run's facts are — rather
than one scroll of nested branches. Values that travel together became named
types (
_LiftRequest,_ResolvedLaunch,_GeakProduct,_RebenchEngagement,_IntegrateMarker,_KeptResultScan,_ExploreLift,_StackKey,_ReplayRegime,_ReplayOverlay), which is also what brought_record_geak_rebench_timelinefrom eight parameters down to five.Three duplicated or dead mechanisms went with it:
pointer twice, under the same condition, in the same call;
accuracy_failurefields thatan accuracy failure returns before it can ever reach;
_promote_baselineguarded twogetattr(task, ...)reads with atask is not Noneternary that could not change their result.Observable effect
One line of output changes. The GEAK 2a "did not validate" warning no longer
prints an
accuracy_failure=field. On that path an accuracy failure returnsbefore the field can be filled, so an operator reading that warning on
mainsees an empty value; they will now see no field at all. Nothing else an operator
can observe changes: no interface, default, flag, artifact, persisted shape or
ordering of effects.
The framework-levers ordering is not a difference against
main. An earlierrevision of this PR had
_promote_integrate_patchregister the patch's leversbefore
_confirmed_integrate_completion— that check sets the stop reason,saves the session and raises when a tree recovery is unconfirmed, so the levers
would have been persisted for a patch that resume may roll back, leaving explore
benching switches whose code is no longer on disk.
main's order is restored, soan unconfirmed recovery persists no levers, exactly as it does today.
Merged with current
mainmainmoved while this was in flight. The branch now containsf7f64aceb(#1984, Remove the accuracy eval opt-out), and the opt-out is gone from the
extracted code as well as from the module:
_settle_pending_revalidation— thehelper this PR lifted that check into — rules on
accuracy_meets_flooralone,and
disable_run_evalis out of_BASELINE_FINGERPRINT_KEYS.eval_disabledhas no occurrence left in the tree. #1985, #1986, #1918 and #2021 are contained
too; none of them touches any of these files.
Module length
writeback.pygoes from 4716 lines onmainto 4156, even after adding 91 namedhelpers (78 functions on
main, 169 across the two modules now), because theGEAK group moved out.
scripts/code_metrics_baseline.txtonly loses entries andlowers values: 31 entries retired, and
module-lines .../loop/writeback.pylowered from 4716 to 4156._geak_revalidation.pyis 818 lines — over the 800-line review trigger, underthe gate's 1200. It stays one module on purpose: an earlier revision had the 2a
harness replay in a file of its own, and that split was wrong in both directions.
The replay is reached only from the 2b ruling and hands its verdict back as a
_ReplayRuling, so the revalidation called into the replay through the mixinwhile the replay imported the revalidation's decision vocabulary at module level
— a cycle with the evidence removed, which CodeQL reported as four possibly-
undefined names. Deleting the module removed the cycle, the import and a mixin
class rather than relocating any of them.
Why
size-exceptionAgainst
mainthe diff is 2,409 lines added and 2,179 deleted across 4 files,because one coherent group moved out of
writeback.pywhole and the rest wassplit in place: a moved block counts once as a deletion and once as an addition,
and the two modules together are 258 lines longer than the one they replace
(4156 + 818 = 4974, against 4716 on
main). That growth comes with the 91 named helpers that take everyfunction under the complexity ceiling; neither module reaches the 1,200-line gate.
There is one concern here — bring
writeback.pyunder the complexity ceiling —and splitting it across PRs would mean landing a module whose functions are still
over it.
Verification
cc_scan_cmetrics.py 20prints nothing forwriteback.pyor_geak_revalidation.py: no function in them reaches 20.ruff check --select C901 --config "lint.mccabe.max-complexity=20"clean onboth modules;
ruff check/ruff format --checkclean.main'swriteback.pyvalue by value, so no literal was caught by a rename.geak_resultdead store isremoved, the import cycle between the two GEAK modules is deleted with the
module, and the remaining
py/cyclic-importalerts name importsloop/writeback.pyalready makes onmain.