Skip to content

refactor(orchestrator): delete the dead and repeated code in nine over-gate specialist, policy and prompt functions - #2078

Merged
ZhengGong-amd merged 13 commits into
mainfrom
lean/shafeng/cc20-orchestrator-pr
Oct 21, 2026
Merged

ZhengGong-amd merged 13 commits into
mainfrom
lean/shafeng/cc20-orchestrator-pr

Conversation

@fengshaoyi-amd

@fengshaoyi-amd fengshaoyi-amd commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What this does

Nine functions in the orchestrator's specialist, policy and prompt code were over the code-metrics gate: the policy gate's specialist-dispatch and warm-replay checks, the specialist prompt's mandate and roofline sections, patch-root resolution, the runner's subprocess run and finalize, and the dispatch collaborator's round entry and its bookkeeping. This PR brings each one under every limit it broke, mostly by deleting what it did twice or what no producer feeds: a "Kernel roofline" prompt table built from a key nothing writes into the specialist's evidence; shape checks and re-normalisation of prompt inputs the runner already builds in those shapes; the repeat read of a warm-replay patch, with the "changed during validation" denial that only compared the reads; a duplicated hand-off to the freeform dispatch gate; three copies of a resolve-and-is_dir check; two NamedTuple bags (RunOutcome, RunHarvest) that the one caller filled with constants and copies of the SpecialistSubprocessResult it already held; the tool_violation status nothing sets; and, in the round bookkeeping, a getattr fallback for a collaborator without shared state and locals that recomputed what the round entry already holds.

Production code: +268 / −392 (net −124) across 5 files. Tests: +104 / −35. The code-metrics baseline loses 17 entries, lowers 3 module-length entries and gains none.

This continues the complexity cleanup of #2024, #2026, #2052, #2056, #2057 and #2067. The 13 commits are grouped by module and read one at a time; each refactor commit's message says what it deleted and why that is safe. Line numbers in the commit messages are from the base those commits were written against; the ones below are re-checked on this branch. One correction to dd02809's message: the warmer does not copy executive_summary out of last_trace_analyze, it parses it from the analysis file (see Why the deletions are safe); 00a49d9's message records the same correction.

Commit Module Production lines Baseline file
dd02809 delete the dead kernel roofline table and shape re-checks prompts/specialist_prompt_builder.py +7 / −47
1594d5e stop re-normalizing mandate inputs the runner already normalized prompts/specialist_prompt_builder.py +29 / −30
6b638a3 read each warm-replay patch once per validation policy/gate.py +27 / −32
901cf8b merge the duplicate freeform hand-off in specialist dispatch policy/gate.py +26 / −29
6ed626a drop the policy gate and specialist prompt baseline entries now under the gate baseline only 7 removed, 2 lowered
270b450 resolve patch roots through one resolved-dir helper specialists/patch_safety.py +17 / −40
6d5ebf3 thin the specialist round entry and its follow-up specialists/dispatch.py +37 / −44
df0c20e finalize from the subprocess result directly specialists/runner.py +126 / −166
c901e13 drop specialists entries brought under the gate baseline only 10 removed, 1 lowered
1205464 cover the reworked finalize and root-resolution edges tests only
7503fe7 drop the tool-violation status nothing produces specialists/runner.py +1 / −4 1 lowered
a45da05 lower runner.py's module-length entry to its length on main baseline only 1 lowered
00a49d9 keep the constraint comments and drop a duplicated one specialists/runner.py +0 / −2 1 lowered

All five production files are under src/hyperloom/orchestrator/.

Gate numbers per function

Measured with the gate's own collectors (ruff C901, complexipy, function lines), origin/main → this branch. Limits are 20, 30 and 80; bold values were over the limit at base.

Module Function Cyclomatic Cognitive Lines
specialist_prompt_builder.py _section_roofline_evidence 17 → 11 70 → 25 119 → 79
_section_mandate 20 → 13 49 → 27 92 → 68
gate.py PolicyGate._validate_warm_replay_targets 17 → 14 39 → 29 102 → 76
PolicyGate._validate_specialist_dispatch 12 → 5 18 → 8 91 → 64
patch_safety.py resolve_patch_apply_root 21 → 14 40 → 27 112 → 80
runner.py SpecialistRunner._finalize 22 → 6 41 → 12 163 → 79
SpecialistRunner._run_via_subprocess 12 → 2 23 → 1 173 → 79
dispatch.py SpecialistDispatchCollaborator.build_specialist_round_entry 13 → 12 35 → 29 94 → 74
SpecialistDispatchCollaborator.record_specialist_result 12 → 7 25 → 15 128 → 75

Also changed, never over at base: classify_specialist_failure 11 → 10 / 16 → 15 / 46 → 44.

New functions, cyclomatic / cognitive / lines:

  • specialist_prompt_builder.py: _mandate_run_status 7 / 9 / 21.
  • gate.py: PolicyGate._warm_replay_patch_key 4 / 5 / 20, PolicyGate._log_specialist_dispatch_coherence 6 / 10 / 22.
  • patch_safety.py: _resolved_dir 2 / 2 / 7.
  • runner.py: _subprocess_backend_error 5 / 9 / 16, SpecialistRunner._record_subprocess_run 5 / 8 / 61, SpecialistRunner._admit_done_payload 12 / 21 / 71.
  • dispatch.py: SpecialistDispatchCollaborator._follow_up_specialist_round 5 / 7 / 52.

Deleted: runner.RunOutcome, runner.RunHarvest, and the SpecialistFailureType.TOOL_VIOLATION member.

Across the 5 touched production files: C901 total 790 → 773, cognitive total 1,316 → 1,219, functions 157 → 165. Functions over a gate limit: cyclomatic 3 → 1, cognitive 9 → 3, length 20 → 11. What is left is functions this PR does not touch: SpecialistDispatchCollaborator.warm_specialist_params (cyclomatic 39, cognitive 93, 170 lines), maybe_force_stalled_domain_specialist (cognitive 33, 108 lines) and maybe_auto_retry_specialist (134 lines); PolicyGate._validate_payload_paths (cognitive 39) and PolicyGate._validate_specialist_gpu_request (82 lines); SpecialistRunner._prepare (193 lines); vet_patches (107 lines); and five builder functions that are mostly prompt text: _focus_framework_rewrite_specialist, _focus_kernel_switch_specialist, _focus_serving_specialist, _focus_static_recon_specialist and _section_output_protocol.

Module lengths: specialist_prompt_builder.py 2,597 → 2,556, gate.py 1,588 → 1,580, runner.py 1,500 → 1,455, patch_safety.py 1,175 → 1,152, dispatch.py 1,042 → 1,035.

What changes in each module

Specialist prompt builder (specialist_prompt_builder.py)

New name: _mandate_run_status — §0's read-only run status (baseline, current best, validated gain, KEEP threshold, applied stack), empty until a baseline or a KEEP threshold exists. Lifted out of _section_mandate unchanged, except that it passes inp.framework straight to format_primary_metric, which resolves "" and None to the same spec (inference_optimizer/framework_registry.py:105-108).

Removed:

  • In _section_roofline_evidence: the "Kernel roofline" table and its per-cell fallbacks, built from a kernel_roofline_top15 key that no producer writes into roofline_evidence; the isinstance checks on roofline_evidence, executive_summary, hot_kernels_top15 and each hot-kernel row, with the or {} / or [] defaults that fed them; and the str(… or "") around analysis_md_path.
  • In _section_mandate: the (scope or "domain").lower() re-normalisation, the task_description ternary, the or "" fallbacks on gap_canonical_id and task_kind, the framework or None alias, and the per-entry isinstance(dict) skip on prior_attempts.

Policy gate (gate.py)

New names:

  • PolicyGate._warm_replay_patch_key: checks a warm-replay entry's patch_path (a non-empty string, inside the session KB download root, resolvable) and returns its realpath, which keys the one read of that patch. The three checks keep their order, rules and messages.
  • PolicyGate._log_specialist_dispatch_coherence: the five log lines for a tag / scope combination that resolve_specialist_profile will re-infer, which the docstring already describes as logged rather than denied. Moved verbatim.

Removed:

  • In _validate_warm_replay_targets: the repeat read and parse of a patch that several entries name, the "changed during validation" comparison with its warm_replay_patch_target_mismatch denial, the not declared_targets or guard in front of isdisjoint (an empty set is disjoint from everything), and an or [] in front of a falsiness check.
  • In _validate_specialist_dispatch: the second call-and-return into the freeform gate, the second if not gap around the ledger backfill, and a single-use max_turns_raw local.

Patch-root resolution (patch_safety.py)

New name: _resolved_dir — a path resolved, or None when it cannot be resolved or is not a directory.

Removed from resolve_patch_apply_root: three copies of that resolve-and-is_dir check (explicit root, candidates, default root), the seen-set loop that dict.fromkeys replaces, the empty-parsed guard that an early empty-texts return makes redundant, the str(text or "") coercion of each patch text, and two inline comments that restated the docstring's create-only and no_candidate_roots paragraphs (patch_safety.py:344-358).

Specialist runner (runner.py)

New names:

  • _subprocess_backend_error: the error a reaped subprocess reports, led by the token the failure classifier keys on (subprocess_timeout, subprocess_stale_heartbeat, subprocess_error:, subprocess_exit_code:), or "" when it ended cleanly. Moved out of _run_via_subprocess unchanged.
  • SpecialistRunner._record_subprocess_run: the transcript line, the per-turn token-ledger rows, the intel calls, the conversation turn and the finished heartbeat, in that order.
  • SpecialistRunner._admit_done_payload: reconciles claimed and scanned patches, stamps each proposal with the dispatch scope and normalises its citations, and runs the patch-safety gate, in the order _finalize ran them.

Removed: RunOutcome and RunHarvest; _finalize's specialist_done_payload, outcome and harvest parameters (now one sub_result); a second loop over the proposals; the tool_violations notes and status branch, and with them SpecialistFailureType.TOOL_VIOLATION, its classify_specialist_failure branch and the NamedTuple import; and, in the ledger writes, a second computation of the latency _subprocess_call_outcome already puts on the call span, plus the list(…) / dict(…) copies of turn_usages, patches and patch_roots.

Specialist dispatch (dispatch.py)

New names:

  • _ROUND_CARRIED_KEYS: the nine optional round-row fields taken from the done payload or else the dispatch params.
  • SpecialistDispatchCollaborator._follow_up_specialist_round: what runs after a round is recorded and saved — findings harvest, static-recon consumption, research evidence, the gap attempt, the gap refresh, a requested enablement build, and the two autosubmits.

Removed: the getattr chain around shared_state.phase, the round_id and tags locals, or [] defaults in front of isinstance checks, a bool(…) around a condition, and in record_specialist_result the domain, proposals and is_empty locals that recomputed what the round entry already holds.

Why the deletions are safe

Line numbers are on this branch unless marked otherwise. Paths are under src/hyperloom/orchestrator/, except those starting with inference_optimizer/, which are under src/hyperloom/.

  • Kernel roofline table. The only writer of roofline_evidence is warm_specialist_params, which writes four keys (specialists/dispatch.py:269-274): analysis_md_path, roofline_snapshot_id and the first eight hot_kernels_top15 rows from last_trace_analyze, plus an executive_summary parsed from the analysis file (dispatch.py:256-267). It never copies kernel_roofline_top15, although last_trace_analyze carries that key (state/shared_state.py:2122), and it leaves a value already in the dispatch params alone (dispatch.py:253). The deleted table read the key at specialist_prompt_builder.py:1775 on main. No prompt, skill or doc under src/hyperloom names roofline_evidence, so no agent is asked to write one.
  • Roofline shape checks. The runner builds the field with dict(…) (specialists/runner.py:654). extract_workload_summary always returns a dict (inference_optimizer/roofline_snapshot.py:84-106), and the hot-kernel rows are the list[dict] that _build_hot_kernel_summaries builds for record_trace_analyze (state/shared_state.py:2063, :2161-2168).
  • Mandate inputs. The runner is the only production constructor of SpecialistPromptInputs (runner.py:643); SpecialistRunner.run also accepts pre-built inputs (runner.py:537), but no caller passes any. It passes profile.scope (:708), which resolve_specialist_profile lowercases and forces into SCOPE_VALUES (specialists/profile.py:128-131); gap_canonical_id and task_description as stripped strings (runner.py:582, :586, passed at :647, :711); task_kind as a string (:725); and only the dict entries of prior_attempts (:726).
  • Warm-replay repeat read. Each KB download writes a fresh mkdtemp generation directory (knowledge/remote_recipe/client.py:377-382) rather than rewriting files in place, so two reads in one validation could differ only if another process rewrote the file between them. _remote_recipe_files_root only resolves the session directory and returns None on failure (policy/gate.py:1231-1238), so resolving it per entry has no side effect. The plan's shape checks stay, because validate_dispatched_task re-validates rows read back from coordinator.db and runs this check (gate.py:326-360; called from loop/dispatcher.py:522 and loop/sub_agent_runner.py:295).
  • Freeform hand-off. normalize_dispatch_tags and the _tag_to_kb_anchor it calls only read the params and the domain catalogue (specialists/domains.py:255-276, get_domain at :316-321), so computing the tags before the freeform check has no effect a caller can see. _autofill_gap_from_ledger returns "" when nothing matches (gate.py:1021-1043), so if not gap and not self._autofill_gap_from_ledger(…) calls it and denies exactly when the old two-step check did.
  • Patch texts. Every caller of resolve_patch_apply_root passes strings: ground_patch_text passes the one patch text it already ran through is_unified_diff and patch_escapes_tree (specialists/patch_safety.py:688-705); vet_patches passes file text from read_text (:978, call at :1020-1025); integrate_patch passes str(text) and _read_patch_texts output (actions/executors/integrate_patch.py:613-614, _read_patch_texts at :500-514, call at :626-631).
  • RunOutcome / RunHarvest. Neither was exported, and on main nothing outside runner.py and its unit test named them (bypass_report.RunOutcome is an unrelated class). The one construction (runner.py:1073 on main) filled them from the sub_result in scope and always passed turns_used=1 and tool_violations=[]. _finalize's only caller is _run_via_subprocess (runner.py:984), which returns before it when there is no workspace (:926-942).
  • Ledger latency. _subprocess_call_outcome computes the same int(float(elapsed_seconds) * 1000) and leaves the key out on the same TypeError / ValueError (runner.py:108-111); _run_via_subprocess hands it over with .get, so a missing latency is still None (:983).
  • List and dict fields. SpecialistSubprocessResult declares patches and turn_usages as lists and patch_roots as a dict, all with defaults (specialists/subprocess_.py:547, :553, :578). Every producer passes those types: the reaper's early returns use the defaults (:924, :993, :1011, :1023, :1063), its full return (:1171-1185) takes patches and patch_roots from _collect_patches (:1139, typed tuple[list[str], dict[str, str]] at :1655-1662) and turn_usages from the two parsers (:1155, :1166; inference_optimizer/trace/parse_usage.py:195-197, :441-443), and the rehearsal stub builds patches as a list (rehearsal/specialist.py:147-163).
  • tool_violation. On main its only producer was _finalize's if outcome.tool_violations (runner.py:1220-1221 on main), fed by that one RunOutcome with tool_violations=[], so the status never fired. classify_specialist_failure is called only by maybe_auto_retry_specialist on the live result (dispatch.py:356), which returns before using the failure type when the attempt is not retry-eligible (:357-358); tool_violation was never retry-eligible, and a failure_type is written only past that point (:364-477). Nothing converts a stored value back into SpecialistFailureType, and no tool_violation string is left in the tree.
  • Round entry and its follow-up. build_specialist_round_entry's only production caller is record_specialist_result (dispatch.py:854), whose only caller is the dispatcher, through the Coordinator's collaborator (loop/dispatcher.py:1185-1190); a collaborator's shared_state is the Coordinator's (collaborator.py:56-58). The scorer call and the follow-up read domain, proposal_set and proposals_total from the round entry, which builds them from the same expressions the deleted locals used (dispatch.py:743-745, :759, :763-764). Nothing in between changes those fields: record_specialist_round stores a copy (state/_shared_state/phase_state.py:254) and _record_specialist_round_product only reads the entry (dispatch.py:797-833).
  • Proposal scorer. The only scorer is ProposalScorer, built by _build_proposal_scorer (inference_optimizer/cli/backends.py:186-203) and handed to the collaborator at loop/coordinator.py:322. Its score returns a dict on every path (scoring/proposal_scorer.py:411, :417-421, :448) and rebinds proposals to a filtered copy before using it (:409).

Observable effect

Nothing an operator sees changes. Specialist prompts; policy decisions on specialist dispatches and warm-replay plans, with their rules, messages and hints; specialist round rows, specialist_done.json, run results and notes; ledger rows; retry decisions; and log lines are the same for every input the repo's own callers produce.

The differences below are reachable only by input that no caller in the repo produces:

  • _section_roofline_evidence given a roofline_evidence the warmer did not write, reachable only through dispatch params that already carry one, which the warmer then leaves alone (specialists/dispatch.py:253):
    • A kernel_roofline_top15 list no longer renders a "Kernel roofline" table.
    • A non-empty executive_summary that is not a dict, or a non-empty hot_kernels_top15 that is not a list of dicts, raises while the prompt is built instead of being skipped. _prepare builds the prompt outside any try (specialists/runner.py:731) and the specialist executor does not catch it (inference_optimizer/cli/executors.py:166); SubAgentRunner.run_task records the task as failed with the exception's repr and class name (loop/sub_agent_runner.py:359-369), and the session continues.
    • A non-empty roofline_evidence that is not a dict raises as well, but only in a SpecialistPromptInputs built directly; the runner always passes a dict (runner.py:654).
  • _section_mandate given a SpecialistPromptInputs built directly: a mixed-case scope is compared as written, so "Freeform" or "Domains" gets the deliverable line and anchor of a single-domain dispatch; None in task_kind, or in task_description with scope "freeform", raises AttributeError; a non-dict prior_attempts entry raises instead of being skipped.
  • _validate_warm_replay_targets while another process rewrites a KB file: a patch that several plan entries name is read once per validation, through its realpath. If the file is rewritten between what used to be two reads, the plan is judged against the one read instead of being denied with warm_replay_patch_target_mismatch ("changed during validation"). The session KB root is resolved per entry rather than once per plan, so a session directory re-pointed during the validation is seen by the later entries.
  • resolve_patch_apply_root given a patch text that is not a string: None raises AttributeError instead of being skipped, and other values are no longer passed through str() first.
  • build_specialist_round_entry on a collaborator without shared_state: raises AttributeError instead of reading the phase as empty.
  • record_specialist_result with a proposal scorer whose score returns None: raises AttributeError instead of recording no ensemble scores.
  • A SpecialistSubprocessResult with None in a list or dict field: None turn_usages now raises TypeError while the run is recorded, where it used to give one cumulative ledger row. None patches or patch_roots used to raise TypeError on every run; now they raise only where finalize reads them, so a run without a done payload finalizes as empty, and None patch_roots with an empty patch set or a declared framework_source_root reads as empty.
  • A tool_violation runner status: classify_specialist_failure returns UNKNOWN instead of TOOL_VIOLATION. Neither is retry-eligible, so the only caller does the same with either.

Two changes are not observable at all: _resolved_dir probes is_dir before de-duplicating, so a candidate root listed twice is probed twice, with the same resolution; and the proposal scorer now receives the round entry's copy of the proposal list instead of the done payload's, which ProposalScorer.score does not mutate.

Checklist

  • Description: above.
  • Linked issue(s): none.
  • Tests:
    • Changed, assertions kept: test_build_rich_user_prompt_sections (test_specialist_prompt_builder_coverage_unit.py:132) loses its kernel_roofline_top15 fixture entry and the `k1` assertion, which pinned only the deleted table; its other assertions still run. test_build_specialist_round_entry_round_id_falls_back_to_task_id (test_specialist_lifecycle.py:264) uses the coord fixture instead of a bare Coordinator.__new__; its round_id assertion is unchanged, and what it no longer exercises is the deleted getattr fallback for a collaborator without shared_state. The _finalize helper (test_specialist_runner_helpers_unit.py:237-246) and test_a_harvested_diff_is_the_whole_round_and_the_specialists_copy_of_it_is_not_added (:249) pass a SpecialistSubprocessResult instead of the deleted bags; their assertions are unchanged.
    • Deleted: test_classify_tool_violation (test_specialist_runner_helpers_unit.py) and the tool_violation row of the classifier table in test_specialist_auto_retry.py. Both pinned only the deleted TOOL_VIOLATION member. An unrecognised status still classifies as UNKNOWN and is not retried, pinned by test_classify_unknown (test_specialist_runner_helpers_unit.py:117-118) and the weird_status row (test_specialist_auto_retry.py:44). No remaining production path carries the deleted contract, so the replaced-test rule does not stop this PR.
    • Added, one contract each: test_build_specialist_round_entry_records_run_error_and_notes (test_specialist_lifecycle.py:275: a run error sets status, error and run_error, and notes are stringified); test_explicit_root_without_patch_texts_is_accepted (test_specialist_patch_safety_unit.py:616); and in test_specialist_runner_helpers_unit.py, test_subprocess_backend_error_leading_tokens (:499), test_finalize_notes_a_recovered_partial_payload (:516), test_finalize_skips_non_dict_proposals_and_drops_scanned_patches_outside (:527) and test_record_subprocess_run_emits_one_ledger_row_per_turn (:544).
    • Not run locally on this combined branch; Linux CI on this PR is the authority.
  • Size/complexity triggers crossed:
    • 660 changed production lines is over the 400-line review trigger and under the 1,000-line limit, so no size-exception. It stays one PR because it is one concern, the over-gate functions in the orchestrator's specialist, policy and prompt code, and a specialist round runs through all of them: the policy gate admits the dispatch, the prompt builder and the runner build the prompt and finalize the result, patch-root resolution grounds the patches, and the dispatch collaborator records the round. Split, it would be two PRs on the same baseline file that need the same review context. Net −124.
    • No added or changed function crosses a gate limit. resolve_patch_apply_root sits at the 80-line limit, 38 of those lines its docstring; _section_roofline_evidence, _finalize and _run_via_subprocess are at 79.
    • Over the cyclomatic-10 trigger: _validate_warm_replay_targets (14, from 17) and resolve_patch_apply_root (14, from 21) are each one validation ladder whose branches are its denial or failure reasons; _section_mandate (13, from 20) and _section_roofline_evidence (11, from 17) are one prompt section each; build_specialist_round_entry (12, from 13) builds one row in place; _admit_done_payload (12, new) is the admission sequence _finalize carried, in the same order.
    • Over the ~60-line trigger and under 80: resolve_patch_apply_root (80), _section_roofline_evidence, _finalize and _run_via_subprocess (79), _validate_warm_replay_targets (76), record_specialist_result (75), build_specialist_round_entry (74), _admit_done_payload (71), _section_mandate (68), _validate_specialist_dispatch (64) and _record_subprocess_run (61). Each is one ladder, one prompt section, one record, or one call with its keyword arguments.
    • specialist_prompt_builder.py (2,556), gate.py (1,580) and runner.py (1,455) stay over the 1,200-line module limit they were already over; patch_safety.py (1,152) and dispatch.py (1,035) stay over the 800-line warning. Each gets shorter.
  • If this simplifies or refactors:
    • Mechanisms removed: listed per module under What changes in each module.
    • Contract preserved: every in-repo caller and all persisted data.
    • Tests: as above.
  • Observable effect: none; see the section above for why, and for the out-of-contract inputs that now behave differently.
  • Breaking changes: no. RunOutcome, RunHarvest and _finalize's parameters are private to runner.py. SpecialistFailureType is exported, and losing TOOL_VIOLATION breaks only code that names the member: nothing in the tree does after this PR, no code path on main could produce the value, and nothing converts a stored value back into the enum.
  • PR addresses single concern: yes, bringing nine over-gate functions in the orchestrator's specialist, policy and prompt code under the gate.
  • Root cause is upstream: no.

Known merge interactions

Trial merges used git merge-tree --write-tree, against this branch and against origin/main at 9da8788, with each open PR's head at the time. A conflict that also exists against main alone is not introduced by this branch.

Gate

scripts/code_metrics.py --base-ref origin/main (base 931af8a) passes on the 11 touched files: 0 new, 0 worse and 0 out-of-date entries, no new or worse duplication or dead code, and 0 findings from the comment, English-only, production-imports-test-code and repeated-literal checks and the A1 and A8–A12 design checks. Baseline: −17 entries, 3 module-length entries lowered (specialist_prompt_builder.py 2,571 → 2,556, gate.py 1,588 → 1,580, runner.py 1,500 → 1,455), +0. On main specialist_prompt_builder.py was already 26 lines over its entry (2,597 against 2,571).

The gate lists under Already over at the base SpecialistRunner._prepare at 193 lines (baseline entry 192), which this PR does not touch, and warns that patch_safety.py (1,152) and dispatch.py (1,035) are over 800 lines. ruff check and ruff format --check are clean on the touched files.

fengshaoyi-amd and others added 12 commits October 21, 2026 10:47
…-checks

_section_roofline_evidence rendered a "Kernel roofline" table from
roofline_evidence["kernel_roofline_top15"]. The only writer of
roofline_evidence, warm_specialist_params (specialists/dispatch.py:264),
copies analysis_md_path, roofline_snapshot_id, executive_summary and
hot_kernels_top15 out of last_trace_analyze and never that key, so the
table and its per-row guards could not render from any in-repo producer.
The kernel_roofline_top15 list inside last_trace_analyze is untouched.

The isinstance checks on roofline_evidence, executive_summary,
hot_kernels_top15 and its rows are deleted as well. The field is a typed
dict the runner builds with dict(...) (specialists/runner.py:660); the
warmer writes executive_summary as extract_workload_summary()'s dict and
hot_kernels_top15 as a list of the dict rows record_trace_analyze stores
(state/shared_state.py:2135, rows built at 2209). The str(... or "")
around analysis_md_path is gone because the f-string formats the value
the same way.

Only an agent-written params.roofline_evidence, which the warmer leaves
in place and no prompt asks for, can carry other shapes: a
kernel_roofline_top15 list no longer renders a table, and a non-dict
executive_summary or a hot_kernels_top15 that is not a list of dicts now
raises while the prompt is built instead of being skipped.

The coverage test's kernel_roofline_top15 fixture and its `k1` assertion
only exercised the deleted table and are removed with it.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ady normalized

_section_mandate re-derived values its producer already guarantees:

- scope: (inp.scope or "domain").lower() is gone. The runner passes
  profile.scope (specialists/runner.py:714), which
  resolve_specialist_profile lowercases into SCOPE_VALUES
  (specialists/profile.py:125-126); _section_identity already reads
  inp.scope raw. An empty scope still takes the non-freeform branch.
- task_description / gap_canonical_id / task_kind: the ternary and the
  `or ""` fallbacks are gone. They are typed str fields the runner fills
  with str(...) (runner.py:592, 653, 731), and "".split("\n")[0] is "".
- framework: the `or None` alias is gone; format_primary_metric resolves
  "" and None to the same spec (framework_registry._spec_or_default).
- prior_attempts: the per-entry isinstance(dict) skip is gone; the runner
  keeps only dict entries (runner.py:732).

The run-status block (its own header, gated on a baseline or a KEEP
threshold) moves unchanged into _mandate_run_status, which is the one
job left over once the redundant normalization is deleted.

Only a SpecialistPromptInputs built directly, never through the runner,
can tell the difference: a mixed-case scope is compared as written,
None in task_kind (or in task_description with scope "freeform") raises
AttributeError, and a non-dict prior_attempts entry raises instead of
being skipped.

Co-authored-by: Cursor <cursoragent@cursor.com>
_validate_warm_replay_targets re-read and re-parsed a patch for every
plan entry that named it, then denied the dispatch ("changed during
validation") when two reads of the same realpath disagreed. The patch is
now read once per validation, keyed by its realpath, so the comparison
and its denial are gone. KB downloads land in a fresh mkdtemp generation
directory (knowledge/remote_recipe/client.py:377-383) and nothing in the
tree rewrites a downloaded patch in place, so two reads inside one
validation could only differ under an outside writer racing the gate;
that case is now judged against the first read.

Also deleted:
- `not declared_targets or` before declared_targets.isdisjoint(...): an
  empty frozenset is disjoint from everything, so the denial is the same.
- `or []` after entry.get("resolved_patch_targets"): the next line's
  `if not raw_targets: continue` already skips None.

The patch_path checks (non-empty string, inside the session KB download
root, resolvable) move unchanged, in the same order, into
_warm_replay_patch_key. Every rule, message and raise order on
in-contract input is the same; the shape checks on the persisted plan
stay because validate_dispatched_task re-validates coordinator.db rows.

Co-authored-by: Cursor <cursoragent@cursor.com>
… dispatch

_validate_specialist_dispatch handed off to the freeform gate from two
places, once for scope='freeform' and once for a bare dispatch with no
scope and no tags, each with its own call and return. It is now one
condition. normalize_dispatch_tags only reads params and the domain
catalogue (specialists/domains.py:253-274: no logging, no mutation, and
nothing in it raises on JSON-shaped params), so computing the tags before
the freeform check changes nothing a caller can see.

The gap check tested `if not gap` twice around the ledger backfill; it is
now `if not gap and not self._autofill_gap_from_ledger(...)`, which calls
the backfill exactly when the old code did and denies exactly when it
returns no id. The single-use max_turns_raw local is inlined.

The tag / scope coherence logging, which the docstring already describes
as logged rather than denied, moves unchanged into
_log_specialist_dispatch_coherence. Every raise and log line happens in
the same order with the same text on every path.

Co-authored-by: Cursor <cursoragent@cursor.com>
…e entries now under the gate

Removes the seven code_metrics_baseline.txt entries of the four functions
this branch brought under every gate limit:

- cognitive-complexity: PolicyGate._validate_warm_replay_targets (39),
  _section_mandate (49), _section_roofline_evidence (70)
- function-lines: PolicyGate._validate_specialist_dispatch (91),
  PolicyGate._validate_warm_replay_targets (102), _section_mandate (92),
  _section_roofline_evidence (119)

and lowers the module-lines entries of policy/gate.py (1588 -> 1580) and
prompts/specialist_prompt_builder.py (2571 -> 2556) to the gate's own
count. No other entry changes, and none is added or raised.

Co-authored-by: Cursor <cursoragent@cursor.com>
…elper

Delete three copies of the resolve-plus-is_dir pattern, the seen-set loop that
dict.fromkeys replaces, the empty-parsed guard that an early empty-texts return
makes redundant, and the two comments that only restated the docstring.

Safe because every in-repo caller already passes str patch texts and directory
candidates; duplicate candidates only gain an extra is_dir stat.

Co-authored-by: Cursor <cursoragent@cursor.com>
Delete the getattr chain around shared_state.phase, the redundant or-[]
normalisation of proposal_set and notes, and the duplicate proposal_set /
domain locals in record_specialist_result. Lift the nine carried keys into
_ROUND_CARRIED_KEYS and the post-save harvest / gap / autosubmit chain into
_follow_up_specialist_round so each names one job.

Safe because Task.params is always a dict, every production caller has a
Coordinator-backed shared_state, and the one test that built a bare
Coordinator for the round_id fallback now uses the fixture that supplies one.

Co-authored-by: Cursor <cursoragent@cursor.com>
Delete RunOutcome and RunHarvest: the only production caller already held a
SpecialistSubprocessResult, and the bags only restated fields it already owns.
Decode the backend error once, admit a done payload through one method that
keeps the reconcile/stamp/vet order, and fold the ledger writes plus the
finished heartbeat into _record_subprocess_run.

Safe because every in-repo path reaches finalize after a workspace is set and
with the dataclass defaults for patches/patch_roots/turn_usages; the unit tests
that built the bags now pass a SpecialistSubprocessResult instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
The five target functions are under every limit they violated; remove their
baseline rows and lower the runner.py module-lines entry to match.

Co-authored-by: Cursor <cursoragent@cursor.com>
Pin the backend-error tokens, recovered-partial note, multi-turn ledger rows,
run_error/notes round entry fields, and the explicit-root empty-texts path so
the changed statements clear the diff-coverage gate.

Co-authored-by: Cursor <cursoragent@cursor.com>
The finalize rework removed the only place that set a run's status to
tool_violation, which fired on a RunOutcome field every caller left empty.
Delete the TOOL_VIOLATION failure type and its classify branch, and the two
test rows that fed the classifier a status the runner can no longer emit.

Co-authored-by: Cursor <cursoragent@cursor.com>
…on main

#2041 removed five lines from SpecialistRunner._prepare after the specialist
rework had lowered this entry to 1462, so on top of main the module is 1457
lines and the gate reports the entry as out of date. Written by
scripts/code_metrics.py --update-baseline, which changed only this row.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Code metrics gate: PASSED

Judged: the 11 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension Limit New Worse Baseline out of date Baseline entries vs base
Cyclomatic complexity fails > 20 0 0 0 53 -2
Cognitive complexity fails > 30 0 0 0 207 -6
Function length (lines) fails > 80 0 0 0 347 -9
Nested block depth fails > 5 0 0 0 21 +0
Parameter count fails > 5 0 0 0 95 +0
Module length (lines) warns > 800, fails > 1200 0 0 0 49 +0
Duplicated lines per file fails > 0 0 0 0 96 +0
Dead code (vulture) fails > 0 0 0 0 2 +0
Check Findings Effect
Comments 0
English only 0
Production code imports test code 0
Repeated literals 0
A1 Private name of another package 0
A8 Flag argument 0
A9 Tuple return 0
A10 Raw vocabulary comparison 0
A11 Hidden global write 0
A12 Pure forwarder 0
A12 Single-implementation abstraction 0

Module length warning: over 800 lines, within the 1200-line failure limit (not failing): 2

Module Lines Limit
src/hyperloom/orchestrator/specialists/dispatch.py 1035 warns > 800
src/hyperloom/orchestrator/specialists/patch_safety.py 1152 warns > 800

Already over at the base (backlog, not this change's): 1

Unit Dimension Base Now
src/hyperloom/orchestrator/specialists/runner.py:559 SpecialistRunner._prepare Function length (lines) 193 193
108 findings in files this change does not touch

Outside this change (informational): 108

Unit Dimension Baseline Now
src/hyperloom/orchestrator/state/shared_state.py resolve_graded_comparison Cognitive complexity absent 34
src/hyperloom_kb/collect/mapping.py _experience Cognitive complexity absent 31
src/hyperloom_kb/schema.py FieldDeclaration.__post_init__ Cognitive complexity absent 32
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Cognitive complexity absent 36
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Cognitive complexity absent 40
src/kernelforge/cli/forge_loop_command.py forge_loop Cognitive complexity absent 31
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Cognitive complexity absent 44
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Cognitive complexity absent 38
src/hyperloom/inference_optimizer/cli/__init__.py _reexport_session_environment Function length (lines) absent 111
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_agent_lineup Function length (lines) absent 89
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_run_topology Function length (lines) absent 92
src/hyperloom/inference_optimizer/cli/__init__.py _resume_session Function length (lines) absent 170
src/hyperloom/inference_optimizer/cli/__init__.py _seed_fresh_session Function length (lines) absent 164
src/hyperloom/inference_optimizer/multi_node/cli.py _restart_multinode Function length (lines) absent 107
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _build_parser Function length (lines) absent 84
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _spawn_rank_actors Function length (lines) absent 84
src/hyperloom_kb/sync.py GlobalSync.pull Function length (lines) absent 84
src/kernelforge/cli/forge_loop_command.py _RunReporter.build_result Function length (lines) absent 92
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Function length (lines) absent 136
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Function length (lines) absent 149
src/kernelforge/loop/runner.py IterationLoop._close_iteration Function length (lines) absent 117
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Function length (lines) absent 185
src/kernelforge/loop/runner.py IterationLoop._implementer_history Function length (lines) absent 112
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Function length (lines) absent 102
src/kernelforge/loop/runner.py IterationLoop._source_attempt Function length (lines) absent 175

...and 83 more; the job log lists every row.

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, ruff 0.16.2, vulture 2.16.

…cated one

Every comment the earlier commits removed was checked against the head code.
The constraints they stated are still written where the code does the job: the
freeform-lane and gap-backfill notes in PolicyGate._validate_specialist_dispatch,
the create-only and no_candidate_roots rules in resolve_patch_apply_root's
docstring, the leading-token rule in _subprocess_backend_error's docstring and
the scope stamp in _admit_done_payload. None needed restoring, and comments that
only narrated the code stay deleted. This drops the second of two consecutive
comments in SpecialistRunner._record_subprocess_run that stated the same ledger
rule, and lowers runner.py's module-length entry to 1455.

dd02809's message says warm_specialist_params copies executive_summary out of
last_trace_analyze; it parses executive_summary from the analysis file with
extract_workload_summary and copies only analysis_md_path, roofline_snapshot_id
and the hot-kernel rows.

Co-authored-by: Cursor <cursoragent@cursor.com>
@fengshaoyi-amd
fengshaoyi-amd marked this pull request as ready for review October 21, 2026 06:46
@fengshaoyi-amd
fengshaoyi-amd requested a review from a team as a code owner October 21, 2026 06:46

@ZhengGong-amd ZhengGong-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed: gate numbers re-measured (ruff C901, complexipy, function lines) and match the description; each removed guard / "producer always sets it" claim checked against its producers and call sites; base vs head differential runs show no undisclosed behaviour change; tests covering the touched modules pass.

LGTM.

@ZhengGong-amd
ZhengGong-amd merged commit 3a54819 into main Oct 21, 2026
41 of 42 checks passed
@ZhengGong-amd
ZhengGong-amd deleted the lean/shafeng/cc20-orchestrator-pr branch October 21, 2026 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants