Skip to content

refactor: one method answers a kernel request, and a delegate is assembled in named steps - #1888

Merged
chaojhou merged 10 commits into
mainfrom
fix/chenyi/kernel-reply-one-envelope
Oct 20, 2026
Merged

chaojhou merged 10 commits into
mainfrom
fix/chenyi/kernel-reply-one-envelope

Conversation

@chennyiiis

@chennyiiis chennyiiis commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor
  • Description: what and why

Three things now answer a kernel request and enqueue a delegate, where one method did each.

One reply envelope. _auto_reject collapsed three copies of the refusal envelope but left the envelope itself written three times: the refusal path, the lanes_busy deferred reply and the programmatic-handler reply all built the same {in_reply_to, kind: <kind>_done, status, result, source} message, and two of them also decided whether the outcome belonged in the failure ledger. _reply(request_msg, result, source=...) does all of it: the status comes from the result it is handed, the reply kind and the recipient from the request it answers, and a failed or errored status is logged for the FAILURE RECOVERY prompt block. _auto_reject goes.

path status source recorded
refusals (3) hard-coded failed coordinator_auto_reject always, and every result is failed
lanes_busy deferred lanes_busy no, and deferred is neither failed nor error
programmatic result.get("status", "ok") cache source, else programmatic_handler when failed or error

Resolving a kernel request moves to its own module. _handle_request built the answer inline, five levels deep: look the result up in shared state, short-circuit a kernel patch already rejected, inject the baseline throughput an integrate omitted, take the action's catalogue lanes, bracket the handler with lifecycle events, then fold what came back into shared state and the trajectory timeline. loop/kernel_request.py now owns the resolution and reports one KernelRequestOutcome (the result, the source the reply names, whether it came from shared state). _handle_request refuses what no handler will run, replies with the outcome, and hands it to _record_kernel_outcome. _lifecycle_paths moved there as lifecycle_paths; the three test modules that import it follow.

A sibling module rather than more of intent_router.py: the file was already over the module-length limit and in the baseline, so the new helpers had to leave it, not grow it. It drops from 1629 lines to 1469, and kernel_request.py is 244.

A delegate is assembled in named steps. handle_delegate validated the delegate, seeded its params per action, warmed a specialist's params, derived an idempotency key and drove the registry's create-or-reuse retry loop in one body. _seed_grid_params gives a sweep or explore delegate the params an approved proposal would have carried, _is_specialist_wave names the shape that fans out into one task per entry; _prepare_specialist_dispatch warms a specialist's params and refuses the delegate when its patch fails preflight; _create_delegated_task registers the task, stepping the idempotency key past terminated twins. A DelegateRequest carries the source, intent, action name and params those steps share, so no step takes a long parameter list.

  • Linked issue(s): close/fix refs

Review feedback on #1881.

  • Tests: added/updated? commands run?

test_coordinator_request_unknown_kind_auto_rejected and test_coordinator_request_kernel_disabled_auto_rejected are deleted as superseded, per review finding 3 on #1881. They were not quite strict subsets - the first also asserted that the refusal lists valid_kinds - so the parametrized envelope test gains a column for the result keys a refusal must carry, and that assertion comes with it. The three modules that imported _lifecycle_paths import lifecycle_paths from its new module; their assertions are unchanged.

Run locally: pytest over the 94 test modules naming the intent router, a delegate, a kernel request kind or the lifecycle paths - 3925 passed, including every envelope, lane-deferral, lifecycle and delegate-denial lock. The remaining failures are a missing optional TraceLens install and the recipe/dotenv and vendored-SDK modules, which this diff does not touch; CI runs the full suite.

  • Size/complexity triggers crossed: split, or why the shape is right:

Every unit in both files is at or under the cyclomatic ceiling of 20, and every unit this change adds is under all four gated limits.

unit cyclomatic (C901) cognitive function-lines
handle_delegate 22 to 12 42 to 16 183 to 92
_handle_request 21 to 7 84 to 8 214 to 61
_record_kernel_outcome (new) 8 12 27
_create_delegated_task (new) 6 13 63
_prepare_specialist_dispatch (new) 5 5 27
_seed_grid_params (new) 4 3 11
kernel_request.run_kernel_handler (new) 5 5 53
kernel_request.remembered_outcome (new) 4 3 40
kernel_request.close_kernel_step (new) 1 4 19
kernel_request.seed_integrate_base_tput (new) 3 4 7

scripts/code_metrics_baseline.txt loses handle_delegate's cyclomatic entry, both cognitive entries and _handle_request's function-lines entry, and lowers handle_delegate to 92 lines and the module entry to 1438. handle_delegate stays over the 80-line trigger and keeps its (lowered) baseline entry: what is left in it is the validate-then-enqueue flow itself, and cutting it further would buy helpers named for their position rather than their job. Measured with the pinned ruff 0.16.2 and complexipy 8.0.1 and the gate's own collectors.

  • If this simplifies or refactors: which mechanism was removed, which contract was preserved, and the tests that show it:

Removed: two hand-written copies of the response envelope, the second copy of the failed-outcome logging rule, and the five-level nest that stood in for naming what resolving a kernel request does. Preserved: every reply keeps its status, source, reply kind and recipient; the ledger records exactly the outcomes it recorded before; the lifecycle START/END/ERROR events, their detail strings and their artifact sets are unchanged, as are the delegate's pruned advisory, its policy denials with their rules and hints, the specialist wave fan-out and the auto-key fingerprint.

  • Observable effect:

None - refactor with nothing observable.

🤖 Generated with Claude Code

@chennyiiis
chennyiiis requested a review from a team as a code owner October 14, 2026 11:14
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Code metrics gate: PASSED

Judged: the 7 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension Limit New Worse Baseline out of date Baseline entries vs base
Cyclomatic complexity fails > 20 0 0 0 67 -2
Cognitive complexity fails > 30 0 0 0 273 -2
Function length (lines) fails > 80 0 0 0 413 -1
Nested block depth fails > 5 0 0 0 26 +0
Parameter count fails > 5 0 0 0 95 +0
Module length (lines) warns > 800, fails > 1200 0 0 0 51 +0
Duplicated lines per file fails > 0 0 0 0 97 +0
Dead code (vulture) fails > 0 0 0 0 2 +0
Check Findings Effect
Comments 0
English only 0
Production code imports test code 0
Repeated literals 0
A1 Private name of another package 0
A8 Flag argument 0
A9 Tuple return 0
A10 Raw vocabulary comparison 0
A11 Hidden global write 0
A12 Pure forwarder 0
A12 Single-implementation abstraction 0

Already over at the base (backlog, not this change's): 1

Unit Dimension Base Now
src/hyperloom/orchestrator/loop/intent_router.py:1 Module length (lines) 1629 1469
102 findings in files this change does not touch

Outside this change (informational): 102

Unit Dimension Baseline Now
src/hyperloom/orchestrator/enablement/recipe/projections.py project_launch_evidence Cognitive complexity absent 36
src/hyperloom/orchestrator/state/shared_state.py resolve_graded_comparison Cognitive complexity absent 34
src/hyperloom_kb/collect/mapping.py _experience Cognitive complexity absent 31
src/hyperloom_kb/schema.py FieldDeclaration.__post_init__ Cognitive complexity absent 32
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Cognitive complexity absent 36
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Cognitive complexity absent 40
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Cognitive complexity absent 44
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Cognitive complexity absent 38
src/hyperloom/inference_optimizer/cli/__init__.py _reexport_session_environment Function length (lines) absent 111
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_agent_lineup Function length (lines) absent 89
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_run_topology Function length (lines) absent 92
src/hyperloom/inference_optimizer/cli/__init__.py _resume_session Function length (lines) absent 170
src/hyperloom/inference_optimizer/cli/__init__.py _seed_fresh_session Function length (lines) absent 170
src/hyperloom/inference_optimizer/multi_node/cli.py _restart_multinode Function length (lines) absent 107
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _build_parser Function length (lines) absent 84
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _spawn_rank_actors Function length (lines) absent 84
src/hyperloom_kb/sync.py GlobalSync.pull Function length (lines) absent 84
src/kernelforge/cli/forge_loop_command.py _RunReporter.build_result Function length (lines) absent 91
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Function length (lines) absent 136
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Function length (lines) absent 149
src/kernelforge/loop/runner.py IterationLoop._close_iteration Function length (lines) absent 117
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Function length (lines) absent 184
src/kernelforge/loop/runner.py IterationLoop._implementer_history Function length (lines) absent 112
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Function length (lines) absent 102
src/kernelforge/loop/runner.py IterationLoop._run_supervisor_review Function length (lines) absent 88

...and 77 more; the job log lists every row.

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, ruff 0.16.2, vulture 2.16.

Three places built the same response envelope -- the reply kind, the status,
the result and the source marker -- and two of them also decided whether the
outcome belonged in the failure ledger. `_reply` does all of it: the status
comes from the result it is handed, the reply kind and the recipient from the
request it answers, and a failed or errored status is logged for the FAILURE
RECOVERY block. `_auto_reject` was that method with the status hard-coded, so
it goes.

Cyclomatic complexity of _handle_request drops from 21 to 20, which takes it
off the baseline; cognitive from 84 to 82, its length from 214 lines to 191,
and the module from 1596 to 1578. The new method reads 2.

The two auto-reject tests this supersedes are deleted; the parametrized
envelope test they are subsets of carries their assertions, including the
valid_kinds the unknown-kind refusal lists.

Refactor with nothing observable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis
chennyiiis force-pushed the fix/chenyi/kernel-reply-one-envelope branch from 06c3f64 to d714dc7 Compare October 15, 2026 02:30

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1888 -- refactor: one method answers a kernel request

What it does: IntentRouter._handle_request used to build the <kind>_done response envelope three separate ways (the _auto_reject refusal helper, the lanes_busy deferred reply, the programmatic-handler reply), and wrote the rule that sends failed or errored outcomes to the failure ledger twice. This PR replaces all of that with one IntentRouter._reply(request_msg, result, *, source). It takes the status from the result and the reply kind and recipients from the request, and records failed/error outcomes. Status, source, kind, sender, recipient and ledger behaviour are unchanged on all five reply paths, and _handle_request cyclomatic complexity drops from 21 to 20.

Blocking issues: 1

  1. [V5] The PR conflicts with main. The conflict comes from #1893, not #1889 [verified]
    Problem: mergeable: CONFLICTING. git merge-tree origin/main d714dc7f8 conflicts in src/hyperloom/orchestrator/loop/intent_router.py:1296. Main (#1893, 5c45242) replaced the line after the programmatic reply, which used to cache only successful trace_analyze results, with self._record_trace_analyze_outcome(merged_payload, result). This PR replaces the reply envelope directly above that line. #1889's target_agent = str(intent.payload["target_agent"]).strip() merges cleanly and is kept.
    Impact: The PR cannot land as is. If the hunk is resolved by taking the PR side, #1893 is reverted without any error: a failed trace_analyze is no longer remembered, so the same trace gets re-analysed.
    Action: Rebase onto main and resolve the hunk as await self._reply(request_msg, result, source=cache_hit_source or "programmatic_handler") followed by main's if kind == "trace_analyze" and cache_hit_source is None: self._record_trace_analyze_outcome(merged_payload, result). That resolution measures _handle_request at C901 = 20 with ruff 0.16.2. After the rebase, re-measure the size table in the description and the scripts/code_metrics_baseline.txt entries.

SKIPPED: V4 -- openprs.txt has # PARTIAL: lines for #1869, #1675, #1276, #1154 and #1032 (only each PR's first 100 files were compared), so overlap with those PRs was not checked.
SKIPPED: scoped pytest -- the only local interpreter (Python 3.12) has no pytest. Per-path equivalence was checked by reading base against head; CI test shards are green at d714dc7.

Checked: intent_router.py _reply, _handle_request, removed _auto_reject (base 190604f vs head d714dc7 vs origin/main 7e57b1e); test_profile_and_kernel_handlers.py test_coordinator_auto_reject_answers_in_one_envelope; test_regression_locks.py test_integrate_deferred_when_lanes_busy; tree-wide git grep at the head for _auto_reject, coordinator_auto_reject, programmatic_handler and lanes_busy across src, docs and scripts; docs/reference/kernel-execution-path.md:29 | Ran: git merge-tree --write-tree origin/main d714dc7f8; ruff 0.16.2 C901/PLR0913 on base, head, main and a simulated resolution (_handle_request 21 -> 20, and 20 after resolution) | Base: 190604f | Head: d714dc7

chennyiiis and others added 2 commits October 19, 2026 11:11
…ly-one-envelope

# Conflicts:
#	scripts/code_metrics_baseline.txt
#	src/hyperloom/orchestrator/loop/intent_router.py
@chennyiiis

Copy link
Copy Markdown
Contributor Author

Rebased onto main; the one blocking issue is resolved exactly as prescribed.

The hunk at intent_router.py:1296 now reads:

await self._reply(request_msg, result, source=cache_hit_source or "programmatic_handler")
# Cache a successful trace_analyze, or remember a failure the same trace reproduces.
if kind == "trace_analyze" and cache_hit_source is None:
    self._record_trace_analyze_outcome(merged_payload, result)

#1893 is kept: a failed trace_analyze is still remembered, so the same trace is not re-analysed. #1889's target_agent line merged cleanly and is kept.

Re-measured on the rebased tree as asked:

Value
_handle_request C901 (ruff 0.16.2) 20
scripts/code_metrics_baseline.txt current — --update-baseline reports 0 entries lowered
CI 33 pass / 0 fail

Verification: pytest across the 50 test files that reference the intent router, _handle_request or trace_analyze — 2363 passed. Three failures are a missing TraceLens install locally and fail the same way on the base commit.

@chaojhou ready for another look.

@chennyiiis
chennyiiis requested a review from chaojhou October 19, 2026 05:09
chennyiiis and others added 3 commits October 19, 2026 14:51
`IntentRouter._handle_request` built the answer to a kernel_agent request
inline: a five-level nest that looked the result up in shared state, ran the
handler under its action's lanes, bracketed it with lifecycle events, and then
folded what came back into shared state and the trajectory timeline.

`loop/kernel_request.py` now owns the resolution. It answers a request from
shared state (`remembered_outcome`) or by running its handler
(`run_kernel_handler`), and reports one `KernelRequestOutcome`: the result, the
source the reply names, and whether it came from shared state. `_handle_request`
refuses what no handler will run, replies with the outcome, and hands it to
`_record_kernel_outcome`.

`_lifecycle_paths` moved there as `lifecycle_paths`; the three test modules that
import it follow.

No observable change: internal structure only. Status, source, kind, sender,
recipient, lifecycle events and ledger behaviour are unchanged on every reply
path, including the lane deferral.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`handle_delegate` validated the delegate, seeded its params per action, warmed a
specialist's params, derived an idempotency key and drove the registry's
create-or-reuse retry loop in one body.

The per-action work now sits in steps that name it: `_seed_grid_params` gives a
sweep or explore delegate the params an approved proposal would have carried,
`_prepare_specialist_dispatch` warms a specialist's params and refuses the
delegate when its patch fails preflight, and `_create_delegated_task` registers
the task, stepping the idempotency key past terminated twins. A `DelegateRequest`
carries the source, intent, action name and params those steps share.

No observable change: internal structure only. The pruned advisory, the policy
denials and their rules and hints, the wave fan-out, the auto-key fingerprint
and the queued event are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis chennyiiis changed the title refactor: one method answers a kernel request refactor: one method answers a kernel request, and a delegate is assembled in named steps Oct 19, 2026
@chennyiiis

Copy link
Copy Markdown
Contributor Author

Rebased onto main (the conflict the review flagged is settled: mergeable is MERGEABLE, and the hunk reads exactly as prescribed - await self._reply(request_msg, result, source=cache_hit_source or "programmatic_handler") followed by main's _record_trace_analyze_outcome guard, which now lives in _record_kernel_outcome).

Two further changes since your review, both pure structure:

  • resolving a kernel request moved to loop/kernel_request.py and reports one KernelRequestOutcome. A sibling module rather than more of intent_router.py, because that file was already over the module-length limit; it drops 1629 -> 1408 lines.
  • handle_delegate is assembled in named steps (_seed_grid_params, _prepare_specialist_dispatch, _create_delegated_task), sharing a DelegateRequest.

Every unit in both files is now at or under the cyclomatic ceiling of 20: handle_delegate 22 -> 11, _handle_request 21 -> 7 (ruff 0.16.2). Cognitive 42 -> 16 and 84 -> 8; function lines 183 -> 98 and 214 -> 61. The baseline loses four entries and lowers two.

Re-requesting review.

finally:
task.cancel()
with suppress(asyncio.CancelledError):
await task
…ly-one-envelope

# Conflicts:
#	src/hyperloom/orchestrator/loop/intent_router.py
Comment thread src/hyperloom/orchestrator/loop/intent_router.py Fixed
Comment thread src/hyperloom/orchestrator/loop/intent_router.py Fixed
Comment thread src/hyperloom/orchestrator/loop/intent_router.py Fixed
Comment thread src/hyperloom/orchestrator/loop/intent_router.py Fixed
Comment thread src/hyperloom/orchestrator/loop/kernel_request.py Fixed
finally:
task.cancel()
with suppress(asyncio.CancelledError):
await task
Test and others added 2 commits October 19, 2026 16:55
… an import cycle

`phases` reaches back into `loop`, so `kernel_request` importing
`phases.machine_state` at module level put `loop.intent_router` inside a cycle
whose names CodeQL reports as possibly undefined. The interval is read where the
heartbeat uses it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
async def kernel_step_heartbeat(router: Any, kind: str, started: float):
"""Keep orchestration's bus timestamp moving through an inline step."""
# Imported here: ``phases`` reaches back into ``loop``, so taking it at module level would close an import cycle.
from ..phases.machine_state import KERNEL_HEARTBEAT_SEC
@chennyiiis

Copy link
Copy Markdown
Contributor Author

CI is green on 8dea37ea: 40 checks, 33 pass, 7 skipped, 0 fail. The code-metrics gate passes with 0 new and 0 worse on all eight dimensions and 0 findings on every design check; the baseline loses two cyclomatic, two cognitive and one function-length entry, and intent_router.py drops 1629 -> 1469 lines.

One fix since the last push: CodeQL flagged py/unsafe-cyclic-import because kernel_request took KERNEL_HEARTBEAT_SEC from phases.machine_state at module level, and phases reaches back into loop. The interval is now read where the heartbeat uses it, and CodeQL passes.

@chaojhou review is re-requested.

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1888 -- refactor: one method answers a kernel request, and a delegate is assembled in named steps

What it does: This is a pure refactor of IntentRouter's two request entry points. The three kernel-reply envelopes become one _reply. Kernel-request resolution moves to a new loop/kernel_request.py, which handles the cache and rejection short-circuits, the lanes and lease, the lifecycle bracket and the heartbeat. handle_delegate is split into _seed_grid_params, _prepare_specialist_dispatch and _create_delegated_task. ruff 0.16.2 measures C901 for handle_delegate going from 22 to 12 and for _handle_request from 21 to 7. A script that ran the base and head modules on 27 identical scenarios produced the same event traces, apart from one theoretical edge where a None status is turned into the string "None".

Blocking issues: none

Re-review of the 2026-10-17 finding: resolved. The branch now merges cleanly with main, and #1893's self._record_trace_analyze_outcome(merged_payload, result) is present after the programmatic reply and reached on the same paths.

Checked: IntentRouter._handle_request (+ _reply, _record_kernel_outcome, kernel_request.remembered_outcome / run_kernel_handler / seed_integrate_base (yes); IntentRouter.handle_delegate (+ _seed_grid_params, _is_specialist_wave, _prepare_specialist_dispatch, _create_delegated_task, DelegateReques (yes); IntentRouter._auto_reject (removed) -> _reply (yes) | Read diff, body and the round-1 review (gh api, read-only, chaojhou CHANGES_REQUESTED at d714dc7). Exported base ef93838 and head 08a61a7 with git archive to a local script,head}. Ran the equivalence driver C:/Users/chaojhou/AppData/Local/Temp/hl-cc-1888/drv/drive.py on both, giving 27 scenarios across _handle_request and handle_delegate; the only diff was the status None to 'None' edge. Measured C901 with ruff 0.16.2 (pip --target into a local script: base handle_delegate 22 and _handle_request 21; head 12 and 7; new helpers at most 8. Ran git merge-tree against origin/main: clean. Ran git log ef93838..origin/main for each PR file: only baseline commits #2021, #1984, #1986. Ran a tree-wide git grep for _auto_reject, _lifecycle_paths, _kernel_step_heartbeat, _KERNEL_HEARTBEAT_SEC, REQUEST_KIND_TO_OWNED_ACTION and intent_router logger users. Checked docs/reference | Base: ef93838 | Head: 08a61a7

@chaojhou
chaojhou merged commit e1889f0 into main Oct 20, 2026
40 checks passed
@chaojhou
chaojhou deleted the fix/chenyi/kernel-reply-one-envelope branch October 20, 2026 01:06
@ZhengGong-amd

Copy link
Copy Markdown
Collaborator

Follow-up needed: _reply is a real simplification, but kernel_request.py and DelegateRequest mostly moved code and added layers. Please open a follow-up PR that removes them.

Production code: +456/−372 (net +84), plus a new module and three NamedTuples (KernelRequest, KernelRequestOutcome, DelegateRequest). Total cyclomatic complexity rose 172→186, so the branches were spread across more units rather than removed. The follow-up should bring this change to net zero or below by deleting the mechanisms below.

Follow-up

  1. Every function in kernel_request.py takes router: Any and reaches into router._coord. run_kernel_handler also calls back router._reply when lanes are busy and returns None as an "already replied" sentinel (kernel_request.py:192-218). Make it always return an outcome (deferred included) and let _handle_request send the single _reply. Fold these functions back into IntentRouter, or into a typed CoordinatorCollaborator.
  2. DelegateRequest (intent_router.py:76) exists alongside handle_delegate's own locals (:1006). Delete it and pass the 2-5 arguments each helper needs.
  3. terminal_states at intent_router.py:886 is a hand-copied task_states.TERMINAL_STATES. Use the original. The specialist lanes/TTL rule at :894-900 is recomputed on every retry and is the third copy of specialists/dispatch.py:365-372 and GpuLanes. Hoist it out of the loop and keep one owner in GpuLanes.
  4. Delete the redundant isinstance(result, dict) guard at intent_router.py:1217.

Simplify instead of relocate

  • Keep _reply, which is the one mechanism this PR actually removed.
  • Delete the kernel_request.py free-function layer and the None sentinel rather than typing it further.
  • Delete DelegateRequest instead of adopting it everywhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants