Skip to content

refactor(prelude): read a warm-start prior into named values before PRELUDE acts on it - #1958

Merged
chaojhou merged 29 commits into
mainfrom
feature/chenyi/cc-prelude-warm-replay
Oct 22, 2026
Merged

chaojhou merged 29 commits into
mainfrom
feature/chenyi/cc-prelude-warm-replay

Conversation

@chennyiiis

@chennyiiis chennyiiis commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

What this does

Refactors warm replay inside orchestrator/phases/prelude.py: kernel preparation and rollback, replay configuration and enqueueing, throughput/accuracy grading, and promotion are separate named stages. The history-row traversal, donor-field list, combined rollback, and kernel staging paths are shared by their callers. No module is added or removed.

Relative to main bb2da63332dcbc46bdf10702c516ef43eb80a016, prelude.py changes by +843/-1006 (163 fewer production lines). Including 119 added test lines and the metrics baseline, the complete PR is 55 lines smaller. AST-counted docstring source spans remain 138 lines on each side.

The existing size-exception covers 1849 changed production lines, counting additions plus deletions. Enqueue, rollback, grading and promotion exchange the same pending replay record, so the extracted stages and their callers change together. The remaining over-threshold plan/config readers retain their single parsing responsibilities and lower their recorded debt; moving them into sibling modules would not remove that responsibility or simplify the replay lifecycle.

Observable effect

The normal replay contract is preserved: a replay must pass the throughput and applicable quality/accuracy gates before promotion; a failed required replay restores both recipe and kernel changes. On a failed replay, combined rollback still runs before the failure's error_class is stamped or the warm-replay failed log is emitted. If rollback fails, its failure outcome is preserved without adding that replay error field or log.

Numeric parsing now uses hyperloom.common.coerce.to_float for replay confidence, expected gains, throughput and audit measurements. Invalid, boolean and non-finite inputs use the call site's fallback rather than being admitted as numbers or propagating a float-conversion exception. For example, non-finite replay throughput falls back to zero and is rejected by the throughput gate; an invalid donor expected gain falls back to zero. A negative warm-up throughput is now retained in the audit-only cold_tput field; main previously recorded None for a non-positive reading. The replay is still graded on its hot throughput.

Complexity and validation

Measured with ruff 0.16.2 and complexipy 8.0.1 against the main commit above; lines are physical spans from def through the last statement:

Method Cyclomatic Cognitive Lines
maybe_enqueue_warm_replay 49 → 10 191 → 14 497 → 53
_settle_warm_replay 43 → 5 113 → 9 389 → 31

The baseline diff removes 11 entries, lowers 5, and adds none. Remaining recorded debt includes cognitive complexity for _collect_warm_kernel_plan 42→33, _warm_kernel_extra_envs 42→34, and _merge_named_current_recipe_configs 39→32; _judge_warm_replay_accuracy function-lines 123→92; module-lines 2640→2529. This does not claim every helper is below every review threshold.

Validation: 139 tests passed in test_prelude_warm_kernel_kb.py and test_warm_replay.py under Linux/Python 3.10. Ruff lint and format checks passed. The head already contains the main commit named above.

…tle_warm_replay

_settle_warm_replay held two settlements in one body: the drift rejection for a
replay that did not reproduce its prior, and -- inside a 206-line branch -- every
path a reproduced replay can take. Lift the second one out and name four jobs
inside it: the checkout that would not promote, the patches that actually landed,
the audit metadata a replayed stack entry carries, and the journal append.

_settle_warm_replay: 390 -> 197 lines, cognitive 113 -> 48, cyclomatic 43 -> 25

_ReplayMeasurement replaces six loose throughput locals that the settlement
threaded by hand; hot_tput, single_round_tput and tput were the same value by
the time any of them was read.

Observable effect: none -- pure refactor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis
chennyiiis requested a review from a team as a code owner October 17, 2026 06:37
@chennyiiis chennyiiis added the baseline-raise Waives new, worse and baseline-growth code-metrics findings; reason required in the PR description label Oct 17, 2026
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Code metrics gate: PASSED

Judged: the 4 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension Limit New Worse Baseline out of date Baseline entries vs base
Cyclomatic complexity fails > 20 0 0 0 42 -3
Cognitive complexity fails > 30 0 0 0 165 -5
Function length (lines) fails > 80 0 0 0 312 -3
Nested block depth fails > 5 0 0 0 19 +0
Parameter count fails > 5 0 0 0 89 +0
Module length (lines) warns > 800, fails > 1200 0 0 0 47 +0
Duplicated lines per file fails > 0 0 0 0 96 +0
Dead code (vulture) fails > 0 0 0 0 2 +0
Check Findings Effect
Comments 0
English only 0
Production code imports test code 0
Repeated literals 0
A1 Private name of another package 0
A8 Flag argument 0
A9 Tuple return 0
A10 Raw vocabulary comparison 0
A11 Hidden global write 0
A12 Pure forwarder 0
A12 Single-implementation abstraction 0
85 findings in files this change does not touch

Outside this change (informational): 85

Unit Dimension Baseline Now
src/hyperloom/orchestrator/state/shared_state.py resolve_graded_comparison Cognitive complexity absent 34
src/hyperloom_kb/collect/mapping.py _experience Cognitive complexity absent 31
src/kernelforge/cli/forge_loop_command.py forge_loop Cognitive complexity absent 31
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Cognitive complexity absent 44
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Cognitive complexity absent 38
src/hyperloom/inference_optimizer/cli/__init__.py _reexport_session_environment Function length (lines) absent 107
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_agent_lineup Function length (lines) absent 89
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_run_topology Function length (lines) absent 92
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _build_parser Function length (lines) absent 84
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _spawn_rank_actors Function length (lines) absent 84
src/hyperloom_kb/sync.py GlobalSync.pull Function length (lines) absent 84
src/kernelforge/loop/runner.py IterationLoop._close_iteration Function length (lines) absent 117
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Function length (lines) absent 185
src/kernelforge/loop/runner.py IterationLoop._implementer_history Function length (lines) absent 112
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Function length (lines) absent 102
src/kernelforge/loop/runner.py IterationLoop._source_attempt Function length (lines) absent 175
src/hyperloom_kb/http_service.py Module length (lines) absent 1525
src/kernelforge/cli/forge_loop_command.py Module length (lines) absent 2003
src/hyperloom/orchestrator/actions/executors/_geak_sweep.py sweep_via_geak Cognitive complexity 67 70
src/hyperloom/orchestrator/actions/executors/_grid_runner.py run_grid Cognitive complexity 177 180
src/hyperloom/orchestrator/actions/executors/explore.py ExploreExecutor.__call__ Cognitive complexity 369 383
src/hyperloom/orchestrator/kernel/request_handlers.py _run_forge_gemm_tuning Cognitive complexity 124 125
src/hyperloom/orchestrator/phases/kernel.py KernelPhase._run_geak_kernel_phase Cognitive complexity 122 130
src/hyperloom/orchestrator/actions/executors/_grid_runner.py run_grid Cyclomatic complexity 55 56
src/hyperloom/orchestrator/actions/executors/explore.py ExploreExecutor.__call__ Cyclomatic complexity 76 83

...and 60 more; the job log lists every row.

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, ruff 0.16.2, vulture 2.16.

chennyiiis and others added 6 commits October 19, 2026 11:38
…RELUDE acts on it

Every unit in prelude.py now measures within the complexity ceiling. Two of them
carried most of the weight: maybe_enqueue_warm_replay (497 lines, cyclomatic 48)
derived a replay request and refused it at eleven gates in one body, and
_prepare_warm_kernel_kb (269 lines, cyclomatic 33) staged, landed, settled and
persisted the kernel half in another.

The derivation moves to a sibling module. _warm_prior.py reads the records a
prior arrives as -- its history, its kernel section, a donor's recommendation --
into values PRELUDE can act on: a launch overlay and where it came from, the
kernel plan and the overlay it brings, the gain anchor, the checkouts an overlay
was measured on. It reads no session state and writes none, so what remains in
PreludePhase is the state the replay moves through and the refusals it publishes.

Four mechanisms are gone rather than relocated:

* the kernel unwind that four branches of the enqueue path spelled out
  separately is one _unwind_warm_kernel_half;
* the staging loop that _preview_current_kernel_config and
  _prepare_warm_kernel_kb each carried is one _staged_warm_kernel_plan;
* the donor field list that maybe_enqueue_warm_replay restated beside
  _WARM_REPLAY_DONOR_FIELDS is read from the constant;
* warm_recipe_proven_items and inject_warm_recipe_history_into_ledger both
  walked recipe -> attrs -> exact_history by hand; both now ask
  warm_recipe_history_rows.

_WarmReplaySettlement carries the four values every branch of a settlement
threaded by hand (the result, the task, the outcome it writes and the recorder it
rules on), which is what leaves room under the parameter limit for the gates to
be named.

Observable effect: none -- pure refactor. No behaviour, interface, default, flag
or artifact changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…eplay units

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis chennyiiis removed the baseline-raise Waives new, worse and baseline-growth code-metrics findings; reason required in the PR description label Oct 19, 2026
@chennyiiis chennyiiis changed the title refactor(prelude): split the reproduced-replay settlement out of _settle_warm_replay refactor(prelude): read a warm-start prior into named values before PRELUDE acts on it Oct 19, 2026
@chennyiiis

Copy link
Copy Markdown
Contributor Author

Updated: every function in prelude.py now measures at or below the Cmetrics ceiling of 20 (previously ten were over, the worst at 169), the baseline-raise label is removed, and the fifteen prelude.py baseline entries are gone.

The derivation half of maybe_enqueue_warm_replay moved to a new sibling module phases/_warm_prior.py, which keeps prelude.py at 2 633 lines rather than growing it past the 2 692 it had on main.

@AMD-AGI/Hyperloom#1962 is stacked on this branch and rewrites maybe_enqueue_warm_replay too, so it will need a git merge of feature/chenyi/cc-prelude-warm-replay after this lands; the 169 -> 96 step it takes is already past here, so the merge should resolve in favour of this branch for that method.

@chennyiiis chennyiiis added the size-exception Waives the PR diff budget; added by a maintainer with a reason in the PR description label Oct 19, 2026
Test and others added 3 commits October 19, 2026 16:47
… preparation does

Two refusals on the enqueue path had no test of their own: which record a
replay's kernel launch overlay is read from, and what a kernel preparation that
failed with a patched tree still standing does about it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…lude-warm-replay

# Conflicts:
#	scripts/code_metrics_baseline.txt
#	src/hyperloom/orchestrator/phases/prelude.py

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1958 -- refactor(prelude): read a warm-start prior into named values before PRELUDE acts on it

What it does: PR #1958 splits the ten over-ceiling functions in phases/prelude.py into named helpers. The warm-prior parsing moves into a new sibling module, phases/_warm_prior.py, and the 15 prelude baseline entries are dropped. Route: extract-function refactor plus four de-duplications (kernel unwind, staging loop, donor field list, history-row walk). Behaviour is equivalent on every path I drove with base-vs-head differential scripts, except one narrow failure path in _settle_warm_replay, which now persists and logs error_class even when the rollback itself fails.

Blocking issues: 1

  1. [settle-failed-error-class-before-rollback] A failed replay whose rollback also fails now persists error_class and logs 'warm-replay failed', contradicting 'Observable effect: None' [verified]
    Problem: In base, the status!=succeeded branch ran _require_combined_warm_rollback first and only stamped outcome['error_class'] and logged 'warm-replay failed (...)' once the rollback succeeded. Head stamps error_class and logs at lines 1963-1964 before _fail_warm_replay runs the rollback. When the rollback fails, the rollback_failed outcome that _require_combined_warm_rollback persists now carries error_class, and the INFO line is emitted. src/hyperloom/orchestrator/phases/prelude.py:1963
    Impact: Several operator-visible artifacts change: state.warm_replay_outcome gains error_class, and the warm-replay breakdown event verdict.error_class (warm_replay_event._verdict reads settled['error_class']) goes from '' to 'Boom'. A new INFO log line also appears. The PR states no observable change. The new behaviour is arguably more informative, but it is undisclosed in a PR billed as a pure refactor.
    Action: Either restore the base order: run the rollback first, then stamp error_class and log (e.g. check _require_combined_warm_rollback in the branch before setting error_class), or state the change in the PR's Observable effect section.

Checked: maybe_enqueue_warm_replay (+ _read_replayable_current_recipe, _prepared_warm_kernel_section, _refuse_dirty_kernel_prepar (yes); _prepare_warm_kernel_kb (+ _staged_warm_kernel_plan, _settle_empty_warm_kernel_plan, _record_warm_kernel_progress, _appl (yes); _settle_warm_replay (+ _warm_replay_measurement, _record_tput_gate, _record_replay_measurement, _fail_warm_replay, _judg (no); _judge_warm_replay_accuracy (+ _read_replay_accuracy, _admit_unjudged_replay_accuracy, _record_accuracy_gate) (yes); _resolve_promoted_recipe_checkout / rollback_combined_warm (+ _replayed_patch_trees, _promotable_tree, _restore_replayed (yes); _merge_named_current_recipe_configs -> _warm_prior.merge_named_current_recipe_configs (yes); _warm_kernel_extra_envs (yes); inject_warm_recipe_history_into_ledger / warm_recipe_proven_items (+ warm_recipe_history_rows, warm_recipe_rejection, _r (yes); _collect_warm_kernel_plan (+ warm_kernel_plan_entry) (yes); _preview_current_kernel_config / _read_current_recipe_replay (yes) | Conflict with main none | Base: ef93838 | Head: 2b05075

…lude-warm-replay

# Conflicts:
#	scripts/code_metrics_baseline.txt
@ZhengGong-amd

Copy link
Copy Markdown
Collaborator

Requesting changes: the split makes the complexity numbers pass, but grows production code by ~400 lines and changes one failure path despite "Observable effect: None".

Production code: +1780/−1383 (net +397), plus 13 new NamedTuples (base had 0) and prelude.py going from 50 to 97 functions. A complexity refactor should shrink production code. Bring this to net-zero or negative by deleting mechanisms (below), not by moving them into more helpers and tuples.

Blocking

  1. Rebase onto main: scripts/code_metrics_baseline.txt conflicts.
  2. The failure settlement now writes outcome["error_class"] and logs "warm-replay failed" before rollback (src/hyperloom/orchestrator/phases/prelude.py:1963). Base did both only after rollback succeeded. Repro: rollback returns {"ok": False} with result {"status":"failed","error_class":"Boom"}. Base persists error_class=None and no log line; head persists error_class=Boom and logs an extra INFO line. warm_replay_outcome is persisted and read by breakdown. Move both into _fail_warm_replay after _require_combined_warm_rollback returns True, and add a test for the rollback-failed case.

Simplify instead of relocate

  • Delete _WarmReplaySettlement (prelude.py:239). It only bundles the old 4 parameters to get under the 5-parameter limit. Give each gate only what it uses. Also drop _KernelHalf/_PromotableTree/_ReplayMeasurement/_AccuracyReading and the _warm_prior tuples that only pass between one pair of functions (StagedKernelPlan, WarmKernelApply, RecordedRoots).
  • Delete _warm_prior.as_float (_warm_prior.py:35) and use hyperloom.common.coerce.to_float(value, 0.0).
  • Pass an explicit current_remote: bool instead of checking whether the sdk_replay dict is empty (prelude.py:990, :1283).
  • Keep _warm_prior.py as the pure-derivation boundary, but have it return 1–2 domain types, not 8.

Tests: the 6 new tests call private helpers, and _StubPrelude binds 10 private methods, so these tests pin the method layout. Assert the same behaviour through maybe_enqueue_warm_replay with _make_coord.

Metrics: rollback_combined_warm (7/27/77) and _resolve_promoted_recipe_checkout (7/22/61) were never over the thresholds (cyclomatic 20 / cognitive 30 / 80 lines). Leave them alone.

Test and others added 4 commits October 20, 2026 10:48
…orted, after its trees come back

The extraction had hoisted the error class and the line that reports it
above the rollback, so a settlement that ended as rollback_failed carried
a class and printed a line the same settlement never did before. Both now
sit behind the rollback guard, where the original kept them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…onical coercion

``common.coerce.to_float`` answers 0.0 for a non-finite value, so swapping it
in here would read an ``inf`` throughput as no throughput at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tore errors and nothing else

Drives promote_warm_replay with a rollback that refuses: the settled outcome
is rollback_failed carrying the restore errors, it carries no error_class, and
no "warm-replay failed" line is logged. An extraction that lifts the class and
the log above the rollback passes every other warm-replay test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chennyiiis

Copy link
Copy Markdown
Contributor Author

@ZhengGong-amd @chaojhou thanks — both blocking findings are fixed. Point by point.

Fixed

1. Rebase / baseline conflict. origin/main is merged in (merged rather than rebased: this branch is not force-pushed). scripts/code_metrics_baseline.txt is resolved, the PR is MERGEABLE, and the code-metrics gate reports 0 new, 0 worse, 0 out of date.

2. error_class and the log line before the rollback. You are both right, and the repro is exact. Restored to the base order in _fail_warm_replay_on_task_error (prelude.py:2059): the rollback runs first, and only a settlement that got its trees back stamps error_class, writes the reason and logs. A settlement that did not is a rollback_failed outcome whose account is the restore errors and nothing else, as on main.

Test added as asked — test_a_failed_replay_whose_rollback_fails_reports_only_the_rollback. It drives the real entry point promote_warm_replay with a rollback that refuses, and pins all three facts: status == "rollback_failed", "error_class" not in outcome, and no warm-replay failed line in caplog. It passes on main and on head and fails on the revision you flagged, which is the point: it pins a contract an extraction can move silently.

I also re-ran the check mechanically rather than by eye: the set and count of every log.<level> call (attribute, message literal, occurrences) is now identical between origin/main's prelude.py and head's prelude.py + _warm_prior.py, and main's string-constant set is a subset of head's across the two files. Both comparisons are in the description.

3. Lead with the deletions. Done — the description now opens with the four mechanisms this removes rather than relocates, and states the net line count with the reason.

Where I disagree, with the reason

Net +398 production lines. Agreed this is the wrong direction, and I do not think it can reach net-zero without declining to split the two functions at all. The deletions pay for part of it; what they cannot pay for is that eleven refusal paths which were if ...: self._skip_warm_replay(...); return None inline now each carry a signature, a docstring and a named return. prelude.py itself is -62, and _warm_prior.py imports nothing from prelude.py, so there is no module-level cycle (the #1956 CodeQL shape).

The NamedTuples. Eight of the thirteen carry 3 or more fields (WarmReplayRequest 8, WarmReplayConfig 7, _ReplayMeasurement 5, WarmKernelOverlay 5, _WarmReplaySettlement 4, StagedKernelPlan 4, WarmKernelApply 4, _AccuracyReading 3). Removing those means returning a 3-to-8-element tuple, which design check A9 fails (fails > 2) — the gate enforces it on this diff. The five two-field ones (_KernelHalf, _PromotableTree, WarmLaunch, WarmPatchPlan, RecordedRoots) would pass A9 as bare tuples, but AGENTS.md's code-shape bullet asks the opposite: "A result is one named type: no positional tuple of loosely related values." _WarmReplaySettlement specifically: giving each gate only what it uses is what the base did, and it is why _settle_warm_replay threaded four values by hand through fourteen branches; the five-parameter limit is a symptom there, not the reason.

as_float -> common.coerce.to_float(value, 0.0). This one is not equivalent, and swapping it is an observable change in a PR that states none. to_float returns the default for non-finite values and for bool: to_float(inf, 0.0) is 0.0 where as_float(inf) is inf. Every site as_float replaces was, on main, an inline try: float(x or 0.0) except: 0.0 — as_float's semantics — and three of them are throughput and gain values that feed the gates comparing them, so an inf throughput would start reading as no throughput at all. The byte-identical copy in state/_shared_state/warm_start.py is behind an underscore package that phases/ may not import from (A1). I have pinned the reason in the docstring at _warm_prior.py:35 so the next reader does not make the swap silently. Happy to do it as its own PR with the effect stated.

Explicit current_remote: bool instead of the sdk_replay emptiness check. bool(sdk_replay) is current_remote, not an approximation of it: _read_current_recipe_replay() either raises or returns a nine-key dict, and it is called only under _is_current_remote_recipe(warm) (prelude.py:990); the value is {} in exactly the other case. A current_remote parameter would therefore be a second copy of a decision the value already carries ("duplicated state or a duplicated decision is the same problem"), and a boolean parameter selecting between two bodies is what design check A8 fails. I checked the one place this could have drifted — sdk_replay.get("kernel_config") or kernel at prelude.py:1150 — and _preview_current_kernel_config always returns a two-key dict, so the fallback is unreachable when the prior is current-remote. The two new tests at the end of test_warm_replay.py pin that seam in both directions.

_warm_prior.py returning 1-2 domain types. The eight it returns answer eight different questions a caller asks separately (the launch config, the kernel overlay, the patch plan, the recorded roots, the gain anchor, the donor fields, the staged plan, the apply result). Collapsing them into one or two means a type whose optional fields stand for states, which is the shape the same AGENTS.md bullet rules out.

Tests pinning the method layout. The entry point under test is unchanged: test_prelude_warm_kernel_kb.py still drives stub._prepare_warm_kernel_kb(...), exactly as on main. The ten added _StubPrelude bindings are the mechanical consequence of that entry point now delegating — the stub binds methods explicitly precisely so it does not need PreludePhase's real constructor, so a split shows up there as bindings rather than as new assertions. No assertion changed. Of the new tests, the one you asked for drives promote_warm_replay; the other five cover two seams the split created (which overlay a current-remote recipe launches under, and whether a dirty kernel preparation stops the run) and carry rules of their own rather than threading a flow, which is the carve-out in AGENTS.md's testing bullet.

rollback_combined_warm (7/27/77) and _resolve_promoted_recipe_checkout (7/22/61) were never over. Correct on the gate's own three numbers, and I would leave them alone on that basis. The bar applied across this set of PRs is the stricter Cmetrics scale, which adds each and/or operand past the first, each ternary, and each comprehension for/if clause on top of C901; on it they measured 31 and 22. They are at 11 and 7 now with no behaviour change — say the word and I will revert both to main's bodies.

@chennyiiis

Copy link
Copy Markdown
Contributor Author

PR #1958 -- refactor(prelude): read a warm-start prior into named values before PRELUDE acts on it

Self-review, run with .claude/skills/review-pr at head 02fdaef3d. Posting the card without an LGTM: I am the author, and @chaojhou's CHANGES_REQUESTED is the one that closes.

What it does: brings the ten over-ceiling units in phases/prelude.py under the complexity ceiling, by deleting four duplicated mechanisms (the kernel unwind spelled out in four branches, the staging loop carried twice, the restated donor-field list, the hand-walked recipe -> attrs -> exact_history) and splitting what remained. The prior-derivation half moves to a new sibling module phases/_warm_prior.py, which reads records only and imports nothing back.

Blocking issues: none at this head.

The one that fired is fixed and is described above: [X2] the description claimed no observable change while outcome["error_class"] and a warm-replay failed log line had been hoisted above the rollback guard (prelude.py:1963 at the time). Restored in _fail_warm_replay_on_task_error and pinned by test_a_failed_replay_whose_rollback_fails_reports_only_the_rollback.

Four further candidates were raised and refuted:

  • [D2] as_float (_warm_prior.py:35) duplicates common/coerce.py's canonical to_float. Dropped: to_float returns its default for non-finite values, so adopting it reads an inf throughput as no throughput — an observable change. Pinned in the docstring instead.
  • [free:lost-writes] nine outcome/pending/recorder key names have a lower write count than on main (status, reason, decision, apply_result, recipe_row, record_gate, record_measurement, warm_history_injected, warm_replay_pending). Dropped: each was opened and traced to a consolidation that still writes the key on the same path — recipe_row moved to a dict literal at _warm_prior.py:127, record_measurement is one helper called twice at prelude.py:1971 and :1978, GATE_ACCURACY's four branch-local calls are one helper covering the same four paths with the same passed/reason/observed/threshold.
  • [free:accuracy-observed-type] an integer accuracy is now recorded on the timeline as 1.0 rather than 1. Dropped: the persisted outcome value is float on both sides and nothing reads the field as an int.
  • [free:current-remote] bool(sdk_replay) replaced main's explicit current_remote local. Dropped: _read_current_recipe_replay() raises or returns a nine-key dict and is called only under _is_current_remote_recipe, and _preview_current_kernel_config always returns a two-key dict, so the or kernel fallback at prelude.py:1150 is unreachable when the prior is current-remote.

SKIPPED: Step 7 independent reader -- no second agent was available, so the refutation pass was run by the same agent that wrote the findings. What it rests on is mechanical and reproducible from the Ran: line.

Checked: prelude.py vs origin/main for every log.<level> call (attribute, message literal, count -- identical), every string constant (main's set is a subset of head's across both files), every string-subscript write, state attribute write and recorder.* call by name; _judge_warm_replay_accuracy / _record_accuracy_gate against main's four accuracy branches; inject_warm_recipe_history_into_ledger / warm_recipe_history_rows / warm_recipe_rejection against main's inline walk; _warm_kernel_overlay / _combined_warm_launch against main's current_remote branches; _warm_prior.py import list for cycles; scripts/code_metrics.py excuse_base_backlog / tightened for the module-lines ratchet | Ran: cc_scan_cmetrics.py 20 (nothing for either file), ruff check, ruff format --check, vulture --min-confidence 80, gh pr checks (40 checks, 0 failures) | Base: adbc1879e4601d6ee6f1b09f65f78bc4bf82aa7b | Head: 02fdaef3d

@chennyiiis

Copy link
Copy Markdown
Contributor Author

PR #1958 -- Step 7 independent reader, run

The card at head 02fdaef3d carried SKIPPED: Step 7 independent reader. That gap is now
closed: a second reader, given the diff, base.txt and the card with every finding false
until defended, re-adjudicated this PR at the same head. No finding survives. Blocking
issues: none.

The reader was aimed at the two defect classes this refactor effort has actually produced.

A statement that ended up on the other side of the check protecting it. Base and head
prelude.py were parsed with ast and partitioned: 31 functions unparse byte-identically,
4 are pure renames, 15 genuinely changed. All 15 were read side by side against
adbc1879:phases/prelude.py for statement order and dominating conditions. baseline_accuracy
is still read before _read_replay_accuracy (prelude.py:1607-1608); the first
record_measurement still precedes the quality gate and the second still follows the accuracy
gate (:1971, :1978); _require_combined_warm_rollback is still the first statement of
_fail_warm_replay_on_task_error (:2061); the per-tree rollback guard order in
_restore_replayed_patch_tree (:175-196) is unchanged; the early framework-root refusal
still runs before _prepare_warm_kernel_kb and the late one after it. Two near-misses were
chased and killed: _warm_kernel_overlay's sdk_replay.get("kernel_config") or kernel
fallback (:1128) is unreachable because _preview_current_kernel_config always returns a
two-key dict, and _staged_warm_kernel_plan's not any(targets) to not targets tightening
coincides because _resolve_kernel_target_paths appends str(Path) and never an empty string.

A shared-helper swap that changes the domain. Absent outright. The PR introduces zero
hyperloom.common.coerce call sites; the only match in the 182 KB diff is the docstring at
_warm_prior.py:38 explaining why as_float is not to_float. as_float is
float(value or 0.0) under a TypeError/ValueError guard -- main's exact local domain at
all 13 call sites, and strictly wider than the two bare float() reads it absorbed
(baseline_tput, expected_gain_pct).

The "Observable effect: None" claim was re-tested mechanically rather than read: the multiset
of log.<level>(<literal>) calls is identical in both directions, and no string constant of
main's is missing at head -- so no changed log text, no new error_class, no new reason string.

Checked: phases/prelude.py, phases/_warm_prior.py, hyperloom/common/coerce.py | Ran: per-function ast equality base vs head; log-literal multiset diff both directions; git show adbc1879:... for all 15 changed functions | Base: adbc1879e4601d6ee6f1b09f65f78bc4bf82aa7b | Head: 02fdaef3d | CI: 40 checks, 0 failures

@ZhengGong-amd

Copy link
Copy Markdown
Collaborator

Production code is still too large. The net growth is +398 lines (it was +397), and it needs to keep shrinking before this can merge.

I classified every changed line in the touched production files with an AST pass (merge-base vs head 08b2a321b):

net lines
executable logic -114
docstrings +183
def/class signatures +184
NamedTuple/dataclass fields +50
blank lines between new defs +136
imports / __all__ +31

Functions go from 50 to 123, record types from 0 to 13, and total parameters from 80 to 215. Of the 90 new definitions, 47 are referenced exactly once in src/.

The logic barely changed (-114). The growth is the scaffolding around single-use units: their signatures, docstrings, fields and spacing (about +584 lines). That is the same control flow in more places, not a simpler one (AGENTS.md Simplify by removing a mechanism). The parameter-count and tuple-return limits in [tool.hyperloom.code_metrics] are review triggers, not a reason to add a type or a helper. When a limit and the clearer code disagree, keep the clearer code and say so (Size is a design signal).

Cut next:

  • _WarmReplaySettlement (prelude.py:239): it only exists to pass four parameters under the parameter limit.
  • The 5 NamedTuples in prelude.py and 8 in _warm_prior.py that move values between one producer and one consumer. Have _warm_prior return 1-2 domain types.
  • current_remote encoded as if sdk_replay: (_warm_prior.py:359,415, prelude.py:1278): pass an explicit bool.
  • Inline the remaining helpers that have one caller and only thread arguments through, and drop the docstrings that restate a private helper's name (Comment below the local average).

Target: production net ≤ 0, or name each remaining new type in the description with what it removes.

…lude-warm-replay

# Conflicts:
#	scripts/code_metrics_baseline.txt
@chennyiiis chennyiiis closed this Oct 20, 2026
@chennyiiis chennyiiis reopened this Oct 20, 2026
Test and others added 6 commits October 20, 2026 18:48
…lude-warm-replay

# Conflicts:
#	scripts/code_metrics_baseline.txt
Review asked why these PRs add so many docstrings. The files already document
96-98% of their functions, so the growth is the function count the complexity
ceiling forced, not a higher documentation rate. Every new helper keeps a
docstring; the Args:/Returns: blocks on them go.

Mechanical, via collapse_docstrings.py --added-vs origin/main: only functions
this branch adds are touched, pre-existing functions, module and class
docstrings are left alone. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…pse improved

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…lude-warm-replay

Only conflict was scripts/code_metrics_baseline.txt: took the lower value
per entry, kept this branch's retirement of the PreludePhase warm-replay
entries, and took main's lowered dispatch/parser/runner numbers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Test and others added 6 commits October 21, 2026 15:15
… sibling module

Rework the warm-replay refactor delete-first. `phases/_warm_prior.py` is gone and
its contents are back in `prelude.py`; every NamedTuple the split introduced is
gone with it, and the functions that were under every repo threshold are back at
their merge-base bodies.

What this removes rather than relocates:

- the `recipe -> attrs -> exact_history` walk, spelled out twice;
- the primary-tree fallback, spelled out twice;
- the kernel-half unwind and its pending restatement, spelled out four times;
- the preparation rollback and its `ROLLED_BACK` marking, spelled out twice;
- the `deferred` / `errors` counters, which restate the plan's own decisions;
- the three local copies of `try: float(x) except: 0.0`, now `common.coerce.to_float`;
- the `order` list beside `pairs`, the `prepared`/`decision_params` duplicate reads,
  the `current_remote` branch on a call whose default already matched, and a dozen
  guards whose producers are typed.

Observable effects:

- `to_float` rejects `bool` and non-finite values where the old inline coercions
  did not: a replay reporting `inf` or `True` for `output_throughput`,
  `warmup_round_tput`, `baseline_tput_anchor`, a warm-recipe `confidence` or a
  session `gain_pct` now reads as `0.0`. All five are benchmark readings where a
  non-finite value is not a measurement.
- `_judge_warm_replay_accuracy` no longer swallows a non-numeric
  `shared_state.baseline_accuracy`; the field is typed `float`.

Nothing else changes: the string constants and the `log` calls are identical to
the merge base, and no outcome, pending or recorder key lost a writer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t seam

Restore the no-argument preparation call a local prior takes, and name the
uplift the replay gates grade on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…names

The thirty-one one-line docstrings on the warm-replay helpers repeat the name
above them and nothing else. Delete them and lower the module-lines baseline
they inflated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolve the scripts/code_metrics_baseline.txt ratchet by keeping the lower
value on each side.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1958 -- refactor(prelude): read a warm-start prior into named values before PRELUDE acts on it

What it does: Splits the warm-replay lifecycle in phases/prelude.py (maybe_enqueue_warm_replay, _settle_warm_replay, _prepare_warm_kernel_kb, _judge_warm_replay_accuracy) into named stages and shares the kernel-rollback settle, patch-tree walk, history-row walk and donor field list that were previously written out two to four times; prelude.py shrinks by 163 lines. The stated behaviour changes are deliberate: warm-start numbers are read through hyperloom.common.coerce.to_float, so bool, NaN and inf values and invalid input take the fallback (a NaN replay throughput is now rejected by the throughput gate), and a negative warm-up throughput is kept in the audit-only cold_tput.

Blocking issues: none

SKIPPED: local pytest -- test modules import fcntl and do not collect on this Windows host; relied on CI test shards (py3.10/3.11, 6/6 each) green at the head.

Checked: prelude.py base vs head, unit by unit, for maybe_enqueue_warm_replay (+_read_replayable_current_recipe, _refused_dirty_warm_kernel, _warm_kernel_overlay, _enqueue_warm_replay, _warm_replay_launch_config, _replayable_patch_plan, _combined_warm_launch, _workload_compatible_launch, _warm_replay_task_params, _open_warm_replay_task, _warm_expected_gain); _settle_warm_replay (+_grade_warm_replay, _record_replay_measurement, _rule_on_warm_replay_keep, _adopt_reproduced_warm_replay, _push_warm_replay_onto_stack, _warm_replay_stack_entry, _warm_replay_candidate, _journal_warm_replay); _prepare_warm_kernel_kb (+_apply_warm_kernel_plan, _land_warm_kernel_entry, _settle_warm_kernel_preparation, _unwind_prepared_warm_kernel, _warm_kernel_persist_failure); _judge_warm_replay_accuracy; rollback_combined_warm / _resolve_promoted_recipe_checkout (+_warm_patch_trees); _collect_warm_kernel_plan; _warm_kernel_extra_envs; _merge_named_current_recipe_configs; inject_warm_recipe_history_into_ledger / warm_recipe_proven_items (+_warm_recipe_history_rows); agent_kb.prior_member/read*; shared_state field types; common/coerce.to_float; warm_replay_event.record_measurement | Ran: AST function-set diff base vs head (0 removed, 12 changed, 32 added); per-unit source diff; grep of external callers (loop/writeback.py:2105) and cold_tput/expected_gain_pct readers | Base: bb2da63 | Head: ac511a0

LGTM

@chaojhou
chaojhou merged commit e940668 into main Oct 22, 2026
43 of 44 checks passed
@chaojhou
chaojhou deleted the feature/chenyi/cc-prelude-warm-replay branch October 22, 2026 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size-exception Waives the PR diff budget; added by a maintainer with a reason in the PR description

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants