Repository navigation
fix(grid): a profiled AgentX round that Magpie completed is a succeeded round of run_grid - #2139
Merged
Merged
Conversation
…ed round of run_grid run_grid counted a round a success only when its measurement was valid. Magpie marks a profiled AgentX replay diagnostic_only and not publishable, so such a round was a failed variant (benchmark_report_invalid_metric) even though Magpie reported the profile capture complete. run_grid now asks round_succeeded: for a native AgentX round it is Magpie's own verdict (success, Magpie-owned, valid or diagnostic), for any other round it is valid_measurement. ProfileExecutor already used that predicate inline; it now shares the one in benchmark_result. A diagnostic round stays diagnostic_only and not benchmark_valid, so it is no KEEP evidence. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Code metrics gate: PASSEDJudged: the 4 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.
Module length warning: over 800 lines, within the 1200-line failure limit (not failing): 1
26 findings in files this change does not touchOutside this change (informational): 26
...and 1 more; the job log lists every row.
Thresholds and sources: |
siliangchen-amd
added a commit
that referenced
this pull request
Oct 23, 2026
main 5c33315 into dev 90fd129. Five files conflicted, all of them code main runs: - common/gpu_power_settings.py, common/provenance.py (toolbox): main's file with dev's outside_facts constants, as yihan specified in #121 c2431. - orchestrator/actions/executors/_grid_runner.py (toolbox): main's file with the _grid_runner.py hunks of #2139 and #2140, which are not on main yet. - inference_optimizer/cli/benchmark_config.py (specialist): dev's AgentXWorkload refactor, calling main's renamed gpu_runner_type. - orchestrator/actions/executors/_magpie_patcher.py (specialist): main's removal of the client tokenizer hook (#2054), with dev's lm-eval install-once patch kept. scripts/code_metrics_baseline.txt takes main's copy. Shared-File: src/hyperloom/common/gpu_power_settings.py @yihan.wang-agent Shared-File: src/hyperloom/common/provenance.py @yihan.wang-agent Shared-File: src/hyperloom/orchestrator/actions/executors/_grid_runner.py @yihan.wang-agent Shared-File: src/hyperloom/inference_optimizer/cli/benchmark_config.py @xiaofei.zheng-agent Shared-File: src/hyperloom/orchestrator/actions/executors/_magpie_patcher.py @xiaofei.zheng-agent Co-authored-by: Cursor <cursoragent@cursor.com>
Collaborator
PR #2139 -- fix(grid): a profiled AgentX round that Magpie completed is a succeeded round of run_gridWhat it does: Blocking issues: none Checked: LGTM |
chaojhou
approved these changes
Oct 23, 2026
chaojhou
left a comment
chaojhou
left a comment
Collaborator
There was a problem hiding this comment.
Approved for merge at the maintainer's request; see the LGTM comment above.
This was referenced Oct 23, 2026
fix(grid): a round reaped before its server was ready names the server log its workspace holds
#2140
Merged
xiaofei-zheng
added a commit
that referenced
this pull request
Oct 24, 2026
* Move profiler setup abd patching to common path * Fix shared profiler preparation for native AgentX * Simplify shared profiling setup and refresh patch status * Add default values for num_steps * Set detailed annotations to always true as Magpie check compatibility * chore(metrics): lower the _workload_envs module-lines baseline to 1664 * refactor(writeback): bring every function in the writeback module under the complexity ceiling (#1956) * refactor(writeback): lift the GEAK revalidation ruling out of _promote_explore _promote_explore carried the whole GEAK stack-revalidation path inline, so one method held the rebench reading, the grading that decides it, the overlay survival check, the material gate, and each of the four ways a ruling can be carried out. Give each of those its own name. _promote_explore: 493 -> 185 lines, cognitive 237 -> 79, cyclomatic 48 -> 24, nested depth 7 -> within the limit. Observable effect: none -- this is a pure refactor. No behaviour, interface, default, flag, or artifact changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): thread the rebench through one record, not eight locals The GEAK ruling's steps each took the measurement, the hash, the current best and the candidate as separate arguments, which both repeated the derivation and pushed three signatures past the parameter limit. Read the rebench once into _GeakRebench and pass that: current_best and candidate are then captured before a promotion can move them, so every arm judges the state the rebench was measured against. _rule_on_geak_rebench: 109 -> 78 lines, 7 -> 3 parameters. Observable effect: none -- pure refactor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): lower the baseline the merge left out of date Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): drop the geak_result assignment nothing reads CodeQL flagged it on this PR because the extraction moved the line, but it is the same dead assignment main carries: the except branch binds geak_result and the refusal branch below rebinds it before any read, on every path that reaches either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): lower the module-length entry the dead-line removal left stale Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): give the GEAK revalidation, harness replay and launch identity their own modules The writeback module carried four jobs at once: settling a task result into SharedState, ruling on a GEAK revalidation rebench, replaying a GEAK win through GEAK's own harness, and describing the launch `current_best` was measured on. The last three now live beside it, each as one module reading as one job, and the settlement handlers it keeps name the steps they take rather than inlining them. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): name the steps of the GEAK revalidation and the launch descriptor Each of the three moved modules now states its work as a sequence of named steps: what an accepted GEAK result offers to deploy, the params that re-measure it, how a 2a replay is run and what its accuracy and overlay earn it, and which layers reconstruct the launch `current_best` was measured on. The rebench's engagement evidence travels as one value instead of four parallel keyword arguments. The "did not validate" summary drops the accuracy fields it could never reach: an accuracy failure returns before it. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): settle each promote path through steps that name what they settle The per-kind promote handlers, the current_best lift and the unpromotable-result path each read as the sequence of decisions they make: what grading refuses, what a stack row is attributed to, which recipe delta a KEEP re-applies, what a failed baseline is charged to, and what the anchor run's facts are. The values that travel together — a lift request, the launch it resolves to, a crashed integrate window's marker, a scan of the event log — are named types rather than parallel locals and keyword arguments. Two duplicated decisions go with it: the profile and roofline handlers cleared the pending auto-roofline pointer twice under the same condition, and the GEAK 2a summary carried accuracy fields that an accuracy failure returns before it can ever reach. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): name the two rulings a GEAK revalidation can reach `validated` and `fallback` are the 2b ruling's own vocabulary, read by the harness replay as well, so they are named once where the ruling lives. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(writeback): store the writeback modules with LF endings A scripted edit on Windows rewrote them with CRLF, so every line of all four read as changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(writeback): report an orphaned framework KEEP under the kind that produced it The resume alert for a KEEP the stack lost reported the stack family label, ``framework``, where it had always reported the producing kind, ``framework_agent``. An operator filtering the resume report on the kind no longer saw them. The eval-failure trigger helpers lost their dependency on the collaborator with it: the enablement contract tests bind `_persist_eval_failure` onto a stand-in that carries SharedState alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): give the promote outcome its own module so the writeback modules do not import each other in a cycle ``_PromoteOutcome`` is the vocabulary the promote handlers and the GEAK revalidation share, so it sits beside both rather than in one of them. The GEAK modules now name `Task` through a plain import as well, which is what it always was. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): drop the function-length entry _resume_consistency_pass no longer earns Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(writeback): confirm integrate recovery before registering its levers An unconfirmed recovery sets the stop reason, saves the session and raises, so the levers a patch authored must not already be in that saved state: on resume the recovery can roll the patch back while explore keeps benching switches whose code is no longer on disk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): record the line writeback.py lost with the eval opt-out Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): rule on a GEAK win in one module instead of two that import each other The 2a harness replay is the 2b rebench's own fallback: it is reached from the rebench ruling, hands its verdict back as a _ReplayRuling, and shares that ruling's decision vocabulary. Splitting it off left the replay importing the revalidation at module level while the revalidation called into the replay through the mixin, which is a cycle with the evidence removed. One module, one mixin and the cross-module import go. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(loop): reduce the helpers' docstrings to their summary line The restructuring these files needed took writeback's function count from 76 to 176, and the per-function docstring convention took the docstring budget with it. The helpers added here keep a docstring, reduced to its summary line; the Args:/Returns: blocks go, and so does elaboration the signature already carries. The one sentence that states an invariant, a constraint from outside the file, or why a slower path is the correct one is kept where it exists. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(code-metrics): lower the module-length baseline the merge made stale Merging origin/main shortened the module, so its recorded module-lines entry no longer matches the tree and the gate reports the baseline as out of date. Lowered to the value CI measured; nothing else changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(orchestrator): let the collaborator base declare what it borrows `collaborator.py` is imported by every collaborator in the orchestrator, and it named `Coordinator` and `CoordinatorState` to annotate the object it borrows from. That pointed the package root back up into `loop.coordinator`, and it was the return leg of every import cycle CodeQL reports across the `loop` package: 321 cyclic import edges repo-wide collapse to 95 once it is gone. The borrowed surface is declared structurally instead, inside the `TYPE_CHECKING` block the file already had, so the contract is stated by the layer that consumes it and points down. `Coordinator` satisfies it unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(orchestrator): drop the protocol, widen the borrowed annotation The protocol restated nineteen members of the Coordinator in the module that borrows them -- a second declaration of a surface the Coordinator already owns, and it narrowed `_coord` for every subclass that reaches past those nineteen (245 new mypy findings). Widening the parameter instead deletes the same import with no second declaration: `cast` keeps the borrowed types the properties hand to callers, and mypy is back at its pre-change count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Lower the writeback module-lines baseline the merge made stale The merge hoisted the perf_metric imports to module scope, removing three lines from writeback.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): keep the launch descriptor and the promote outcome in writeback Fold `_launch_identity.py` and `_promote_outcome.py` back into `writeback.py` and restore every function they held to its merge-base body, except `build_env_spec`: its two near-identical `source_patch` appends become one comprehension, the recipe path is read once instead of three times, and the promoted-measurement fallback is read in place. `_geak_revalidation.py` no longer imports `_PromoteOutcome`: the GEAK ruling returns the verdict and `_promote_explore` records it, so the loop package's import graph points one way. `_integrate_keep_recipe_delta` and `_replayed_recipe_delta` built the same recipe delta from two different fallbacks; they are now one function that takes the fallback. `_server_arg_workload_fields` is back inside its one caller. No observable change: internal structure only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(writeback): drop the helper docstrings that restate their own names The one-line docstrings on the promote, keep-origin and GEAK-revalidation helpers repeat the name above them and nothing else; the GEAK module's own header restated the method list below it. Delete them, and fold the two branch-free record builders back into their single callers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(writeback): keep the revalidation-conclusion invariant with the method ``_record_geak_rebench_conclusion`` only moved module; its docstring states that the recorder declines silently once the kernel event has closed, which is an invariant the name does not carry. Restore it verbatim from the merge base. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(metrics): record the writeback module length the merges left out of date The entry read 3972 while the file is 4156; the gate excused it as base backlog because the merge base is 4717, so nothing reported the drift. --------- Co-authored-by: chennyiiis <chenyi@amd.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Test <test@test.com> Co-authored-by: xiaofei.zheng <xiaofei.zheng@amd.com> * feat(plugs): add the plug kernel, built-in profiles and the events socket (#2110) * feat(plugs): add the plug kernel, built-in profiles and the events socket Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * refactor(plugs): read the entry-point name once Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * Fix CodeQL findings in test_host activated test Co-Authored-By: Claude <noreply@anthropic.com> * fix(plugs): record a plugin name two distributions declare, and drop the test-only legacy groups iter_entry_point_objects skipped every name it had already yielded, so a second distribution declaring the same implementation name for an open socket never reached the registry: the first one won by metadata order and plugin_errors stayed empty. That dedup existed only for legacy_groups, which no caller passes. Both are gone; the registry's own duplicate check now records the conflict. load_manifest_file's docstring says JSON, which is all it reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Hou <chaojhou@amd.com> * feat(hosted): read and check the host-injected inputs in one place (#2114) * feat(hosted): read and check the host-injected inputs in one place Add hosted/host_info.py: read_host_info(env, require=...) checks the inputs a host injects into a hosted run, names every missing or malformed variable in a single error, and returns a frozen HostInfo. Credentials are held in a Secret that never prints. Document the variables. Co-Authored-By: Claude <noreply@anthropic.com> * fix(hosted): keep a URL query and a template key out of host-info errors and reprs A credential in a URL query or fragment would have been stored and printed in the HostInfo repr, and a malformed template entry's key was echoed in the error. Refuse query and fragment in the URL inputs and name no key in the template error. Co-Authored-By: Claude <noreply@anthropic.com> * test(hosted): derive the credential sentinels instead of spelling key-shaped literals The secret scanner flagged the fixture values. Sentinels derived from the variable's position are as distinctive for a substring search and carry no key-shaped literal. Co-Authored-By: Claude <noreply@anthropic.com> * feat(hosted): take_host_info removes every host-only input from the environment A scheduler that starts children without env= cannot strip inputs per launch, so the inputs must not stay in os.environ. NODE_NAME and CLAW_SESSION_ID stay because other code reads them. Co-Authored-By: Claude <noreply@anthropic.com> * feat(hosted): hold the archive region in HostInfo Co-Authored-By: Claude <noreply@anthropic.com> * feat(hosted): make the result archive optional; shared storage when unset HYPERLOOM_RESULT_ARCHIVE is unset (results stay on the shared storage) or s3://<bucket>/<prefix>. The AWS key pair is required only for s3://. Co-Authored-By: Claude <noreply@anthropic.com> * fix(hosted): refuse an archive prefix with an empty or '.' segment instead of rewriting it _archive_location split the prefix with PurePosixPath, which drops '.' segments and collapses repeated slashes before the check runs, so s3://b/a/./b, s3://b//a, s3://b/a// and s3://b/a/. were accepted and rewritten to another prefix. S3 keys are literal, so the results would land under a key the host did not name. The raw path is now split as written; one trailing slash still names the same prefix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Hou <chaojhou@amd.com> * [RSI] Block preflight unless the framework probe exits clean (#2099) * [RSI] Block preflight when the framework probe finds no usable GPU * Move preflight device-init detection to cli/device_init * Block preflight on any probe without a ROCm verdict * ci(code-metrics): tighten function length and nesting rules, require a reason on broad noqa (#2127) * ci(code-metrics): tighten function length and nesting, require a reason on broad noqa - Function length counts the lines of code in the body (no def lines, decorators, blank, comment or docstring lines); the limit drops from 80 to 50. - Nested block depth limit drops from 5 to 4. - Enable ruff B006 (mutable default argument) and B012 (jump in finally). - An added '# noqa: BLE001' must give a reason on its line. - Teach the gate a directory-size metric (entries directly in a directory); it stays pending until its threshold is added. - Re-seed the function-lines and nested-blocks baseline entries. * refactor(code-metrics): keep the A10 comparison walk within four nested blocks The gate scripts carry no baseline entries; split the match-case and compare branches of _comparisons into their own generators. * docs(agents): two review triggers remain once function length is gated --------- Co-authored-by: chaojhou <chaojhou@amd.com> * fix(prelude): commit a promoted warm replay's overlays on the framework tree (#2122) * fix(prelude): commit a promoted warm replay's overlays on the framework tree The overlays were left as uncommitted worktree changes, so every integrate_patch candidate stashed them before it applied and was benched on the stock tree. Checkout promotion now commits them on each git root; a commit that cannot be made fails the promotion, and a lift that refuses the replay moves HEAD back before the rollback restores the tree. Refs #2120 * fix(prelude): undo the promotion commit in every warm replay rollback The commit was undone only when the lift refused the replay. A commit failing on a later tree left the earlier trees committed, and a session killed after the commit resumed with a pending record still naming the pre-patch sha; both rollbacks then refused on head_mismatch. The pending record now says the overlays are being committed, persisted before the commit, and rollback_combined_warm moves a tree back to its pre_sha before it restores the files. * fix(prelude): undo the promotion commit on a tree the replay found already holding the overlays rollback_combined_warm skips a tree the replay did not write to, so a present_in_dirty_worktree overlay stayed committed after a refused replay. The undo now runs before that skip and resets the index too, leaving the overlays uncommitted as they were before the promotion. --------- Co-authored-by: BaoYunkai <269084791+BaoYunkai@users.noreply.github.com> * refactor(preflight): delete the framework-deps installer and preflight's unreachable branches (#2128) * refactor(framework-agent): drop unused __version__ constant Nothing reads the package-level version string. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): remove the per-framework dependency-manifest installer assets/framework_deps/ no longer exists, so the installer never had a manifest to act on. Drops framework_deps.py, the preflight framework_deps step, the install.sh pass and its tests and doc entries. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): drop the unreachable no-LLM-endpoint branch of _export_llm_endpoints _validate_credentials exits before endpoint export unless a base URL resolves, so resolved_urls is never None. Narrow it to tuple[str, str] across preflight and the model-resolution helpers and remove the dead None guards. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): make args required in _preflight and drop the recipe_kb_enabled copy _preflight and its helpers took an optional args only for tests; the sole production caller passes a Namespace, so the args-is-None branches and the IR-3 args_unavailable skip are gone. recipe_kb_enabled duplicated not args.degraded_kb and is derived at its one reader instead. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): share the rocm-smi GPU probe with detect_gpu_count device_init._rocm_smi_visible_gpus carried a line-for-line copy of detect_gpu_count's rocm-smi call and GPU[ dedup parse. visible_devices now owns the probe, keeping "could not probe" apart from "zero GPUs", and both use it. Co-Authored-By: Claude <noreply@anthropic.com> * chore(metrics): ratchet the code-metrics baseline for the preflight cleanup Lower the preflight.py `_preflight` function-length and module-length entries to match the tree after the cleanup. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): derive the diagnostics KB status from kb_degraded_reason The IR-3 step sets kb_degraded_reason before the diagnostics step runs, so the diagnostics no longer re-read degraded_kb. Also pin the skipped install-step contract of _check_gpu_visibility when rocm-smi cannot be probed, and restore a docstring in the auth-override fixture. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(preflight): replace the RocmSmiProbe result type with a raising rocm_smi_gpu_count The probe now returns a plain count and raises the standard subprocess/OS errors; device_init and detect_gpu_count map them themselves, so the dataclass, its Literal and their exports go. Drop the probe tests that duplicate detect_gpu_count and preflight coverage, the tautological degraded_kb asserts, the inaccurate "falls back to the other side" docstring, and the now-false "no LLM base URL resolved" diagnostics text. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Hou <chaojhou@amd.com> * Delete .github/workflows/auto-label.yml (#2137) * fix(grid): a profiled AgentX round that Magpie completed is a succeeded round of run_grid (#2139) run_grid counted a round a success only when its measurement was valid. Magpie marks a profiled AgentX replay diagnostic_only and not publishable, so such a round was a failed variant (benchmark_report_invalid_metric) even though Magpie reported the profile capture complete. run_grid now asks round_succeeded: for a native AgentX round it is Magpie's own verdict (success, Magpie-owned, valid or diagnostic), for any other round it is valid_measurement. ProfileExecutor already used that predicate inline; it now shares the one in benchmark_result. A diagnostic round stays diagnostic_only and not benchmark_valid, so it is no KEEP evidence. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: chaojhou <chaojhou@amd.com> * ci(code-metrics): gate directory size at 50 entries (#2136) * ci(code-metrics): gate directory size at 50 entries Directory size counts the production modules directly in a directory and its subdirectories that hold one. Baseline the two directories already over. * chore(code-metrics): drop function-args from PENDING Its threshold has been in pyproject.toml since the parameter-count gate landed, so the name should have left PENDING then; a config without it is now refused. * fix: parse every diff header shape, reject malformed HYPERLOOM_MN_* settings, restore kept patches on resume (#2130) * fix(common): decode quoted diff paths as UTF-8 and parse every header shape parse_patch_manifest read git's octal path escapes as Latin-1 characters ("b/caf\303\251.py" became "b/café.py"), rejected the header-only deletion of a binary or empty file (it has no --- line), rejected header-only binary and mode-only sections whose names are quoted or carry no a/ b/ prefix, and split non-git diffs at hunk lines such as "--- old comment", adding a write target the patch never makes. The no-git applier, the kernel snapshot apply and snapshot materialization all take their targets from this manifest, so each of these refused or misdirected a valid patch. Quoted names are now C-unquoted to bytes and read as strict UTF-8, the two diff --git names are split the way git apply splits them, a header-only deletion takes its path from that line, and non-git sections are walked by their hunk line counts. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): read HYPERLOOM_MN_POLL_TIMEOUT_S through one strict resolver Three readers applied three rules to the same variable: the multi-node CLI warned and fell back to 110s on a malformed value, optimize's rayjob adoption crashed on it (and on a whitespace-only value) with a bare int() ValueError, and the restart path read it with env_int and then took max() with the CLI's lenient read, which was never decisive. resolve_poll_timeout_s(default) is now the only reader: blank means unset, the value is floored at one second, and a malformed value raises EnvValueError naming the variable. The multi-node CLI reports that as a config error, not as a transient failure a re-run could fix. Observable effect: valid values behave as before. A malformed HYPERLOOM_MN_POLL_TIMEOUT_S now fails a multi-node CLI subcommand with exit 3 and optimize's cluster adoption with an EnvValueError naming the variable, instead of a warning plus a 110s budget or an unexplained int() error; a whitespace-only value counts as unset. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(executors): validate restart settings before the multi-node restart starts Every numeric HYPERLOOM_MN_* setting a per-round restart reads is now resolved once, up front, into RestartSettings. A malformed one used to surface wherever it was first read: the poll timeout and the health waits raised a bare EnvValueError past the restart's callers, which handle only ServerRestartFailed, and the health-poll, worker and probe settings were read inside the restart attempt's broad except, so each attempt killed and relaunched the server, swept its logs and reclaimed VRAM before failing with "post-launch /health wait raised". The restart now raises ServerRestartFailed("invalid multi-node restart setting: ...") before the TraceLens fan-out or any kill, which every caller already handles. A blank HYPERLOOM_MN_HEALTH_WAIT_S now counts as unset for the aiter wait widening, as it already did for the wait itself. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): read the serving-probe settings like the restart path The resume probe in restart-server and the post-restart wait read HYPERLOOM_MN_COMPLETION_PROBE_TOKENS and _MIN_TOKENS so that they agree on what "serving" means, but the probe used its own reader, which turned a malformed value into the default. It now reads both with env_int before its first request, outside the except that maps a ValueError to "not serving", so a malformed value raises EnvValueError naming the variable: the multi-node CLI exits 3, and a per-round restart has already rejected it up front. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kernel-agent): back every kernel-patch apply up under the session patches dir The apply backup root came from the payload when one was given, so a kernel patch's manifest could land anywhere. Every apply now backs up under <session>/patches/<kernel_id>/backup/, the one place the multi-node adoption looks for kept patches, and the policy gate no longer lists backup_root as a payload path. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): restore kept kernel patches on the pods when adopting a cluster A pod created after a KEEP starts from the image's sources. The old replay scanned kernel-agent-workspace for manifests with status=applied, a directory and a status no kept patch ever has, so it never ran. Adopting a cluster now sends every file a finalized patch changed, as the sandbox holds it, to every pod, carrying the AITER JIT invalidation once per build dir, and stops the run when a file cannot be restored. A cluster without server control (benchmark-only) is left as it is, since no restart would load the files. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(kernel): describe the kept-patch restore on cluster adoption Co-authored-by: Cursor <cursoragent@cursor.com> * chore(code-metrics): lower the duplicated-lines entries the removed replay test leaves behind test_multi_node_patch_replay_skip_and_failure_paths was a copy shared by test_common_utils.py and test_coverage_gap_units.py; deleting it drops their duplicated lines to 785 and 581, the values CI measured. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(multi-node): restore kept kernel patches in the resume pass, after the kept source layers The restore ran when the optimize CLI adopted the cluster, which made hyperloom.inference_optimizer.cli import the orchestrator: an upward import the layer contract forbids. It now runs in the Coordinator's resume pass, right after the kept source layers are back on this host's tree, so the pods get the kept files from a tree that holds them. A kept file that cannot be read or sent halts the resume with patch_recovery_incomplete, as a source layer that cannot be reinstalled does. A run that is not multi-node, or a cluster without server control, sends nothing. external_has_server_control is exported from hyperloom.inference_optimizer.multi_node so the orchestrator reaches it without the package's _internal modules. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kernel): let a failed pod backup cleanup leave the resume running The restore's cleanup dispatch raises TimeoutExpired when one pod stalls, and nothing caught it, so a resume whose kept kernel files had all been sent died while deleting their pod-side backups. A raised cleanup is now treated like a reported one: a warning, and the resume continues. The kernel tool's docstring now names the resume pass, which replaced the adoption replay, as the reader of the backups. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kernel): skip a kept kernel file a later KEEP deleted on resume A file a later snapshot KEEP deleted is absent from this host, and the restore stopped the resume because a kept file was missing, so every resume of such a session ended as patch_recovery_incomplete. This host's tree is the reference and the pods keep their copy after that KEEP itself, so the file is now not sent. A JIT build dir is now carried only by a file that is sent, so another KEEP's file carries it when the first carrier was deleted. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(multi-node): read the published-service gate's settings in place The gate bound published_ready_s and health_poll_s to local aliases that are never rebound. Reading them from the settings, as the worker and /health waits do, keeps the function's body at its length on main under the body-line count the code-metrics gate now uses. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> * chore(code-metrics): record the executors directory at 84 entries (#2148) #2130 added _multi_node_restart_settings.py to src/hyperloom/orchestrator/actions/executors and was merged right after #2136 set the directory-size threshold, with CI that predated the gate. The directory now has 84 entries against a recorded 83, and directory-entries has no merge-base exemption, so every change touching a file under it fails code-metrics on a count it did not change. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * fix(grid): a round reaped before its server was ready names the server log its workspace holds (#2140) The server log of a round that the session or boot limit reaped sits in the round's benchmark_* directory under the variant slot. The lookup that fills server_log_path searched the slot's warmup round only, so such a round was recorded skipped with no server_log_path, and nothing could quote the line its log ends on. The lookup now searches the benchmark_* directories of the slot as well as those of its warmup round. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: chaojhou <chaojhou@amd.com> * refactor(critic, kb): remove dead Critic request kinds and unused KB plumbing (#2129) * refactor(critic): accept only coordinator_inbox requests and drop dead runtime paths CriticAgentBackend only ever writes coordinator_inbox requests. The four dialogue request kinds (critic_decision_request, kb_draft_request, kb_hint_request, objection_signal) have no producer anywhere in the repo, so remove them together with everything that existed only for them, plus unreachable branches and unread state in the runtime. Runtime changes (src/hyperloom/agents/critic/runtime/): request_models.py - Remove DECISION_REQUEST, KB_DRAFT_REQUEST, KB_HINT_REQUEST, OBJECTION_REQUEST and REQUEST_KINDS; parse_request accepts only coordinator_inbox and always requires a non-empty raw_prompt. - Remove the decision_id auto-generation for decision/objection kinds. - Remove CriticRequest.messages, .decision, .raw and CriticRequest.to_dict. - Add Proposal.from_payload, the single owner of the action_name and predicted_gain_pct extraction (previously copied in inbox_parser). decision_reviewer.py - Remove _commit_decision_request, _commit_kb_draft and _topic_for_decision; commit_review always commits a coordinator inbox. - _fetch_kb_priors looks up priors per proposal only; the trace mode is always per_proposal (an idle round records requests=[]). - Remove JudgeBundle.messages, .decision, .kb_priors_for_decision and CommitOutcome.decision_review, .notes. - DecisionReviewer.__init__ takes a required kb_writer and reuses its session_memory; the kb_client argument and in-memory fallback go. - _populate_inbox drops the kind/raw_prompt guards and the InboxParseError catch; _maybe_write_kb_for_verdict drops the unreachable RuntimeAdapterError catch. - ceiling_importance reads CRITIC_IMPORTANCE_CEILING instead of a literal 0.84. - decisions.jsonl inbox records no longer carry a duplicate inner ts. kb_writer.py - Remove write_kb_drafts and slug_for_kind. - Remove WriteContext.source_role, source_type, topic and extra_metadata; metadata writes "source_role": "critic" directly. - KBWriter requires session_memory. - Fold the three dead-letter except branches of _upsert_with_dead_letter into one: reason is validation_error for KBValidationError and transport_error otherwise; only KBTransportError trips the breaker. kb_client.py, in_memory_kb_client.py - Remove batch_insert and add_edges (Protocol, HTTP and in-memory). cli.py - replay-dead-letter replays only upsert records; batch_insert, edges/add and list branches go. - Reuse DEFAULT_TIMEOUT_MS/DEFAULT_RETRY_MAX; drop the duplicate KB_SERVICE_TOKEN and KB_DEAD_LETTER_DIR reads (owners read them). - _resolve_reviewer builds DecisionReviewer(kb_writer=...). inbox_parser.py - ParsedPrompt keeps only shared_state and proposals; drop agent_name, inbox, kb_hints_text, extras, raw_payload, both to_dict methods, _agent_from_inbox_title and the "Knowledge base hints" branch. - InboxRow becomes the private _InboxRow. intent_envelope.py - build_envelope drops its inline idle fallback (decision_reviewer owns the idle intent; an empty list now fails validation). - build_review_verdict_intent drops the verdict and target checks that _commit_coordinator_inbox already enforces; the source check stays. - DEFAULT_IDLE_BODY leaves __all__. session_memory.py - MergeResult keeps merged and missing_keys; merge_context always persists and drops its no-op second loop. - get_cached_priors drops the now= parameter. slugify.py - Remove the no-op repeated-dash fold and slugify_safe's fallback_prefix. errors.py, importance_mapping.py, category_mapping.py - Remove InboxParseError, importance_for_kb_draft and the whole category_mapping module. Callers and docs: - orchestrator/roles/critic_agent.py: drop kb_priors_for_decision from _build_kb_priors_trace and from the _reason bundle view. - agents/critic/SKILL.md: describe only the Coordinator inbox caller; drop the dialogue Mission item, the request-type table, KB drafts and needs_info mentions. - agents/critic/README.md: replace the A2A hosting guide with the CriticAgentBackend flow; drop the Bash allowlist; keep Exit codes. - Delete actions/review_decision.md, actions/draft_kb.md, actions/objection_signal.md and references/decision_review_schema.md; drop the KB draft sections of references/verdict_schema.md and references/risk_rules.md; update references/coordinator_protocol.md. - pyproject.toml: fix the critic package-data comment. - scripts/code_metrics_baseline.txt: drop the write_kb_drafts entry. Tests: delete the tests of removed behaviour; move the empty-target check to test_commit_review_empty_target_raises; pin the idle-round trace (configured=True, requests=[]); pin that a bare KBError dead-letters as transport_error without opening the breaker. Observable changes: - prepare-review/commit-review reject the four dialogue kinds (rc=2). - An idle round no longer looks up KB priors by decision topic; its kb_priors trace keeps configured=True with requests=[] and prior_count=0. - judge_bundle.json loses messages, decision and kb_priors_for_decision; the Critic prompt's bundle view loses kb_priors_for_decision and SKILL.md changes, so the prompt cache prefix is invalidated once. - emit.json loses notes; decisions.jsonl inbox records lose their duplicate inner ts. - The dead-letter queue no longer produces batch_insert.jsonl; a leftover one now fails replay as an unknown endpoint and stays queued. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(kb): drop dead framework-KB start-up code and unused KB client, Experience KB and recipe-identity surface Framework KB (src/hyperloom/agents/framework/kb.py): - Remove prepare_kb_environment, check_kb_configuration, migrate_legacy_partition_once, _copy_partition_atomically, _LEGACY_WORKSPACE_KB_DIRNAME and _REMOVED_KB_ROOT_ENV, with the logging/shutil/uuid imports they needed. The legacy <workspace>/kb migration only served pre-release alphas, and FRAMEWORK_AGENT_KB_DIR has not been read for several releases. - read_pr_ledger() takes no argument and resolves through framework_optimization_root(), the same owner the writer uses. Preflight (src/hyperloom/inference_optimizer/cli/preflight.py): - Remove the prepare_kb_environment install step and its kb_withdrawn_override/kb_enabled locals; IR-3 still owns recipe_kb_enabled and kb_degraded_reason. - .importlinter: drop the two preflight -> agents.framework.kb ignores that no longer match. Critic KB clients (src/hyperloom/agents/critic/runtime/): - in_memory_kb_client.py: remove the test-only surface (simulate_failure, _maybe_fail, _failure_queue, all_rows, reset), the never-firing scope_value_normalized warning, the metadata filter, the deleted flag on rows, and the local copies of the scope normaliser and scope key. - kb_client.py: remove KBListQuery.metadata_filter, sort_by and include_deleted; HTTPKBClient.list sends sort_by="updated_at_desc" and include_deleted=False as literals, so the request body is unchanged. - scope_builder.py: expose normalise_scope_value, scope_key and UNKNOWN_SCOPE_VALUES as the single owners; scope_cache_key builds on scope_key (keys are byte-identical). - session_memory.py: _is_missing reads UNKNOWN_SCOPE_VALUES instead of its own copy. Experience KB (src/hyperloom/inference_optimizer/experience_kb.py): - Remove _manifest_context and every manifest/architecture fallback in _context: manifest.json never carries task_config or metadata, and SharedState has no architecture, compute_partition_mode, partitions, current_best_tput or best_tput attribute. The read payload sent in production is unchanged; _current_best_throughput stays local (importing orchestrator from inference_optimizer would break the hl-top-layers contract). Remove the now-unused _first_value. - from_env relies on RemoteConfig.from_env returning None for a blank HYPERLOOM_KB_URL instead of repeating the check. - Remove ExperienceKBEvidence.warnings, which nothing reads; the reads-off state is still logged. Recipe identity: - recipe_snapshot_constants.py: remove 15 unreferenced wire constants (F_AUTHORITY, F_CONFIDENCE, F_EVIDENCE_REFS, F_PROVENANCE, F_CANONICAL_ID, F_VERSION, F_METRIC_MIN, F_METRIC_MAX, F_PV_DETAILS, AUTHORITY_EXPERIENTIAL, ORDER_BY_*, FOREGROUND_HTTP_TIMEOUT_SEC, DEFAULT_CONFIDENCE), the test-only canonical_labels with its seven F_LABEL_* keys, and the stale "single source of truth" docstring. - orchestrator/knowledge/recipe_kb/__init__.py and canonical_id.py: stop re-exporting names nobody imports through them (CANONICAL_ID_*, DEFAULT_*_SLUG, LOCK_FILENAME, canonical_labels, detect_framework_version). recipe_canonical_id stays exported, so recipe_journal and recipe_kb_t0 keep their imports. Docs and baselines: - .env.template and docs/reference/environment-variables.md: drop the FRAMEWORK_AGENT_KB_DIR and legacy-partition migration sentences. - scripts/code_metrics_baseline.txt: drop canonical_labels; lower preflight.py::_preflight to 193 and preflight.py module lines to 2637. Tests: delete the tests of removed behaviour (legacy migration, withdrawn-variable warning, start-up hook, canonical_labels, metadata filter, deleted rows, fault injection); observe the in-memory KB through list(); carry the validation-error dead-letter contract to a stub upsert; pin the HTTP list body including the sort_by and include_deleted literals; read the PR ledger through the env override; build the Experience KB read context from state fields only. Observable changes: - The SBD V6 install event no longer has a prepare_kb_environment step. Setting FRAMEWORK_AGENT_KB_DIR no longer logs a warning or marks the install event degraded; --degraded-kb is still recorded by IR-3, the manifest's kb_degraded_reason and the diagnostics kb_status. - A workspace that only ever ran pre-release alphas no longer has its <workspace>/kb/framework_optimization ledger copied to framework-kb; the FRAMEWORK phase cold-starts that ledger instead. - In-memory critic KB rows no longer carry "deleted": false, and ExperienceKBEvidence no longer carries warnings. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(experience-kb): drop the session_dir the integration no longer reads _manifest_context was the only reader of ExperienceKBIntegration.session_dir, and it went with the dead manifest fallbacks. Remove the parameter from ExperienceKBIntegration.__init__, ExperienceKBIntegration.from_env and integration_for, and from Coordinator.experience_kb and the tests that threaded it through. Nothing observable changes. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(critic): drop the request kind now that the Critic accepts one With only coordinator_inbox left, the request's "kind" carried no information but was still validated, stored and echoed. Remove it end to end: - orchestrator/roles/critic_agent.py: the request no longer carries "kind" and the judge-bundle view sent to the LLM drops it. - runtime/request_models.py: parse_request no longer reads or validates "kind" (a stray "kind" key is ignored like any other unknown key); drop CriticRequest.kind and COORDINATOR_INBOX. - runtime/decision_reviewer.py: drop JudgeBundle.kind and CommitOutcome.kind, so judge_bundle.json and emit.json no longer carry "kind". - SKILL.md: replace the "Request Type" section with a pointer to the inbox action; actions/review_coordinator_inbox.md and README.md stop describing a request kind. - scripts/code_metrics_baseline.txt: lower CriticAgentBackend._reason to 73. - Tests: drop "kind" from every request, bundle and emit fixture, the JudgeBundle construction and the view-key table; delete test_parse_non_inbox_kind_rejected. Observable changes: request.json, judge_bundle.json and emit.json no longer have a "kind" key, and the Critic prompt's judge-bundle view loses its "kind" entry, so the prompt cache prefix changes once more. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(recipe-identity): drop the stale remote-URL comment recipe_snapshot_constants no longer holds any remote wire constants, so the note about there being no default KB_STORE_URL no longer describes anything in the module. Comment-only change. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Hou <chaojhou@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): select plug implementations from optimize, share entry-point discovery (#2111) * feat(plugs): add the plug kernel, built-in profiles and the events socket Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * feat(cli): select plug implementations from optimize, share entry-point discovery Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * refactor(plugs): read the entry-point name once Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * refactor(cli): keep optimize's growth-gated units no larger than main's Move the no-target-GPU warning into bootstrap and the resume flags into their own parser helper, and add the plug flags in one loop, so _run_optimize, _build_parser and the cli module are not larger than at main. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * chore(metrics): tighten the baseline for two units this change shrank Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Claude <noreply@anthropic.com> * Fix CodeQL findings in test_host activated test Co-Authored-By: Claude <noreply@anthropic.com> * Rename the optimize plug flag --profile to --plug-profile The flag no longer shadows the argparse abbreviation --profile -> --profile-osl, so existing invocations keep parsing as before. Adds regression tests for both. Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(cli): resolve and plan the plug manifest before any session state is written A refused plug flag, manifest variable or damaged plugs.resolved.json used to exit 2 only after the session was opened or seeded: each refused --resume-from ... --extend-hours grew the budget again, --force-resume cleared a terminal target_reached, and a fresh run left a half-seeded session. The manifest is now resolved and planned first (resolve_plug_manifest, no writes) and only bound to the session, recorded and announced afterwards (open_plug_host). Co-Authored-By: Claude <noreply@anthropic.com> * test(cli): assert the plug refusal itself, not only exit 2 Co-Authored-By: Claude <noreply@anthropic.com> * fix(kernelforge): log a failed provider load once The shared helper already logs it; the registry logged it again. Also pins that two entry points with one name are recorded as a plugin error. Co-Authored-By: Claude <noreply@anthropic.com> * docs: document the plug flags, the two manifest variables and the resume behaviour Co-Authored-By: Claude <noreply@anthropic.com> * refactor(cli): resolve_plug_manifest reads --resume-from itself; keep cli/__init__.py no longer than main Co-Authored-By: Claude <noreply@anthropic.com> * Lower code-metrics baseline entries that main improved Co-Authored-By: Claude <noreply@anthropic.com> --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: chaojhou <208445444+chaojhou@users.noreply.github.com> * refactor(writeback): give each promotion and recovery job its own module (#2132) `orchestrator/loop/writeback.py` had grown to 4156 lines: a hundred-method `WritebackCollaborator` plus the ~750 lines of helpers it read. The jobs it had collected -- staging a KEEP into the patch column, validating cumulative gain, charging a baseline failure, lifting a winner into current_best, promoting each of the five result kinds, pinning the launch a measurement ran under, replaying a resumed log, recovering the windows a crash left open, and auditing a resumed run -- each get a module, and the five-result dispatcher stays where it is. Every unit moves verbatim; the twelve stage modules are mixins the collaborator composes, so `coordinator.py` still constructs the same class. Co-authored-by: chenyi <test@test.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * set profiler config flags after probing the container. * Revert "set profiler config flags after probing the container." This reverts commit 3205af7. Magpie already probes the container for profiler support before it launches, so a second probe here is redundant, and it is the weaker of the two: it returns False for a server that lacks the fields, for a non-zero exit and for a probe that could not run at all, writes the same detailed_annotations: false in every case, records no degraded reason, and caches the verdict for the process. A probe that times out on a loaded node then costs a full profile round whose trace TraceLens cannot split, with nothing marking the round degraded. Back to the agreed shape: default true, explicit false preserved, Magpie checks runtime support. * fix(agentx): follow the patch outcome for the native annotation default Magpie defaults detailed_annotations to false and probes the server only when it is true, and that probe raises rather than degrading when the server lacks the fields. enable_profile claimed the field unconditionally, so a native profile round on a runtime the TraceLens patch did not reach ended before the server started, where the fixed-sequence path degrades instead. Both paths now read one predicate over the degraded reason patch_for_profile already writes; an explicit operator value is still preserved either way. --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Deval Shah <devashah@amd.com> Co-authored-by: Hao Li <18546749+haofrank@users.noreply.github.com> Co-authored-by: Tharun Adithya Srikrishnan <tsrikris@amd.com> Co-authored-by: Yang, Chenyi <chenyyan@amd.com> Co-authored-by: chennyiiis <chenyi@amd.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Test <test@test.com> Co-authored-by: xiaofei.zheng <xiaofei.zheng@amd.com> Co-authored-by: zoroyihan7 <Yihan.Wang@amd.com> Co-authored-by: Hou <chaojhou@amd.com> Co-authored-by: luochen <chenluo@amd.com> Co-authored-by: lishuoshuo-amd <shuoli@amd.com> Co-authored-by: BaoYunkai <ybao@amd.com> Co-authored-by: BaoYunkai <269084791+BaoYunkai@users.noreply.github.com> Co-authored-by: Zheng Gong <zgong@amd.com> Co-authored-by: fengshaoyi-amd <shafeng@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: meinali-566 <meili@amd.com> Co-authored-by: chaojhou <208445444+chaojhou@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
run_gridcounted a round a success only when its measurement was valid (valid_measurement). Magpie marks a profiled AgentX replaydiagnostic_onlyand not publishable, so a profiling round that ran to completion, with Magpie reporting the profile capture complete, came back as a failed variant (benchmark_report_invalid_metric).benchmark_result.round_succeeded(measurement)states the rule once: a native AgentX round is a success when Magpie's ownround_succeededsays so (success, Magpie-owned, valid or diagnostic); any other round succeeds when it isvalid_measurement.run_gridasks it in the two places that decided onvalid_measurement(_variant_row, and the settle loop in_settled_measurement, which no longer waits out the settle window for a valid measurement a diagnostic round never has). The warmup round still requires a valid measurement.ProfileExecutoralready carried the same expression inline in_measurement_succeeded; it now binds that hook toround_succeeded, so the two paths cannot drift.diagnostic_onlyand notbenchmark_validon itsVariantResult, so it is still no KEEP evidence._grid_runner.pyandprofile.pykeep their line counts (the one-name import of_resolve_probe_pythongoes on one line to pay for the new import).Why now
This is a hunk of the multibranch work that sits in existing
mainfiles; the multibranch measurement backend runs every point, profiles included, throughrun_grid. It is ported onto the grid-phase structure of #1964. The rest of that work arrives as new files in later PRs.Test plan
pytestontest_grid_runner*.py,test_benchmark_result*.py,test_profile_*.py,test_profile_and_kernel_handlers.py: 543 passed; the one failure,test_trace_analyze_handler_dry_run_returns_structured_result, fails the same way onmainin my environment (TraceLensis not installed there).TestADiagnosticAgentxRoundIsAVerdict(3 tests, a realrun_gridover a fake Magpie that writes a native AgentX report): a profiled replay (benchmark_validfalse,diagnostic_only,profile_capturecomplete) issucceededwithdiagnostic_onlytrue andbenchmark_validfalse; a replay that is neither valid nor diagnostic still fails withbenchmark_report_invalid_metric; a valid replay succeeds.valid_measurementback in_variant_rowturns the profiled-replay test red withbenchmark_report_invalid_metric.ruff check,ruff format --check(0.16.2),scripts/code_metrics.py --base-ref origin/main(PASSED, nothing new or worse).Review template
run_grid(explore, integrate bench, conc sweep, rebench) do not profile, andProfileExecutoralready treated a diagnostic round as a success. A caller that does send a profiling AgentX round throughrun_gridnow gets asucceededvariant instead of abenchmark_report_invalid_metricfailure.🤖 Generated with Claude Code