Skip to content

refactor(kernelforge): pass rewrite, orchestration and example-driver inputs as typed objects instead of parallel keywords - #1975

Merged
chaojhou merged 13 commits into
mainfrom
refactor/args-kf-rewrite
Oct 19, 2026
Merged

chaojhou merged 13 commits into
mainfrom
refactor/args-kf-rewrite

Conversation

@lishuoshuo-amd

@lishuoshuo-amd lishuoshuo-amd commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator
  • Description: what and why

    Finishes the kernelforge parameter-count work (function-args > 5) in rewrite_by_flydsl, orchestrator and the example drivers. Values that travel together through the rewrite pipeline and the orchestration turns were carried as long keyword lists; each function now takes the object they belong to:

    • Rewrite pipeline: run_rewrite takes one RewriteRun that forge-rewrite-by-flydsl builds; RewriteWorkspace, RewriteBudget, ApplybackSwitch and OptimizeOptions move from the CLI module down into the rewrite layer that owns them (rewrite_by_flydsl/request.py, optimize.py); OptimizeRequest, PortRequest, DriverPreparation, DriverMode, RewriteSubject, RewriteKbTarget, FlydslMeasurement, ApplybackRequest, CollectedPatch, ResultScope / RewriteOutcome; RewriteSpec gains a framework field so resolve_identity and the apply-back step read it from the spec; the KB helpers become methods of _RewriteKb.
    • Orchestrator: AnalysisSources, _BundleIdentity, _BundleReading, _AnalysisSession in analysis.py (callers build AnalysisOutcome directly instead of through the _build_outcome forwarder); SpecialistRound in contracts.py, shared by orchestration and plan_critic.review; one _Turn for every orchestration turn.
    • Example drivers: the softmax drivers take one _Case; the mori driver uses _TunedOp, _CorrectnessCase, _BenchPlan; the mxfp8 driver uses _Activation / _Weights / _Routing.
    • Parameters no caller passes, or that the body never read, are deleted (evidence below).

    No **kwargs, untyped dict, compatibility shim or new fallback branch is added; every call site is updated.

    Scope: the 50 remaining functions over the limit in those directories (Triton/Gluon @jit kernels excluded); 49 are done. _grouped_gemm_mxfp8 (13) keeps its signature: it is the source kernel the rewrite example ports from, named by --source-entry in run_example.sh, mirroring SGLang's host API, and its source text feeds the port agent and the KB source identity.

    Gate count of functions over the parameter limit (function-args entries in scripts/code_metrics_baseline.txt): 256 on main -> 215 (-41; the 8 example-driver functions are outside the gate's scope); the cyclomatic-complexity entries and the broad-except count are unchanged.

  • Linked issue(s): none.

  • Tests: added/updated? commands run?

    • Tests changed only where a signature changed; assertions are kept, except three assertions on write_flydsl_kb_solution(best_commit=), an argument the function never read; where it mattered the equivalent session_key is asserted instead. test_forge_orchestration.py gains a shared fixture so the touched tests do not duplicate setup.
    • CLI contract: python -m kernelforge.cli <cmd> --help for the root command, forge-loop, forge-rewrite-by-flydsl, forge-fuse, kernel-rewrite-controller, roofline-ceiling, gemm-tune run, and forge-rewrite-by-flydsl --capabilities-json / --applyback-contract-json are byte-identical to main. Two failing forge-rewrite-by-flydsl runs (missing source, missing driver) give the same stdout and result JSON as the base apart from the random attempt-directory name.
    • Every changed production module imports under python:3.10-slim and python:3.11-slim (the CLI module needs click, absent from the slim image). The 4 changed example drivers compile on both versions; they need torch and GPUs to run, which were not available for this change.
    • Locally, per test file with PYTHONPATH=src (never the full suite): 34 related test files, 1336 passed.
    • ruff check / ruff format --check on the changed files; import-linter keeps all 7 contracts; python scripts/code_metrics.py --base-ref origin/main PASSED; baseline lowered with --update-baseline.
  • Size/complexity triggers crossed: split, or why the shape is right:
    Diff budget: 2494 production Python lines changed (+1120 / -1374 in 23 production .py files): rewrite 1370 (including cli/forge_rewrite_command.py), orchestrator 747 (including loop/analysis_runtime.py), examples 377. One concern (the last kernelforge parameter-count targets). Each commit touches at most 5 production files and is grouped by module. The code-metrics gate passes without a waiver.

  • If this simplifies or refactors: which mechanism was removed, which contract was preserved, and the tests that show it:
    Removed: the keyword lists threaded through the rewrite stages and orchestration turns, the _build_outcome forwarder, and the option dataclasses' detour through the CLI module. Preserved: the forge-rewrite-by-flydsl CLI and JSON contracts used by AgentKernelArena (above), the KB records and identities, and each example's documented command.

    Deleted parameters (checked across src, scripts, docs, examples and tests): run_port_loop(kernel_backend=, validate_stage_timeout_sec=) and preflight_candidate(warmup=, iters=) (never passed); try_flydsl_kb_warmstart(validation_timeout_sec=) (only one test passed it, with the default 1800); write_flydsl_kb_solution(best_commit=) (never read).

  • Observable effect: what an operator will now see, and what the old behaviour cost them (if none, why the change is unobservable):
    None: no CLI flag, default, help text, JSON contract, output file or log line changes. kernelforge.cli now imports the rewrite package at startup because the rewrite option types live there (about 17 ms). In the mori example driver one input slice moved from inside the graph-capture block to just before it; slicing launches no GPU work, so the captured graph is the same.

  • Breaking changes: yes, for Python callers only: the refactored functions take their typed objects, and RewriteWorkspace / RewriteBudget / ApplybackSwitch / OptimizeOptions live in kernelforge.rewrite_by_flydsl.request / .optimize instead of the CLI module. The CLI is unchanged. Every in-tree caller is updated.

  • PR addresses single concern: yes (parameter count of rewrite_by_flydsl, orchestrator and the example drivers).

  • Root cause is upstream (Magpie/TraceLens/GEAK/IntelliKit/AgentKernelArena), ticket filed: n/a.

@lishuoshuo-amd lishuoshuo-amd added the size-exception Waives the PR diff budget; added by a maintainer with a reason in the PR description label Oct 18, 2026
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Code metrics gate: PASSED

Judged: the 40 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension Limit New Worse Baseline out of date Baseline entries vs base
Cyclomatic complexity fails > 20 0 0 0 69 +0
Cognitive complexity fails > 30 0 0 0 275 +0
Function length (lines) fails > 80 0 0 0 419 -3
Nested block depth fails > 5 0 0 0 26 +0
Parameter count fails > 5 0 0 0 215 -41
Module length (lines) warns > 800, fails > 1200 0 0 0 52 +0
Duplicated lines per file fails > 0 0 0 0 98 +0
Dead code (vulture) fails > 0 0 0 0 2 +0
Check Findings Effect
Comments 0
English only 0
Production code imports test code 0
Repeated literals 0
A1 Private name of another package 0
A8 Flag argument 0
A9 Tuple return 0
A10 Raw vocabulary comparison 0
A11 Hidden global write 0
A12 Pure forwarder 0
A12 Single-implementation abstraction 0

Module length warning: over 800 lines, within the 1200-line failure limit (not failing): 1

Module Lines Limit
src/kernelforge/rewrite_by_flydsl/applyback.py 929 warns > 800
93 findings in files this change does not touch

Outside this change (informational): 93

Unit Dimension Baseline Now
src/hyperloom/orchestrator/enablement/recipe/projections.py project_launch_evidence Cognitive complexity absent 36
src/hyperloom/orchestrator/state/shared_state.py resolve_graded_comparison Cognitive complexity absent 34
src/hyperloom_kb/collect/mapping.py _experience Cognitive complexity absent 31
src/hyperloom_kb/schema.py FieldDeclaration.__post_init__ Cognitive complexity absent 32
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Cognitive complexity absent 36
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Cognitive complexity absent 40
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Cognitive complexity absent 44
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Cognitive complexity absent 38
src/hyperloom/inference_optimizer/cli/__init__.py _reexport_session_environment Function length (lines) absent 112
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_agent_lineup Function length (lines) absent 89
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_run_topology Function length (lines) absent 92
src/hyperloom/inference_optimizer/cli/__init__.py _resume_session Function length (lines) absent 171
src/hyperloom/inference_optimizer/cli/__init__.py _seed_fresh_session Function length (lines) absent 172
src/hyperloom/inference_optimizer/multi_node/cli.py _restart_multinode Function length (lines) absent 107
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _build_parser Function length (lines) absent 84
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _spawn_rank_actors Function length (lines) absent 90
src/hyperloom_kb/sync.py GlobalSync.pull Function length (lines) absent 84
src/kernelforge/cli/forge_loop_command.py _RunReporter.build_result Function length (lines) absent 91
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Function length (lines) absent 136
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Function length (lines) absent 149
src/kernelforge/loop/runner.py IterationLoop._close_iteration Function length (lines) absent 117
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Function length (lines) absent 184
src/kernelforge/loop/runner.py IterationLoop._implementer_history Function length (lines) absent 112
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Function length (lines) absent 102
src/kernelforge/loop/runner.py IterationLoop._run_supervisor_review Function length (lines) absent 88

...and 68 more; the job log lists every row.

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, ruff 0.16.2, vulture 2.16.

@lishuoshuo-amd

Copy link
Copy Markdown
Collaborator Author

PR #1975 -- refactor(kernelforge): pass rewrite, orchestration and example-driver inputs as typed objects instead of parallel keywords

What it does: kernelforge's rewrite pipeline (run_rewrite now takes one RewriteRun built by forge-rewrite-by-flydsl, with RewriteWorkspace/RewriteBudget/ApplybackSwitch/OptimizeOptions moved from the CLI module into rewrite_by_flydsl/request.py and optimize.py), the orchestrator (SpecialistRound, _Turn, AnalysisSources and friends) and four example drivers carried values that travel together as long keyword lists; each function now takes the typed object they belong to, RewriteSpec gains framework, and never-passed or unread parameters are deleted, with no intended behaviour change. Stacked on #1974; reviewed against its base refactor/args-kf-core only.

Blocking issues: none

deferred: none

Step 7 independent reader: a second reader re-checked the forge-rewrite-by-flydsl help and JSON contracts against the base, the Python 3.10/3.11 imports and the size breakdown (1370 + 747 + 377 = 2494) at head.
SKIPPED: V4 -- # PARTIAL: #1032 changes 1075 files, only its first 100 were compared, overlap there unchecked
SKIPPED: V4 -- # PARTIAL: #1154 changes 1232 files, only its first 100 were compared, overlap there unchecked
SKIPPED: V4 -- # PARTIAL: #1276 changes 1311 files, only its first 100 were compared, overlap there unchecked
SKIPPED: V4 -- # PARTIAL: #1869 changes 427 files, only its first 100 were compared, overlap there unchecked
SKIPPED: V4 -- # PARTIAL: #1974 changes 117 files, only its first 100 were compared, overlap there unchecked

Checked: src/kernelforge/{cli/forge_rewrite_command,loop/analysis_runtime}.py, orchestrator/{analysis,contracts,orchestration,plan_critic}.py, rewrite_by_flydsl/{request,measurement,runner,kb,identity,ingest,spec,applyback,driver_contract,flydsl_rewrite_driver_preparation,optimize,port_loop,report}.py, the 4 example drivers, option_groups.py, .importlinter, AGENTS.md; tree-wide git grep for 34 reshaped symbols, for RewriteSpec(, spec serialisation (asdict/vars/fields), driver_sha256, and _combine_with_config (scope: src excluding hyperloom, docs, scripts) | Ran: body re-fetched at head cf56675 (only the Diff budget line changed; its figures now match numstat); PYTHONPATH=src single-file pytest on test_rewrite_cli_contract, test_rewrite_by_flydsl_applyback, test_rewrite_by_flydsl_kb, test_plan_critic, test_rewrite_driver_contract, test_rewrite_budget, test_rewrite_by_flydsl_pipeline, test_analysis_agent, test_forge_orchestration, test_flydsl_rewrite_driver_preparation, test_rewrite_by_flydsl, test_recovery_analysis (all pass); python -m kernelforge.cli help/capabilities x9 base vs head (cmp); harness_orch.py, harness_analysis.py, harness_rewrite.py (real CLI, stubbed stages), harness_kbrec.py, harness_ab_prompt.py on base and head ($WORK); lint-imports (7 kept, 0 broken); import of 19 changed production modules, RewriteRun() construction and compile of the CLI and all example drivers in python:3.10-slim and python:3.11-slim with /tmp/pydeps; example drivers not executed (no torch/GPU) | Base: 0a0458c | Head: cf56675

LGTM

Base automatically changed from refactor/args-kf-core to main October 19, 2026 03:37
…est; own OptimizeOptions in the optimize stage
…nd FlydslMeasurement; drop the unused best commit and never-passed validation timeout
…llectedPatch and run each attempt from a brief and a turn
…ce, budget and apply-back switch in the rewrite layer
… from one AnalysisSession and read its inputs as AnalysisSources; build outcomes directly
…d; run every orchestration turn from one Turn
@lishuoshuo-amd
lishuoshuo-amd force-pushed the refactor/args-kf-rewrite branch from cf56675 to 4206089 Compare October 19, 2026 03:42
@lishuoshuo-amd

Copy link
Copy Markdown
Collaborator Author

Retargeted to main after #1974 merged and rebased onto it (head 4206089). The 13 commits applied without conflicts and the code-metrics baseline needed no change. Re-verified at this head: code-metrics gate passes, the rewrite / orchestration / tier3 test files pass (235 tests), and CI is green (33 pass, 7 skipped). The earlier LGTM stands.

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1975 -- refactor(kernelforge): pass rewrite, orchestration and example-driver inputs as typed objects instead of parallel keywords

What it does: brings the last kernelforge parameter-count targets under the limit. run_rewrite takes one RewriteRun built by forge-rewrite-by-flydsl; RewriteWorkspace / RewriteBudget / ApplybackSwitch / OptimizeOptions move from the CLI module into the rewrite layer; RewriteSpec carries framework; the orchestrator's turns, analysis session and specialist round each take one object; the example drivers take small case records; and parameters no caller passes or the body never read are deleted. No CLI flag, default, help text, JSON contract or KB identity changes.

Blocking issues: none

Non-blocking:

  • OptimizeOptions.profile_timeout_sec defaults to 1800 while the --profile-timeout-sec option that fills it defaults to 3600, and RewriteBudget.deadline_unix defaults to None against the CLI's 0.0. Only tests use the type defaults, but one value now has two defaults; the CLI-filled types would read better with none.
  • preflight_reference(warmup=10, iters=30) keeps its knobs while the identical candidate values moved into _CANDIDATE_BENCH. The source and candidate must be timed the same way for the speedup to mean anything, so one shared constant for both DriverModes would hold that rule in one place.
  • Every kernelforge.cli subcommand (gemm-tune, forge-fuse, kernel-rewrite-controller, and the hyperloom install.sh probe) now loads the rewrite package at startup, so an import error there would take all of them down. Worth one sentence beside the 17 ms note; the lazy imports left inside forge_rewrite_command.py no longer avoid anything.
  • knowledge/local_wiki/.../mori/operators/ep_dispatch_combine/tuning.md:136 still names _combine_with_config, now _TunedOp.combine_with_config.

Checked: every positional construction of the new records against the old keyword order (DriverPreparation, PortRequest, ApplybackRequest, CollectedPatch, _AuthoringTurn, _AnalysisSession, _BundleIdentity, _BundleReading, SpecialistRound, FlydslMeasurement); defaults and the constants replacing deleted parameters (1800 s timeouts, 10/30 and 1/1 bench runs); RewriteSpec.framework (not serialised; KB identity and _infer_framework unchanged); the _build_outcome removal (replace reproduces the three exhausted branches); the mori capture move (the slice is a view, no kernel launch); consumers outside src/kernelforge (hyperloom reaches kernelforge only through the CLI; no doc or SKILL names a moved type); the CLI import closure (stdlib and kernelforge modules only, no import-time env or logging side effects) | Ran: --help, forge-rewrite-by-flydsl --help / --capabilities-json / --applyback-contract-json, forge-loop --help on base and head (byte-identical); the real forge-rewrite-by-flydsl through click's CliRunner on base and head with stubbed GPU/LLM stages in four modes (full options, defaults, --no-applyback --no-rewrite-kb, best moved off the PORT commit), recording PORT/OPTIMIZE argv, every KB write, resolve_identity, apply-back inputs and the result JSON (identical apart from the attempt-directory name); CI green at head. Not run: pytest (not installed here) and the example drivers (no GPU) | Base: d914a83 | Head: 4206089

LGTM

@chaojhou
chaojhou merged commit a40eec4 into main Oct 19, 2026
40 of 42 checks passed
@chaojhou
chaojhou deleted the refactor/args-kf-rewrite branch October 19, 2026 05:16
@xiaofei-zheng

Copy link
Copy Markdown
Collaborator

What this PR does

Finishes the kernelforge parameter-count work by replacing the long parallel keyword lists in rewrite_by_flydsl, orchestrator and the four example drivers with the object each group of values belongs to, defined in the layer that owns it: RewriteRun / DriverSetup / RewriteWorkspace / RewriteBudget / ApplybackSwitch in the new rewrite_by_flydsl/request.py, OptimizeOptions / OptimizeRequest in optimize.py, plus RewriteSubject, FlydslMeasurement, ApplybackRequest / CollectedPatch, RewriteKbTarget, ResultScope / RewriteOutcome, SpecialistRound, AnalysisSources and the private turn and identity types. The four option dataclasses move out of the CLI module down into the rewrite layer, four superseded forwarders are deleted, and four parameters that no caller passed or no body read are removed. function-args entries in the gate baseline drop 256 -> 215.

Blocking issues: none

Checked, file by file against the merge base: cli/forge_rewrite_command.py, rewrite_by_flydsl/{runner,request,measurement,ingest,spec,identity,kb,applyback,optimize,port_loop,report,driver_contract,flydsl_rewrite_driver_preparation}.py, orchestrator/{analysis,orchestration,contracts,plan_critic}.py, loop/analysis_runtime.py, and the four examples/*/driver.py.

  • Every positional dataclass construction matched field for field against the keyword order it replaced, and every new field default matched the base signature default — including the three literals that became module constants (_VALIDATE_STAGE_TIMEOUT_SEC, _VALIDATION_TIMEOUT_SEC, _CANDIDATE_BENCH / _CANDIDATE_PROBE).
  • The four deleted parameters — run_port_loop(kernel_backend=, validate_stage_timeout_sec=), preflight_candidate(warmup=, iters=), try_flydsl_kb_warmstart(validation_timeout_sec=), write_flydsl_kb_solution(best_commit=) — confirmed unread or never varied by reading the base bodies and grepping src/, scripts/, docs/, *.sh and *.md.
  • The three values that changed source traced to the same value the base produced: spec.framework feeding resolve_identity and _infer_framework (the CLI supplies it from applyback_target.framework, as run_rewrite(framework=) did), _collect_patch recomputing Path(spec.workspace).resolve(), and _run_result reading turn.role where every _run caller leaves the default "orchestration".
  • option_groups.py:22 assembles the relocated CLI dataclasses by field name (kwargs.pop(field.name)), so moving them cannot reorder or drop an option; their new defaults never apply on the CLI path.
  • RewriteSpec is not serialized anywhere in the tree, so the added framework field moves no persisted shape.
  • _setup_failed's closure over the rebound scope keeps the same temporary_paths before and after the attempt directory exists, matching the base's late-bound list.
  • In the mori driver, combine_input[:n_recv] moving out of the torch.cuda.graph block is a dim-0 view with no allocation and no launch, so the captured graph is unchanged; _BenchPlan and _CorrectnessCase are module-level frozen dataclasses and pickle across mp.spawn.
  • Test inventory compared base vs head: 0 test functions removed across all 16 changed test files, assertion counts identical in 15 of 16. The -2 in test_rewrite_by_flydsl_pipeline.py is the three assertions on the dead best_commit argument against one added session_key assertion; the behaviour they could have pinned (each KEEP published from its own commit) is still asserted via content_override at :1716-1717.
  • code_metrics_baseline.txt diffed: 256 -> 215 function-args, every changed function-lines entry lower, no cyclomatic-complexity entry touched.

Not run: the kernelforge suite does not execute on the review host (tracker/experiment.py imports fcntl), and the example drivers need torch and multiple GPUs, so no conclusion here rests on a local test run — the graph-capture and mp.spawn points rest on reading. CI's 12 test shards on 3.10 and 3.11, plus code-metrics, import-linter, mypy and ruff, are green at 4206089.

Not checked: openprs.txt was truncated for #1032, #1154, #1276, #1869, #1962 and #1977, so overlap with those six was not compared.

Base d914a83 / head 4206089.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size-exception Waives the PR diff budget; added by a maintainer with a reason in the PR description

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants