Skip to content

feat(common): add the rule pack and the outside_facts package that owns each outside-project fact - #2155

Open
zoroyihan7 wants to merge 8 commits into
mainfrom
feature/yihan/rule-pack
Open

zoroyihan7 wants to merge 8 commits into
mainfrom
feature/yihan/rule-pack

Conversation

@zoroyihan7

@zoroyihan7 zoroyihan7 commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Review only — merges after the multibranch acceptance run (see AMD-AGI board #121); do not merge before.

Summary

Adds the rule pack and hyperloom.common.outside_facts, the one place that states a fact about a project outside this repository (aiter, SGLang, vLLM, PyTorch, Triton, the HIP runtime, argparse, Transformers, InferenceX), and moves the readers on main that already repeat those facts onto it.

Matches multibranch-dev e81281d (refreshed 2026-10-30): all 18 files under src/ are byte-identical to that commit; the two CI/packaging files are this PR's own. Stack position: the first PR of the toolbox split that the measurement PRs need (#2157 B1's base branch merges this PR with #2154); nothing below it.

Commits on the branch:

  1. feat(common): add the rule pack and the outside_facts package ... (original cut, 2026-10-23).
  2. refactor(common): read the outside-project facts from outside_facts ... (original cut).
  3. ci(packaging): install PyYAML in the pyproject-declarations job (see below).
  4. Merge origin/main into feature/yihan/rule-pack: conflicts in env_safety.py, failure_signature.py and provenance.py resolved by taking multibranch-dev's version of the six readers, which is today's main plus only the outside_facts swaps (checked per file with git diff origin/main origin/multibranch-dev).
  5. chore(common): bring the rule pack up to multibranch-dev e81281d60.

What the package holds now:

  • common/outside_facts/packs/default.yaml (1252 lines) is the pack: each entry names the project that owns the fact, why it exists and its examples. Since the first cut it also states each component's source layout (aiter, triton, torch, sgl_kernel, flydsl, ...) and, per entry, an optional image_check that says how to confirm the fact against an installed image.
  • common/outside_facts/rule_pack.py (363) loads and checks it. A pack is refused, with the file and the entry named, when the file is missing or not one YAML document, a key repeats, an alias or merge key appears, an entry has an unknown kind or lacks project, reason or examples, a pattern does not compile or fails one of its own examples, a component layout or an image_check is malformed or fails its own examples, or the code reads an entry the pack lacks. Nothing falls back to an empty or default pack, because an unreadable fact must never read as "no constraint".
  • common/outside_facts/__init__.py (287) binds each entry to the constant the readers import (HIP_UNDEFINED_SYMBOL, ENV_CONC, ROCR_VISIBLE_DEVICES_ENV, ...) and exposes the component layouts.
  • New since the first cut, added because rule_pack.py on multibranch-dev imports them (same package, same layer, not on main, not in another PR): component_layout.py (158), image_check_spec.py (366), image_check_eval.py (231), image_env.py (316); and image_check.py (304), which evaluates the pack's checks against the environment it runs in (python -m hyperloom.common.outside_facts.image_check, exit 1 when a fact fails). Nothing on main calls image_check.run_image_check; the multibranch session does, and arrives later.
  • Tests: test_rule_pack.py (438), test_component_layout.py (74), test_image_check.py (617), test_image_check_pack.py (281).
  • pyproject.toml: one package-data line so packs/*.yaml ships in the wheel.
  • Readers: failure_signature.py, env_safety.py, provenance.py, visible_devices.py, gpu_power_settings.py and agentx_workload.py swap their literals (the HIP, HSA, import-error, argparse and Transformers log markers, the ROCR mask name and the InferenceX variable names) for the constants. The hipError[A-Za-z]* pattern is now the one of main's HIP_RUNTIME_ERROR rule (fix(failure-signature): a HIP call that failed while running is not a missing kernel #2141 landed) and reads HIP_ERROR_PREFIX + r"[A-Za-z]*".
  • .github/workflows/packaging.yml: the pyproject-declarations job installs PyYAML. That job installs only pytest because the packaging lint is stdlib-only, but the test package's conftest.py imports hyperloom.common.gpu_power_settings, which now reads the rule pack, and the pack is YAML. It is the one place that shows what the pack costs: hyperloom.common is no longer importable on a bare interpreter. Everything that runs the product already has PyYAML (the executors and benchmark_config import it at module level, the runtime extra installs it). If the reviewers want hyperloom.common to stay importable without third-party packages, the pack would have to be read lazily or not as YAML; that changes the loader, which is taken verbatim here, so it is raised and not done.

hyperloom/common directory count: no longer an issue

After main's #2308 moved the shared helpers to hyperloom_common, hyperloom/common has 35 entries on main and 36 with this PR, under the 50-entry limit; the code-metrics gate passes. outside_facts stays in hyperloom.common (optimizer-only code by #2308's rule; the hl-common-is-leaf contract allows it to import hyperloom_common).

CI state

Not yet observed at the refreshed head (e5cdfa0). The earlier head's only reds were the code-metrics verdict above and main's own collection error (#2149).

What is not in this PR, and why

  • measure/tests/test_outside_facts.py (the sentinel that changes every owned value in a fresh interpreter and requires each reader to follow) imports hyperloom_multibranch and hyperloom.measure.aiter_merge_hook, neither of which is on main. It lands with the measurement PRs.
  • The readers in hyperloom.measure, hyperloom.sandbox and hyperloom_multibranch import the same constants (and the multibranch session calls the image check); they arrive with their own PRs. The constants for them are already in the pack so the pack is one document from the start.
  • profile_{sglang,vllm,atom}.yaml keep DEBUG_CLR_GRAPH_PACKET_CAPTURE as literal data (a YAML file cannot import the owner); the sentinel above pins the copies to the pack once it lands.

Review template

  • Linked issue(s): none.
  • Tests: added, taken from multibranch-dev with the code. On Python 3.10 and 3.11 (each, -n 4): 993 passed, 0 failed, 0 skipped — the four outside_facts test files (132) plus every existing test that imports a file this PR touches (test_failure_signature, both test_launch_log_evidence, test_visible_devices, test_env_safety, test_gpu_power_settings, test_apply_gpu_power_settings, test_workload_envs, test_framework_whole_machine_gpu, test_isolated_vllm_pythonpath, test_ray_backend_unit, test_common_provenance, test_tool_version_provenance, test_magpie_agentx_contract, test_host_info, test_llm_attribution), the architecture guard and the packaging lint. Also run: ruff check ., ruff format --check on the 17 changed Python files (ruff 0.16.2), scripts/check_broad_suppress.py src scripts (rc=0), lint-imports --no-cache (11 kept, 0 broken), and scripts/code_metrics.py --base-ref origin/main (the directory finding above is the only one; duplication not measured locally).
  • Size/complexity triggers crossed: yes, size. About 2,100 production Python lines (the seven outside_facts modules, 2,025, plus 70 changed lines in six readers), the 1,252-line YAML pack and 1,410 test lines, over the 1000-line budget; it grew from ~520 because the pack's loader on multibranch-dev now validates component layouts and image checks, and those modules are part of loading the pack. No function crosses a limit.
  • If this refactors: the mechanism removed is the second copy of each outside fact in these readers. The contract preserved is the compiled patterns and name sets (the readers' diff against main is the swap only).
  • Observable effect: none for a run on the legacy flow, which classifies failures and screens environments as before. Two things an operator can now see: the new session input HYPERLOOM_RULE_PACK (a path; without it the reference pack that ships with the package is read), and a pack that cannot be used stops the first import of hyperloom.common.outside_facts, naming the file and the entry, instead of being ignored. python -m hyperloom.common.outside_facts.image_check is new and runnable on its own; nothing on main runs it. The six readers now need PyYAML to import, through that package.
  • Breaking changes: no.
  • PR addresses single concern: yes (the facts about outside projects have one owner, and the existing readers use it).
  • Root cause is upstream: no.

🤖 Generated with Claude Code

Label applied (2026-10-31, workstream owner's decision): size-exception for the diff budget: the pack loader and its four same-layer modules cannot be split from the pack they load.

zoroyihan7 and others added 2 commits October 23, 2026 20:17
…nds each fact to a name

Facts about projects outside this repository (aiter, SGLang, vLLM, PyTorch, the
HIP runtime, argparse, Transformers, InferenceX) that more than one reader
depends on are stated once, in a YAML rule pack. Each entry carries the project
that owns the fact, the reason it exists and its examples; a pattern that fails
one of its own examples, a repeated key, an alias, an unknown kind or an entry
the code reads but the pack lacks refuses the pack, naming the file and the
entry, and nothing falls back to an empty pack.

hyperloom.common.outside_facts binds each entry to the constant the readers
import. HYPERLOOM_RULE_PACK names the pack a session reads; without it the
reference copy beside the module is read. The pack ships as package data.

Nothing reads the constants yet; the next commit moves the readers that already
repeat these facts onto them.

Co-Authored-By: Claude <noreply@anthropic.com>
…n the readers that repeat them

failure_signature, env_safety, provenance, visible_devices, gpu_power_settings
and agentx_workload spelled out facts that the rule pack now owns: the HIP,
HSA, import-error, argparse and Transformers log markers, the ROCR mask name
and the InferenceX benchmark variable names. Each now imports the constant.

No behaviour change. The compiled patterns, the blocked-name set and the
provenance keys are identical to before: every pattern is built from the same
text, compared pattern by pattern.

Co-Authored-By: Claude <noreply@anthropic.com>
@zoroyihan7 zoroyihan7 added the type:feature New capability or improvement label Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Code metrics gate: PASSED

Judged: the 20 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension Limit New Worse Improved Over the limit vs base
Cyclomatic complexity fails > 20 0 0 0 0 +0
Cognitive complexity fails > 30 0 0 0 34 +0
Function length (lines) fails > 60 0 0 0 249 +0
Nested block depth fails > 4 0 0 0 37 +0
Parameter count fails > 5 0 0 0 0 +0
Module length (lines) warns > 800, fails > 1200 0 0 0 42 +0
Duplicated lines per file fails > 0 0 0 0 76 +0
Dead code (vulture) fails > 0 0 0 0 2 +0
Directory size (entries) fails > 50 0 0 0 1 +0
Check Findings Effect
Comments 0
Broad catch without a reason 0
Broad catch that records nothing 0
English only 0
Production code imports test code 0
Repeated literals 0
A1 Private name of another package 0
A8 Flag argument 0
A9 Tuple return 0
A10 Raw vocabulary comparison 0
A11 Hidden global write 0
A12 Pure forwarder 0
A12 Single-implementation abstraction 0

Module length warning: over 800 lines, within the 1200-line failure limit (not failing): 1

Module Lines Limit
src/hyperloom/common/env_safety.py 852 warns > 800

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, radon 6.0.1, ruff 0.16.2, vulture 2.16.

The test package's conftest imports hyperloom.common.gpu_power_settings, which
now reads the rule pack, and the pack is YAML. The job installs only pytest
because the lint itself is stdlib-only, so every test in it failed at setup
with "No module named 'yaml'". The job installs PyYAML beside pytest and stays
seconds long.

Co-Authored-By: Claude <noreply@anthropic.com>
siliangchen-amd added a commit that referenced this pull request Oct 24, 2026
…lds (#121 c2544, c2545)

The sync brings main's directory-entries threshold and main's baseline, common 57, into dev. Without
this line every dev change touching a file directly in src/hyperloom/common would fail gate 5 on a
directory it did not grow. The two entries over main's count are owned and have a route to main:
toolbox's common/outside_facts/ in #2155 (PR-3), and specialist's common/accuracy.py,
which folds into common/gain_math.py in the PR that takes the accuracy bars to main. The comment
above the line says so; --update-baseline rewrites the file without comments, so this message and
#121 keep the record too.

Shared-File: scripts/code_metrics_baseline.txt @yihan.wang-agent @xiaofei.zheng-agent
Co-authored-by: Cursor <cursoragent@cursor.com>
zoroyihan7 and others added 3 commits October 24, 2026 06:09
Co-Authored-By: Claude <noreply@anthropic.com>
Conflicts in env_safety.py, failure_signature.py and provenance.py were
resolved by taking multibranch-dev's version of the six readers, which is
today's main plus only the outside_facts swaps (checked with
git diff origin/main origin/multibranch-dev on each reader).

Co-Authored-By: Claude <noreply@anthropic.com>
Every file of the outside_facts package and its tests now equals
origin/multibranch-dev e81281d byte for byte; the six readers already
did after the merge of main.

Moved: nothing (the package layout outside_facts/{__init__,rule_pack}.py
and packs/default.yaml was already the dev layout).

Added, because rule_pack.py on dev imports them (same package, same
layer, not on main and not in another PR of the stack):
- outside_facts/component_layout.py (each component's layout in the pack)
- outside_facts/image_check_spec.py, image_check_eval.py, image_env.py
  (the image_check entries a pack declares, checked against each entry's
  own examples when the pack loads)
- outside_facts/image_check.py (evaluates those checks against the
  environment it runs in; python -m ... image_check), which completes
  the package and is what test_image_check exercises
- tests: test_component_layout.py, test_image_check.py,
  test_image_check_pack.py

Changed: packs/default.yaml grows with the component layouts and the
image_check entries; rule_pack.py validates them; __init__.py binds the
new constants; test_rule_pack.py follows.

Removed: nothing.

Co-Authored-By: Claude <noreply@anthropic.com>
@zoroyihan7 zoroyihan7 added size-exception Waives the PR diff budget; added by a maintainer with a reason in the PR description baseline-raise Waives new, worse and baseline-growth code-metrics findings; reason required in the PR description labels Oct 31, 2026
zoroyihan7 and others added 2 commits October 31, 2026 16:40
Main's #2308 moved the shared helpers to the top-level hyperloom_common
package. The rule pack stays in hyperloom.common.outside_facts (only the
optimizer reads it, and hl-common-is-leaf allows hyperloom.common to
import hyperloom_common). Resolve the import conflicts in
gpu_power_settings and provenance onto hyperloom_common, and point
outside_facts.image_check at hyperloom_common.io.

Co-Authored-By: Claude <noreply@anthropic.com>
… what was asked

Same change as multibranch-dev 68e612b (CodeQL: non-standard exception in __getattr__).

Co-Authored-By: Claude <noreply@anthropic.com>
@zoroyihan7 zoroyihan7 removed the baseline-raise Waives new, worse and baseline-growth code-metrics findings; reason required in the PR description label Oct 31, 2026
@zoroyihan7
zoroyihan7 marked this pull request as ready for review October 31, 2026 17:09
@zoroyihan7
zoroyihan7 requested a review from a team as a code owner October 31, 2026 17:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size-exception Waives the PR diff budget; added by a maintainer with a reason in the PR description type:feature New capability or improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants