[zoroyihan7]

Review only — merges after the multibranch acceptance run (see AMD-AGI board #121); do not merge before.

Stacked on a merge of the two stacks. The base branch of this PR, feature/yihan/measure-leaves-base, merges feature/yihan/sandbox-layer (#2154, which carries the measurement contract #2153) and feature/yihan/runner-basics (#2156, which carries the rule pack #2155). It has no PR of its own and nothing else is added to it, so the diff below is this PR's own. On 2026-10-30 all four were refreshed to multibranch-dev e81281d60 on top of the current main (20d8b2043), and the base branch merged them again (8604af291). The modules here import hyperloom.contracts.measurement and hyperloom.contracts.recipe (#2153), hyperloom.common.outside_facts (#2155) and, for pyproject.toml, the hosted extra that #2154 defines; no single stack has all of these. The base merge resolves two one-line conflicts by keeping both sides: _DEV_ONLY in plugs/tests/test_architecture.py (main lists hyperloom.measure and hyperloom.sandbox there; #2156 removes measure, #2154 removes sandbox, so the base keeps only hyperloom_multibranch) and the package-data list in pyproject.toml (the rule pack's common/outside_facts/packs/*.yaml; the contracts' schemas are now covered by main's own contracts/*.json). When the four PRs below have merged, this PR retargets to main with git rebase --onto main feature/yihan/measure-leaves-base feature/yihan/measure-leaves.

Summary

Adds the bottom layer of the measurement core, hyperloom.measure: the modules that import only the measurement contract, the common helpers and each other, among them the instruction sheet (spec), the result manifest (manifest), signing and the check of a result bundle (bundle). All of it is taken from multibranch-dev at e81281d60 as it is, with the tests whose imports are all in this change or below it; the exceptions are listed under "What differs from multibranch-dev". Commits on top of the base merge:

1. feat(measure): add the measurement core's leaf modules: the modules and their tests.
2. build(hosted): list cryptography in the hosted extra: signing imports cryptography.
3. fix(measure): place a launcher in the checkout readable by its owner only: the one change CodeQL asked for (below).
4. feat(measure): add the result bundle check and the tests it lets run: bundle and the tests of spec, manifest and bundle.
5. chore(measure): bring the measurement core's leaf modules up to multibranch-dev e81281d60 (after merging the refreshed base): nine modules and four test files changed on multibranch-dev since 555e863eb (aiter_merge_hook, boot_watch, jit/builds, log_causes, manifest, phases, profile_traces, spec, surface; tests jit/tests/test_builds, test_manifest, test_profile_traces, test_spec) and take dev's version. spec now reads the short-prompt and long-context plans of the sheet, so the two plan modules and their package markers are added (short_prompts/plan.py, long_context/plan.py); profile_traces now asks hyperloom.orchestrator.trace_analysis whether a trace is a CUDA-graph capture sidecar, so that package's __init__ exports is_capture_fragment (the function is already on main, in _capture_shapes.py). Nothing was removed.

The modules (lines), and everything each imports from this repository:

Module	Lines	What it is	Imports
spec.py	689	the instruction sheet (MeasurementSpec v1) the application hands a runner, its sha256 and the fingerprint of its measurement content	contracts.measurement, docs, surface, short_prompts.plan, long_context.plan
manifest.py	425	the signed result manifest (MeasurementResult v1) a runner writes, read strictly	contracts.measurement, docs, jit_kinds
bundle.py	445	verifies a result bundle read from a sandbox (the manifest, its signature, the files it lists) and places the files in the output directory, checking each byte again as it is copied and landing all files or none	docs, fsafe, manifest, signing
signing.py	139	Ed25519 signing of the manifest bytes; cryptography is imported on first use	nothing
surface.py	229	the inputs that decide what a test measures, hashed into surface_sha256, and the checks that hold a test's YAML to them	contracts.recipe, common.outside_facts (#2155)
docs.py	175	the strict document readers the sheet and the manifest share	nothing
short_prompts/plan.py, long_context/plan.py (+ two __init__)	106 + 120 + 15 + 15	what the sheet asks of the short-prompt and long-context accuracy passes; the passes themselves are a later layer	docs; long_context.plan also short_prompts.plan
fsafe.py	163	symlink-proof file access under a directory the caller does not control	nothing
phases.py	260	where the time of a test goes, and how the phases add up	contracts.measurement
repro.py	163	runs the case of a repro test against a live server and keeps its verdict	contracts.measurement
boot_watch.py	230	a limit on how long a test's server may take to report ready	orchestrator.actions.cancel_channel (on main)
log_causes.py	163	what a failed test's logs say went wrong, in the log's own words	common.outside_facts (#2155)
returncodes.py, ports.py	34 + 39	the return codes only hosted measurement produces; the ports local measurement leaves alone	nothing
magpie_launchers.py	99	Magpie's launchers in the InferenceX checkout before a test's first round	common.benchmark_config (on main)
profile_traces.py	77	where a profile test leaves its torch-profiler traces, and whether any of them is of a serving step rather than a CUDA-graph capture	orchestrator.trace_analysis (is_capture_fragment, on main; exported by this PR)
aiter_merge_hook.py	161	start-up hook of a local test's server: aiter writes its merged tuned tables below the test, not below /tmp	nothing
framework/tables.py (+ __init__)	117 + 4	which tuned table each AITER_CONFIG_* variable names, read from aiter's source	nothing
jit/builds.py, jit/notes.py, jit_kinds.py (+ jit/__init__)	157 + 68 + 92 + 8	what a server's log says aiter built; the notes a test's use of the compiled-code cache leaves; the kinds of cache	nothing

Measured on a tree that holds only the base and this PR (no other measure or sandbox module from multibranch-dev): each of the 26 measure modules imports in a fresh isolated interpreter (python -I). No module imports socket, local, compile, farm, failures, archive, hosted or any other module of the measurement core above this layer, and none imports a module of hyperloom.sandbox.

Tests (306, all passing): test_spec (93), test_manifest (54), test_bundle (46), test_surface (23), test_phases (19), test_repro (19), test_signing (14), test_profile_traces (12), jit/tests/test_builds (11), test_magpie_launchers (9), jit/tests/test_notes (6), with the helper tests/bundle_factory.py that builds signed bundles and instruction sheets for the first three.

What differs from multibranch-dev

1. Everything is byte-identical to origin/multibranch-dev at e81281d60 (checked file by file: 26 modules and 14 test files, helpers and package markers included), except pyproject.toml and orchestrator/trace_analysis/__init__.py. The third commit is on multibranch-dev too, so that this stays true.
2. pyproject.toml: the hosted extra, which feat(sandbox): add the sandbox socket and the E2B provider #2154 defines with e2b, gains cryptography>=42 and a comment that says what each entry is for. On multibranch-dev the extra also lists boto3; that arrives with the PR whose module imports it (the result archive). With feat(sandbox): add the sandbox socket and the E2B provider #2154's test extra pulling in hosted, the test shards now install cryptography as a declared dependency; until now they had it only through claude-agent-sdk. The runner extra already listed it.
3. measure/profile_client.py (68 lines) is not here. It imports three names (benchmark_serving_path_in, ensure_benchmark_lib_patched, ensure_benchmark_serving_patched) from hyperloom.orchestrator.actions.executors, whose __init__ exports them on multibranch-dev and not on main: that export is one of the small edits to existing main files (the PR-0 kind) and comes with them. test_profile_client.py and its helper tests/_inferencex.py wait with it.
4. measure/tests/conftest.py and gpu_free_host.py (the fixture that makes a runner test independent of the GPUs of the host) are not here: no test in this change runs a runner or reads a GPU. They arrive with the first test that does.
5. orchestrator/trace_analysis/__init__.py (an existing main file): gains one export, is_capture_fragment from _capture_shapes (already on main), which profile_traces imports. On multibranch-dev the same __init__ also exports count_gpu_kernel_events, discover_trace_inputs and TRACELENS_REF for the toolbox; nothing here needs them, so they are not added.

What is not in this PR, and why

• The tests of the modules above that import modules which sit higher in the measurement core. Each arrives with the module it needs:
  • test_log_causes (293) and test_returncodes_and_failures (215): measure/failures.py.
  • test_boot_watch (220) and test_ports (95): measure/local.py and measure/socket.py.
  • test_aiter_merge_hook (252) and the two files under framework/tests/: framework/merge.py, base.py, rows.py.
  • jit/tests/test_key.py, test_locations.py, test_session.py, test_store.py: jit/key.py, locations.py, session.py, store.py.
  • test_spec_sections (53): hosted_spec.py.
• Production modules without a test file of their own here: log_causes, boot_watch, ports, returncodes, aiter_merge_hook, framework/tables, docs, fsafe, jit_kinds and the two plan modules (1408 of the 4193 lines). The plan modules are exercised through test_spec (the sheet reads them); their own packages' tests need the passes, which are a later layer. The first six wait for the tests above. docs, fsafe and jit_kinds have no dedicated test file on multibranch-dev either; here docs is exercised through the tests of spec, manifest and bundle, fsafe through those of bundle and jit_kinds through those of manifest. framework/tables is tested on multibranch-dev by the framework tests above (base and rows).
• profile_client.py and common/accuracy.py (64 lines): the second keeps the common directory at the size feat(common): add the rule pack and the outside_facts package that owns each outside-project fact #2155 left it (the open question on that PR) and has no reader below measure/accuracy.py.
• The rest of the measurement core (the plan on Agent stuck at Download model from HuggingFace #121 listed bundle with the hosted path, PR-6b; it moves here, see "Size"): the middle layer (failures, farm, attachments, compile, archive, framework base, rows, merge and copies, jit key, locations, store and session, pins, round_marks, accuracy), the upper layer (socket, local, hosted_spec, ...), the runner core (run, jit, inventory, rewrite, rounds, snapshot, platform_env, __main__) and the hosted path. They import these modules, and land after them in the order of the revised plan on Agent stuck at Download model from HuggingFace #121.
• .importlinter lines: multibranch-dev has none for measure, so none are added.

Size

4193 production Python lines in measure (plus 4 lines in orchestrator/trace_analysis/__init__.py) against the 1000-line diff budget (size-exception label, reason here), 2401 lines of tests. The reason is the layering, not a choice of grouping: these are the modules of the measurement core that import nothing above it, so they can be read and merged before the rest; the rest cannot be reviewed without them. bundle is here, and not with the hosted path, because the tests of spec and manifest build their documents with bundle_factory, which imports it, and it imports only docs, fsafe, manifest and signing; without it the two largest modules of the layer (1114 lines) would have had no test in the change. The largest file is spec.py (689 lines, module limit 1200), the next is bundle.py (445). The set splits cleanly if the reviewers prefer: spec, manifest, bundle, signing, surface, docs, fsafe, jit_kinds and the two plan modules with the tests of the first three (the instruction sheet, the result manifest and the bundle check: 2613 production lines) against the rest (the phases, the log readers, the local-test helpers and the jit and framework leaves: 1580 lines); spec needs surface, docs and the plans, manifest needs docs and jit_kinds, bundle needs docs, fsafe, manifest and signing, and nothing in the second half depends on the first.

CI state

Not re-checked at the refreshed head (533f6beb9): CI was still to run when this body was written. The earlier head (7cf88a191) was green except main's then-red collection error (fixed on main by #2149, which the refreshed base now contains) and the coverage jobs that followed it.

CodeQL flagged one alert on the first run (py/overly-permissive-file in magpie_launchers.py: the copy of a launcher was given mode 0o755, which is world readable); the third commit narrows it to 0o700, the owner who runs Magpie, as for the other files this layer creates, and CodeQL passes. The test that asserts the mode asserts the new one. The same change is on multibranch-dev, so this PR stays byte-identical to it.

Review template

• Linked issue(s): none.
• Tests: added, taken from multibranch-dev with the code (306 tests, see above). Run locally at 533f6beb9 on Python 3.10 and 3.11 (-n 4), on the tree of base plus this PR only: pytest src/hyperloom/measure src/hyperloom/contracts src/hyperloom/plugs src/hyperloom/sandbox src/hyperloom/orchestrator/trace_analysis scripts/tests src/hyperloom/common/tests/test_dedup_guard.py src/hyperloom/common/tests/test_llm_client_architecture.py src/hyperloom/inference_optimizer/tests/test_common_import_lint.py src/hyperloom/inference_optimizer/tests/test_packaging_lint.py: 2264 passed, 13 skipped on each (the measure tests of this PR alone: 306 passed on each; the skipped ones need a GPU, an envd binary, a checkout of the real aiperf client, node, vulture or complexipy). Also run: ruff check . and ruff format --check . (2406 files), lint-imports --no-cache (11 kept, 0 broken). Not re-run at this head: the wheel-contents check, Gitleaks, bandit and scripts/code_metrics.py (no jscpd on this machine); the results below are from 7cf88a191, and CI runs the gates.
• Findings that do not fail a gate: bandit (advisory in CI) reports B108 and B102 in aiter_merge_hook.py. B108 is the literal /tmp/aiter_configs that the hook names because aiter writes there and nothing else can redirect it; B102 is the exec that runs the sitecustomize.py this file shadows, which is the hook's job (it is read from sys.path as Python's site would have read it). The code-metrics design check A10 (advisory, never fails) notes spec.py:101 comparing the string 'confirm' to Purpose.CONFIRM; it is as on multibranch-dev.
• Size/complexity triggers crossed: the 1000-line diff budget, with the reason under "Size". No function crosses a code-metrics limit.
• If this refactors: not a refactor.
• Observable effect: none. Nothing on main imports hyperloom.measure, no CLI, entry point or flag uses it, and the legacy flow behaves as before. What exists afterwards: 40 new files under hyperloom/measure (26 modules, 14 test files, helpers and package markers), one more export (is_capture_fragment) from hyperloom.orchestrator.trace_analysis, and cryptography in the hosted extra (so in the test extra, where it was already installed through another dependency).
• Breaking changes: no.
• PR addresses single concern: yes (the leaf layer of the measurement core: the instruction sheet, the result manifest, signing and the modules they and the runner stand on).
• Root cause is upstream: no.

🤖 Generated with Claude Code

[github-actions]

Code metrics gate: PASSED

Judged: the 42 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension	Limit	New	Worse	Improved	Over the limit	vs base
Cyclomatic complexity	fails > 20	0	0	0	0	+0
Cognitive complexity	fails > 30	0	0	0	34	+0
Function length (lines)	fails > 60	0	0	0	249	+0
Nested block depth	fails > 4	0	0	0	37	+0
Parameter count	fails > 5	0	0	0	0	+0
Module length (lines)	warns > 800, fails > 1200	0	0	0	42	+0
Duplicated lines per file	fails > 0	0	0	0	76	+0
Dead code (vulture)	fails > 0	0	0	0	2	+0
Directory size (entries)	fails > 50	0	0	0	1	+0

Check	Findings	Effect
Comments	0
Broad catch without a reason	0
Broad catch that records nothing	0
English only	0
Production code imports test code	0
Repeated literals	0
A1 Private name of another package	0
A8 Flag argument	0
A9 Tuple return	0
A10 Raw vocabulary comparison	1	advisory, never fails
A11 Hidden global write	0
A12 Pure forwarder	0
A12 Single-implementation abstraction	0

A10 Raw vocabulary comparison (advisory, never fails): 1

Where	Problem
src/hyperloom/measure/spec.py:107	A10 raw vocabulary: compares the string 'confirm', the value of Purpose.CONFIRM; compare with Purpose.CONFIRM (convert the input to the enum once, where it enters)

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

_{Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, radon 6.0.1, ruff 0.16.2, vulture 2.16.}