[MaxGhenis]

Cuts the CI time of policyengine_us/tests/core/test_behavioral_response_measurement_copies.py (#9913) and keeps every case it ran. Only that test file and a changelog fragment change.

Main's push run 40031737740 spent 1,512 s of the Rest core group's 43:46 on this file; the group took 11:51 in run 38141803955 on 2026-10-11, before the file existed. Across the 57 Rest runs in the timing artifacts from 2026-10-29 19:45Z to 2026-10-30 23:52Z, the file took from 1,318 s to 1,819 s, with a median of 1,628 s. This PR's run 41143310235 took 1,177 s.

Where the time went

The file builds no Microsimulation: every test builds one-person California households. The time goes to copying policy.

• clone_spm_system deep-copies the policy system, including the whole parameter tree, whenever a test builds a reform simulation, twice per new measurement (the reform branch with clone_system=True, and _baseline_measurement_branch) and once per clone().
  • Under cProfile, building a reform simulation and measuring it once took 47.4 s, and the three copies took 25.7 s of that.
  • Within the copies, Parameter.clone (374,838 calls) took 20.5 s cumulative, including 12.9 s in ParameterAtInstant.clone (4.0 million calls).
  • In policyengine-core 3.33.2 both deep-copy their metadata (copy.deepcopy(self.metadata)), and Parameter.clone also clones each of its values. copy.deepcopy took 21.4 s over 15.9 million calls.
• One clone_spm_system call adds 6.9 million objects the collector tracks, and a full collection right after it frees none of them. Every full collection traverses every live copy. Over two rounds of building, measuring, cloning and measuring a reform simulation (12 copies), collection took 51% of the 115.6 s wall time at the default first threshold of 2,000, and full collections alone took 46%.
• test_a_copy_reports_the_response_for_its_inputs_and_measurement_state took a median of 662 s. Under max_examples=3, derandomize=True it ran four explicit pins plus three generated examples. With this test's old strategies, Hypothesis 6.168.3 and CPython 3.14, those three stayed the same on every run. Excluding the one-household references, each example builds a reform simulation, makes up to two measurements and makes one copy.

The parametrized tests stay, because each parameter exercises its own construction path. The [None]/[reform1] tests build simulations with and without a baseline. In the [entity]/[period] tests a reform changes a variable's entity or definition period, and the baseline measurement branch drops the holder that no longer fits baseline policy.

Changes

1. A module-scoped fixture raises the collector's first threshold from 2,000 to 100,000 while these tests run, and restores the previous thresholds at teardown.
   • The threshold applies process-wide, so it governs every collection in the module's pytest process while the module runs.
   • Over three rounds of building, measuring, cloning and branching a reform simulation, full collections fell from 28 to 9. Over the two rounds above, collection fell to 39% of 96.3 s wall time, and full collections to 29%.
   • Higher thresholds trade memory for time. In paired local runs of the file, 200,000 ran 10% to 11% faster than 100,000 and peaked 27% to 28% higher; 300,000 ran 15% faster and peaked 52% higher. This PR keeps 100,000, because neither clears at least 10% faster for at most 5% more peak memory.
   • At an earlier head, freezing the heap at module start (gc.freeze(), with the same threshold) ran 6.8% faster in two paired local runs (415 s against 445 s). This PR leaves it out: for less than a 10% gain, gc.freeze() takes every object alive when the module starts, from whichever modules ran before it in the process, out of collection until teardown.
2. The Hypothesis test runs only its explicit examples (phases=[Phase.explicit]). Four of the five pins build their source as one simulation of three households, at 20,000, 50,000 and 100,000.
   • When one of these pins changes a copy's wages, it gives the copy (50,000, 100,000, 20,000), so each household takes another's wages.
   • The fifth pin reruns the old third generated example: it measures a lone household earning 20,000 (zero response under this reform), clones the simulation and gives the clone wages of 50,000. Each three-household source holds nonzero responses, so only this pin reaches the all-zero state.
   • The checks stay the same. The test compares each clone, and each branch it creates before the source measures, household by household with a new one-household reform simulation at the same wages (fresh_response concatenates them). For a branch it creates after the source measures, it checks the source's measured responses scaled by wages, as d1021 requires, and a snapshot of the source's measurements.
   • Dropping phases makes Hypothesis generate examples from the new strategies, not the old ones: three households in one source, with the copy's wages a permutation of theirs.

The test now builds 8 reform simulations, makes 10 measurements and makes 3 clones, against 10, 12 and 4 before, counting the one-household references it builds and measures.

Coverage of each removed or merged case

Each pin keeps its kind and after_source_measured. Under this reform a household responds with 0 at 20,000, 315.82 at 50,000 and -52.12 at 100,000. Three households side by side in one simulation return those values in both wage orders the pins use (np.array_equal holds against one-household simulations), so a household in a pin stands in for the one-household case.

Old case	Kind, measured first, source → copy wages	What it covered	Now covered by
Pin 1	branch, no, 50,000 → 50,000	an early branch measures for itself, positive response	pin 1, household 2
Pin 2	clone, no, 50,000 → 50,000	an early clone measures for itself, positive response	pin 2, household 2
Pin 3	clone, yes, 50,000 → 100,000	a clone of a measured source remeasures its new wages	pin 3, household 2
Pin 4	branch, yes, 50,000 → 100,000	a branch of a measured source reuses its relative changes (d1021)	pin 4, household 2
Generated 1	clone, no, 20,000 → 20,000	an early clone measures a zero response	pin 2, household 1
Generated 2	branch, no, 100,000 → 100,000	an early branch measures a negative response	pin 1, household 3
Generated 3	clone, yes, 20,000 → 50,000, lone household	a clone of an all-zero source remeasures	pin 5 (same case)

The pins also run five household cases the old test never ran. They:

• clone an unmeasured source earning 100,000;
• branch an unmeasured source earning 20,000;
• measure a source earning 100,000, clone it and move the clone to 20,000;
• measure a source earning 20,000, branch it and move the branch to 50,000;
• measure a source earning 100,000, branch it and move the branch to 20,000.

Mutation check

A plugin patches the code before the test module imports it, and a runner calls every old case and every new pin one by one under each mutant, at the merge with main (9e5416c). Four mutants re-introduce the three defects the module docstring describes, the first in two forms. Two more break invariant 3 and a clone's own measurement.

Mutant	Old cases that fail	New pins that fail (households)
none	none	none
the mutant measures the baseline the pre-#9913 way, on a clone_system=True branch of get_branch("baseline")	pin 1, pin 3, gen 2, gen 3	pin 1 (2, 3), pin 3 (1, 2, 3), pin 5
the mutant measures the baseline under the reform (_baseline_measurement_branch returns get_branch("baseline"))	pin 1, pin 3, gen 2, gen 3	pin 1 (2, 3), pin 3 (1, 2, 3), pin 5
clone shares its source's measurements	pin 3, gen 3	pin 3 (1, 2, 3), pin 5
clone keeps its source's measurements when every cached labor-supply response equals zero	gen 3	pin 5
get_branch deletes the new branch's measurement cache	pin 4	pin 4 (1, 2, 3)
branch shares its parent's dictionary	none	none

• Every old failure reappears, with the same actual and expected values, as a failing household of the pin that now covers it. Under either baseline mutant, old generated 3 got 3,248.13 against 315.82, and so do pin 3's household 1 and pin 5.
• The last mutant survives both versions of this test. test_copies_own_their_measurements and test_a_branch_measures_a_new_period_without_touching_its_parent, both unchanged, fail under it.

Timings

JUnit reports cover the 57 Rest runs in the window above that reported this file: 19 on main and 38 on other PRs. All 57 head commits hold main's version of the test file (blob 58d6c5f2bf). The file ran alone in its process in 47 of them, and those 47 supply process elapsed and max RSS.

Measure	Before, median (range)	This PR at 9e5416c, run 41143310235	This PR at 6fdd639, run 41150119839
test_a_copy_reports_the_response_for_its_inputs_and_measurement_state (s)	662 (517 to 739)	434	397
The other 15 tests (s)	955 (802 to 1,083)	743	665
File total (s)	1,628 (1,318 to 1,819)	1,177	1,062
Process elapsed (GNU time)	27:52 (22:46 to 31:23)	20:35	18:37
Process max RSS (GiB)	6.08 (5.44 to 6.47)	6.70	6.70

• Run 41143310235 recorded 28% less file test time than the median: 34% less in the Hypothesis test and 22% less in the other 15 tests. Its process elapsed came in 26% under the median, and its file test time 11% under the fastest before run.
• Run 41150119839 recorded 35% less file test time than the median: 40% less in the Hypothesis test and 30% less in the other 15 tests. Its process elapsed came in 33% under the median, and its file test time 19% under the fastest before run. Between the two runs this branch changed only the fixture docstring, so their gap measures host noise.
• Each run's peak memory exceeds the before median by 10% and the before maximum by 4%.
• Shard 1 ran only this file in both runs. It took 21.0 minutes in the first, the shortest of six shards, against 33.4 for the longest, and 19.1 minutes in the second, again the shortest, against 28.6.

At 9e5416c, four pytest processes ran concurrently on a shared 18-CPU host in each of two local waves, and each process ran its tests serially.

First wave	pytest total (s)	Process wall (s)	Peak RSS (GB)
Before	837.9	857.3	6.73
Pins only	788.5	808.0	6.72
Pins + threshold 100,000	609.9	630.9	6.95
Pins + threshold 200,000	544.5	564.3	8.89

Second wave	pytest total (s)	Process wall (s)	Peak RSS (GB)
Pins + threshold 100,000	586.0	604.7	6.92
Pins + threshold 200,000	525.0	543.2	8.76
Pins + threshold 300,000	500.2	520.4	10.49
Pins + threshold 300,000	498.4	518.7	10.49

In the first wave, from before to this PR's version, pytest total fell 27% and peak RSS rose 3.3%. Each wave shared the host's load.

Invariants

The file's four invariants (module docstring) and the tests that check them stay the same. Only the Hypothesis test changes: which households one pinned run covers, plus the lone-household pin.

Shards

#10439 plans the Rest shards from the JUnit reports of recent successful release runs on main. After a successful release run on main records this file's new times, later plans can use them. No Makefile change.

impact: none (test file only; no variable, parameter or reform file changes)
axiom: n/a: test-only CI cost change.

🤖 Generated with Claude Code

[MaxGhenis]

The US + core hub now owns landing this PR.

GPT-6.1 Sol (Subfleet, hard tier) approved 97bc1b2 and then the delta to the current head, 6fdd639, with no blocking findings. Its verdicts sit in ~/reviews/pe-us-10440-copies-ci-2026-10-23/review-final.md and review-delta.md, and the evidence behind the PR description sits in rerun-2026-10-30/ in the same folder. The delta review's three voice notes went into the description.

All six Rest shards passed in run 41143310235 at 9e5416c, the merge with main, and a later push cancelled the rest of that run. Since then this branch has changed only the fixture docstring. Run 41150119839 at the current head has not finished.

This PR depends on no other PR and carries no hold or decision. Besides CI, only the required approving review blocks it, so it can land with gh pr merge --admin --match-head-commit 6fdd63922122b3bb1c4828318d0dfe726ff22779 once gh pr checks exits 0, plus an audit comment naming both reviews.

One follow-up remains: when run 41150119839 finishes, add its Rest timings as a second "after" column in the Timings table. Adding the run's id to AFTER in rerun-2026-10-30/ci_stats.py computes them from its downloaded artifacts.

[MaxGhenis]

US + core hub merge audit, us#10440 at 6fdd63922122b3bb1c4828318d0dfe726ff22779 (squash): cut the behavioural-response copy tests' CI time.

• Reviews: GPT-6.1 Sol on Subfleet hard-tier lanes approved 97bc1b244e, then the delta to this head, which changes only the fixture docstring (review-final.md and review-delta.md, named in the hand-off comment on this PR).
• Gates checked live at merge:
  • gh pr checks exits 0;
  • the latest Pull request run at the head (41150119839) concluded success;
  • MERGEABLE;
  • not a draft;
  • no reviewer's latest review requests changes;
  • head pinned.
• Overlap with main: main has moved since this head's CI base. Its changes don't touch this PR's test file, and the merge-tree is clean.
• Impact: none. The PR changes one test file and a changelog fragment.
• Timings: the body's second column adds run 41150119839: 1,062 s of file test time against a before median of 1,628 s, and 6.70 GiB peak memory against a before maximum of 6.47 GiB.
• Axiom: n/a, test only.

[MaxGhenis]

PolicyEngine/policyengine-core#631 merged, and it lowers what these copy tests cost. policyengine_us/tests/core/test_behavioral_response_measurement_copies.py (16 tests), user plus system CPU and peak RSS from /usr/bin/time -l, one venv, policyengine-us bdd08b8d69, two rounds per cell with the order of the sides swapped between rounds:

	Core master	Core #631
Default collector thresholds	633–862 s, 6.4 GB	563–614 s, 6.3 GB
First threshold 100,000 (this PR's setting)	550–624 s, 7.5 GB	393–453 s, 6.4 GB

The ranges span the two rounds. The host stayed busy, so read them as ranges rather than a percentage. These tests pick up the change once policyengine-us installs a Core release that contains it.