Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The sweep's all-government total and each state's net change now pay for health coverage and count TANF once. policyengine-us keeps health coverage out of household net income, so household_benefits held no Medicaid, CHIP, Medicare Savings Program, ACA premium tax credit or state health program, and it counts Massachusetts's TAFDC twice (through tanf and through household_state_benefits). - analysis/health_tanf.py prices each household's coverage with healthcare_benefit_value and splits it by payer: Medicaid, CHIP and MSP by federal matching (federal_benefit_cost, state_benefit_cost), with coverage federal law won't match charged wholly to the state; ACA credits and the Basic Health Program federal; state programs to the state. TANF goes to the states, since the block grant is fixed. It refuses any policyengine-us whose health or benefit lists it would misclassify. california_shift now uses it, so one implementation serves 1.764.6 and 2.x. - fiscal.py reports total_change_with_health_tanf and the identity that the site's federal chart, its state charts and the taxes outside every chart add up to it. - compute_shift_sweep prices every shifted branch's Medicaid enrollees at the baseline's cost per enrollee, makes total_rev_change_b and each state's state_net_change_b count health and TANF, keeps the old measures under explicit names, and records the programs it counts. - labor_inputs.py sets the stored *_before_lsr labor inputs with the built variables, so TANF's default earned-income list and SNAP's self-employment income see the wage cut. The sweep data and site pages follow in later commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
decile_impacts(microdata_dir) reads the scenarios its manifest lists, so a sweep run can write its household files to a temporary folder and still fill metadata.baseline_facts.decile_impacts. A household whose cumulative weight lands exactly on a decile boundary now stays in the lower decile; before, it moved up, which left decile 1 empty when weights were equal. Ties in market income keep row order. Tests: a hand-checked example, refusals, and Hypothesis properties (deciles ordered by market income, each within one household's weight of a tenth of the total, the baseline changes nothing, weighted changes add up). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
src/utils/sweepRevenue.js builds the federal and state chart rows. For a run that counts health coverage and TANF (rows carry health_federal_change_b), the federal chart gains a health band and leaves TANF to the states, and each state's chart gains TANF and health bands, mirroring federal_chart_change and states_chart_change in analysis/fiscal.py. Rows from older runs keep the bands and copy they were published with. The revenue summary now reads correctly when revenue falls at every step or bottoms out before the halfway point. Tests: hand examples, a seeded property test that the charts and the taxes outside them add up to each total, a check that every state's chart total matches the sweep's state_net_change_b, and a check against the sweep's chart identity residual once the data file counts health coverage and TANF. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
analysis/compare_shift_sweeps.py tests a sweep that counts health coverage and TANF against the identities fiscal.py and compute_shift_sweep.py build into it, and splits each shift's change from an earlier run into three parts: the change on the earlier definition, the TANF counted once, and health spending. Its tests build sweeps the way compute_shift_sweep builds them, plant a breach for each check, and check the site's own sweep once it counts health coverage and TANF. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e program household_state_benefits holds Massachusetts's TAFDC at its full amount for every eligible family, while tanf holds it only for families that take TANF up. The frame took the copy to drop from tanf, so the totals kept TAFDC for eligible families that don't take TANF up, and charged Massachusetts for it. The frame now sums the listed programs themselves (review r1, finding 1). Each health and TANF quantity now has one name at both levels: federal_health_change_b, state_health_change_b and tanf_in_state_benefits_change_b (N1). The landing check tests more national fields against the states' rows and each state's health total. The docstrings now say the immigration-status charge overstates the state's cost, name the TANF block grant as the reason the states pay TANF, and drop TPS from the emergency-care-only list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…charts - The state notes and the state ranking now say we charge a state the federal share for enrollees whose immigration status bars federal matching, and that the charge overstates its cost, since federal funds still match their emergency care (review r1, finding 2). - The TANF notes separate the law from our attribution: federal law fixes each state's TANF block grant, so we charge TANF changes to the states (finding 3). - Share mode stays off for runs that count health coverage and TANF, since its baseline counts neither; the state ranking's share mode leaves out states whose baseline doesn't exceed zero, as the decomposition already did, and its note drops the passives (finding 4). - A state's health note names a program it runs on its own only on that state's chart, from the sweep's metadata (finding 5). - The revenue description names the three groups of taxes outside the charts (finding 6). - The revenue summary gives health spending as its second sentence for runs that count it, reads the sweep's last shift, and handles a sweep that never falls below zero (finding 7). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
compare_shift_sweeps now sets the rerun's net and market Gini, top 10% and 1% net income shares and SPM poverty rate beside the earlier run's at every shift, and each decile's change in mean net income at a 10% shift with its baseline mean. It names the decile tie-break that d0fec52 changed: ties in baseline market income now keep row order, and a household that lands exactly on an edge joins the lower decile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and TANF emit_paper_values.py and paper_figures.py read the mechanism sweep's total_rev_change_b, which counts health spending and TANF once the rerun lands. sweep_net_revenue now reads total_rev_change_published_definition_b where a sweep carries it: taxes less refundable credits and benefit outlays, the measure the paper's methodology section defines. compare_sweep_vintages reads the same measure, so it compares sweeps from either side of the change. On the committed site data, emit_paper_values.py still writes values_generated.tex and tables_generated.tex byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review round 2 at d176e09: - stateLineNoteHealthTanf read a program's state from the first word of its name, so on releases that list assigned_ca_premium_subsidy, assigned_co_premium_assistance and assigned_nm_premium_assistance, California's and New Mexico's notes dropped their programs. healthProgramState strips the prefix; a state with named and unnamed programs reads "OmniSalud and other health programs it runs on its own". The tests use STATE_HEALTH's real names. - The revenue summary's health sentence says "we estimate" and points back to the shift the first sentence ends on. - The revenue description names New York City's and Maryland's local income taxes as the ones the state charts hold. - compare_shift_sweeps prints n/a for a decile that compute_decile_impacts gives no pct_change. Nothing here changes the sweep's computation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-core 3.30.3 copies every cached array into each branch that get_branch makes, and policyengine-us 1.764.6's formulas keep about a dozen branches alive (itemizing, not_itemizing, no_salt, state credit branches). On the full microdata the sweep's baseline stage passed a 39 GB footprint while ctc_limiting_tax_liability copied the cache into its no_salt branch, and heavy_run's ceiling stopped it. analysis.branch_arrays ports policyengine-core#556 (c991b38b2) onto 3.30.3: a branch holds read-only views of its parent's arrays and copies one the first time it reads it. Core releases after #556 also change results (#560, #562), so the pinned runtime keeps 3.30.3. #556 alone lets an in-place write into a parent's array after branching reach a branch that hasn't read it yet, and policyengine-us 1.764.6 has one: ga_deductions runs `itemized -= ...` on the cached itemized_taxable_income_deductions. So the parent copies on first read too, and branching drops its fast-cache entries for shared arrays. With freeze_sources (the sweep's setting) branching also marks the shared arrays read-only, so a write through a reference taken before the branch raises instead of changing a result. run_shift_sweep installs it for its run and records it in the metadata. On the full microdata the baseline stage now peaks at 16.2 GB and finishes in 204 s, with no frozen-source error. Tests: example tests for sharing, first-read copies, isolation both ways, snapshots, nested branches, put/delete marks, direct clones and the intended remaining difference; a Hypothesis differential (150 step sequences per freeze setting) against core's own copying branches; and a policyengine-us check on NY, VA, DE and GA households that compares every array in every branch, byte for byte, with copying branches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-us 1.764.6's ut_personal_exemption also adds into a cached array (ut_total_dependents); policyengine-us#9739 fixed it and ga_deductions upstream. Docstring only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 3 found that the parent's copy on first read splits an array core keeps as one: a write into the parent's copy after a branch misses a formula's reference from before the branch, and misses a second variable core stored the same array under. Freezing can't catch it, since nothing writes through the frozen array. The docstring's claim that a finished run reads what full copies give, byte for byte, was false in general. The docstring now names the two ways a run can still differ (a view made before the branch, and that split) and says why neither happens in the sweep: policyengine-us#9739's scan finds five formulas in 1.764.6 that write in place into arrays they read, and none can do either. The tests pin both as intended differences, against core 3.30.3's own behavior. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
python -m analysis.check_branch_arrays draws households from the pinned microdata, runs run_shift_sweep at 0 and 10% twice, once with core's full copies and once with shared arrays, and compares every array in every branch, the result, the household microdata and the decile impacts. It fails on any difference, and when either run shows the comparison tests nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The published sweep ran on the pinned runtime and microdata, but counted Head Start in net income and set only the built labor variables. python -m analysis.diagnose_sweep_change reruns 0 and 10% with that code put back, checks that it reproduces every published value, and splits each change into Head Start, the stored labor inputs, TANF counted once and health coverage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by base Review r3's findings on the port: - F7: making a branch drops each parent fast-cache entry built on the base array of an array the branch shares, so a formula's 0-dim result, which the fast cache holds while the storage holds a reshaped view of it, no longer lets the parent read around its copy. - F6: uninstall gives each storage that still shares an array its own get, put and delete, so a simulation made inside the block keeps copying on first read after it, as it would under core. - F5 and F1: the docstring gives the core releases after 3.30.3 that can change results, names every array over shared memory that a write could reach (views, a formula's 0-dim result, a flat-file dataset's frame), and lists policyengine-us#9739's five in-place formulas with why none reaches a shared array in the sweep. - F2: the policyengine-us test adds a Georgia adjustment and a Utah newborn and runs under both freeze settings, so policyengine-core#556 alone now fails it on values (and, with frozen sources, on ga_deductions' write). - F8: new tests cover masked arrays, enum arrays, branches with their own system, direct holder reads, a yearly input that core stores under every month, and simulations used after the block. Each of seven mutants of the port fails at least one test (#556 alone, a copy the storage doesn't keep, shared masked arrays, no copying after uninstall, no fast-cache drop, fast-cache matching by object, no freezing). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shiftSweepData.json now holds the full-microdata rerun from 706a58c on policyengine-us 1.764.6. Its net government revenue counts what governments spend on Medicaid, CHIP, the Medicare Savings Programs, ACA premium tax credits, the Basic Health Program and state health programs, and counts each family's TANF once; each state's net change counts its share. The rerun's bundle records the Head Start exclusion, so the page drops the note for sweeps from before the September 2026 correction, and IncomeShift.headStartNote.test.jsx keeps that note covered for an older sweep. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9 of 10 tasks
The copy for a run that counts health coverage and TANF now names who acts: Congress sets each state's TANF block grant at a fixed amount, and the federal government still matches emergency care for enrollees whose immigration status bars federal matching. It carries #51's wording for the taxes outside the charts and for the local items in the state figures, and no sentence runs past 32 words. analysis/tests/fixtures/state_health_programs.json lists every program in health_tanf.STATE_HEALTH with the state that runs it. test_health_tanf.py checks the list against STATE_HEALTH, and ShiftSweep.test.jsx reads it in place of a hand copy (review r3's F10). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
count-health-tanf-d2671
branch
from
November 10, 2026 15:10
100fe5f to
09fb09a
Compare
Review r3's F8 listed paths no test covered. The property-based differential now mixes core's cache purge (_invalidate_all_caches, which apply_reform runs) into its steps. A fixed case purges a parent after branching, writes into an input the purge put back, and checks both trees against core's copies. A string variable with no max_length, which core stores as an object array, shares until first read and reads as core's after a parent write. All seven mutants of the port still fail at least one test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The comparison report named a pre-rebase commit for the change to baseline_deciles' tie and edge rules, and printed the note beside any earlier run. It now prints the note only when the earlier run records no health_tanf_programs, since this pull request brings both, and says what changed: ties in baseline market income now sort by row order, not by NumPy's default sort, and a household whose cumulative weight lands exactly on a decile edge now goes in the lower decile, not the upper one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Same code and words; Prettier rewraps seven spots in ShiftSweep.jsx, its test and the metadata test. One of them matches the rewrap on #51. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iors The module docstring now says that two of the parent's reads differ from core's with no write at all. After branching, the parent reads a formula's 0-dim result as the (1,) view its storage holds, and it reads a variable its own system neutralized after caching as the variable's default. Tests pin both, and the differential's _contents now compares shapes too. Four tests cover behaviors that no test reached: a parent write into an enum after branching, an input set after the block over a key the parent still shares, a branch that sets an input again after making its own branch, and nested blocks with different freezing. _keep_copying now stores functools.partial objects instead of bound methods, so a storage that still shares after the block survives pickle and copy.deepcopy, and a test round-trips one. A country-template test runs check_branch_arrays' tree walkers on a branch of a branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #51's credit of EZ Save to the Los Angeles Department of Water and Power, the AI scenario note that names the selected wage variant, the reworded federal-line and ranking sentences, and the state exposure script's docstring and scope note. ShiftSweep.jsx conflicted in two places. This branch keeps its OUTSIDE_CHART_TAXES_NOTE and the health and TANF notes around it, with #51's last sentence ("We put New York City's and Maryland's local income taxes in those states' charts instead."). The state ranking description keeps this branch's two forms, with and without health coverage and TANF, in #51's words ("We count state SSI supplements...", and California's LADWP and Los Angeles County programs) in each. At this commit, eslint, vitest (12 files, 51 tests) and the CI-like pytest run (280 passed, 66 skipped) all pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- C3: the federal note for a run with health coverage and TANF drops the
outside-chart taxes sentence, which the description above the chart
already gives.
- C4, C10: the revenue summary starts "We estimate", gives amounts in
prose with a verb for the sign ("falls by $970B at a 50% shift and by
$2,011B at a 100% shift"), names a low point that comes before the end
and where the sweep ends, and writes "an 80% shift". The health
spending sentence drops its second "we estimate".
- C6: "We put New York City's and Maryland's local income taxes in those
states' charts instead."
- C11: "at the baseline cost of an enrollee".
- C12: "and on its own OmniSalud program".
- C8: state_only_coverage's docstring says the federal test leaves out
the election in 42 USC 1396b(v)(4)(A), and why the published run moves
no one it covers: build p assigns no one TPS and leaves
years_since_us_entry at 5, so only undocumented and DACA enrollees move.
Tests pin the new sentences and cover the summary's 80%, rebound, early
low point and mixed-sign forms. eslint, vitest (12 files, 52 tests), the
engine suite and the CI-like pytest run (280 passed, 66 skipped) pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 5's property run found false claims in cases the live sweeps don't reach: "rises by $0.0B" for an end at the baseline, "the fall narrows to $5.0B" after a $5.0B low point, "$10.0B" for $9.96B, "never falls below its baseline and falls by" for a sweep that skips 0%, and "a 8.5% shift". revenuePath now picks every verb and the low point from the amounts as proseBillions prints them, takes the last row at the low point, says when an amount stays at or returns to the baseline, and takes the article from a shift's whole number. The live US and UK summaries read as before. A seeded property test checks 20,000 sweeps against an oracle written apart from the component, and nine pinned cases cover the edges. Six of seven mutants that bring back the old behaviors fail it; the seventh is equivalent. The health sentence says "the same" for spending that prints as $0.0B, the health run's test checks that the federal note leaves the outside-chart taxes to the description, and state_only_coverage's docstring says policyengine-us 2.25.0 models the lawfully-residing election. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict, in the state ranking description: this branch keeps its two forms of the measure, with and without health coverage and TANF, and takes #51's closing sentence, which names only San Francisco's Working Families Tax Credit among California's local items. eslint, vitest (13 files, 55 tests) and the CI-like pytest run (280 passed, 66 skipped) pass at this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK sweep's callout now ends with #51's full sentence on what the UK total counts. A test renders the UK data file and pins the whole callout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The round-5 code review found a third read where the port's parent differs from core's with no write. Core's Simulation.delete_arrays with a period drops only that period's fast-cache entry, and Holder.delete_arrays drops none. So after a branch, a delete of a year or a delete through the holder leaves core serving the deleted month from its fast cache. Here the branch dropped the parent's entry, so the parent finds nothing stored and computes the month again. The sweep deletes no arrays. policyengine-us 1.764.6 calls Simulation.delete_arrays only without a period. It calls Holder.delete_arrays only while it builds a simulation, before the simulation has any branch, and in BranchedSimulation, a class it imports but never uses. - branch_arrays: the docstring now lists three no-write differences and says why the sweep reaches none of them. It also names the run that would set include_ss_benefits_as_earnings, and _keep_copying's docstring drops two trailing which-clauses. - test_branch_arrays: two new tests pin the reads. A delete of the year or through the holder reads core's old value under core and the new one here. A delete of the month or of every period reads the same both ways. The differential's delete step now picks every period or one month. At 1,500 examples per setting it passed, with 305 and 318 examples that delete a month after a branch. The module docstring counts the third difference and lists the three in-place write cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
state_only_coverage moves the whole federal share policyengine-us books for an enrollee whose immigration status the federal test doesn't match. The test leaves out the election in 42 USC 1396b(v)(4)(A), under which a state can cover lawfully residing children and pregnant women with federal matching. For the election, CMS's letter SHO #10-006 counts TPS holders as lawfully present, and SHO #12-002 excludes DACA recipients. Undocumented immigrants hold none of the statuses SHO #10-006 lists. Microcosm's build p assigns no one TPS and leaves years_since_us_entry at its default of 5, so the run moves only undocumented and DACA enrollees. A new check, check_moved_statuses, raises if anyone else would move. A later build that imputes TPS or entry years then stops the run before it charges a state for coverage that federal funds may match. The docstring now cites SHO #10-006 and #12-002 for the election in place of the Marketplace rule, 45 CFR 155.20. HHS wrote that rule for Exchanges and the Basic Health Program. New tests check the guard with examples and a property test, and a simulation checks that the move refuses a California TPS child's federal share. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For a run that counts health coverage and TANF, the summary gave the path and then "At that shift, the federal government spends ...". After a low point, the path's last sentence already opens on the last shift, so two sentences opened on the same shift. revenuePath now takes an end clause. After a low point, the clause joins that sentence: "At a 100% shift, the fall narrows to $40B, and the federal government spends ...". In every other form, it follows "At that shift,". The property test now words each of its 20,000 sweeps with and without a clause. It checks that the clause leaves the path's own words alone and that only one sentence opens on a shift. New pinned cases cover a low point at 50%, two equal minima, and the "an" before 11% and 18%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…age after deletes The port's docstring and tests listed the parent's reads that differ from core's with no write as a closed set, and now state the rule behind them. After a branch, the parent reads each array the branch shares through Simulation._calculate, while core serves it from the fast cache, so wherever the two paths disagree, the parent gets _calculate's answer. We know three cases where they disagree, and the tests pin each one. The delete pins now also check what the parent stores, and they run with and without new salaries. After a delete of the month's year, or one through its holder, core serves the deleted month from its fast cache and stores nothing. The port computes the month again and stores it, with the same value when no input changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…government funds Round 6's health review found the state-only move charged states for coverage no state funds. policyengine-us 1.764.6 books CHIP for a DACA child in any state, since its CHIP test bars only an UNDOCUMENTED label, and it gives the Medicare Savings Programs no immigration test. The move charged Texas $12,114.80 for an undocumented senior's Medicare Savings Program benefits and Florida $3,072.99 for a DACA teen's CHIP. For people whose status fails the federal test, state_only_split now sorts each share. Medicaid's federal share moves to the state, since policyengine-us enrolls such people in Medicaid only through a state rule. It stays federal for a pregnant woman in a state that covers unborn children through CHIP, with income under that option's limit. CHIP's federal share moves where the state's undocumented_immigrant rule covers the person. Elsewhere, both shares of CHIP and of the Medicare Savings Programs leave both lines, and the frame records them as unfunded_federal and unfunded_state. The frame also leaves out Colorado's OmniSalud. policyengine-us 1.764.6 adds the benchmark premium to co_omnisalud for a citizen whose undocumented spouse the program would cover, on top of the citizen's premium tax credit. It adds nothing for the undocumented adult the program covers. The Healthier Oregon overlap now also drops the cost for anyone policyengine-us enrolls in CHIP or who doesn't take Medicaid up. The sweep exports the unfunded and excluded changes, and compare_shift_sweeps checks that health spending equals healthcare_benefit_value less the overlap, the excluded programs and the unfunded coverage. New engine tests pin each case the review reproduced, and the guard now has tests for a Texas permanent-resident child in the five-year bar and a Texas refugee child in 2027. The site's notes now say we likely overstate a state's cost, and they name the pregnancy exception and leave OmniSalud out. The published sweep still reflects the old accounting until the rerun. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Counts health coverage and TANF in the income-shift sweep. The sweep's totals and state lines now count Medicaid, CHIP, the Medicare Savings Programs, ACA premium tax credits, the Basic Health Program and state health programs. We split that spending between the federal government and each state, and we charge TANF to the states. Stacked on #51.
policyengine-us keeps health coverage out of household net income, so
household_benefits, and the revenue lines we built from it, counted none of it.household_benefitsalso counts Massachusetts's TAFDC twice: throughtanffor families that take TANF up, and at the full amount throughhousehold_state_benefitsfor every eligible family. We drop the second copy from the new totals and count each family's TANF once, astanfpays it.Plan
analysis/health_tanf.py) for the sweep and the California scripttotal_rev_change_band each state'sstate_net_change_b. Keep the old measures under explicit names, and record the programs counted*_before_lsrlabor inputs with the built variables, so TANF's default earned-income list and SNAP's self-employment income see the wage cutanalysis/branch_arrays.py), so the rerun fits under our job runner's memory ceilingsrc/data/shiftSweepData.jsonanalysis/diagnose_sweep_change.py)analysis/check_branch_arrays.py)compute_decile_impacts.decile_impactsreads any foldermicrodata_exportwrites, so the rerun computes them from microdata in a temporary folder and deletes it before it saves the sweepsrc/utils/sweepRevenue.js,ShiftSweep.jsx): for data that count health coverage and TANF, the federal chart gains a health band and leaves TANF to the states. Each state's chart gains TANF and health bands, and the method copy says what each band holds. Data from older runs keep the bands and copy we published them withemit_paper_values.pyandpaper_figures.pyreadtotal_rev_change_published_definition_bwhere a sweep carries itThe two unchecked runs wait for disk space on our shared machine.
Scenarios, the state ranking and the paper follow in a separate PR, which waits on #45. That PR also takes two items from review round 1: it drops
ma_tafdcfrom the runtime'shousehold_state_benefits, and it moves the Additional Medicare Tax into the federal chart's payroll band (finding 10).The rerun
We reran the sweep at 706a58c on policyengine 5.0.1, policyengine-us 1.764.6 and Microcosm build p. The data record that commit as
metadata.code_commit, and the tagshift-sweep-run-706a58ckeeps it reachable once this branch merges. Since then, we have edited copy inanalysis/and added checks and tests. We also changed how the port handles two cases the sweep never reaches: a formula's 0-dim result, and a simulation used after the port's block ends.python -m analysis.compare_shift_sweeps src/data/shiftSweepData.json --expect-commit 706a58cexits 0 at 54a85f3. Every identity holds, and the largest chart identity residual rounds to $0.0000B.At a 10% shift, we estimate net government revenue falls by $195.2B, against $135.3B in the published sweep. Counting each family's TANF once adds $0.2B, all of it from Massachusetts's TAFDC, and counting health coverage subtracts $66.2B. Under the published sweep's definition, the code changes between the runs shrink the loss by $6.2B.
For example, at a 10% shift we estimate California gains $2.2B in state revenue, against $5.0B in the published sweep: TANF costs it $0.1B and health coverage $2.7B.
In the baseline, we estimate a net Gini of 0.4754, against 0.4699 in the published sweep. Mean net income falls in the bottom deciles, most in the second, from $44,183 to $40,949. Market income and SPM poverty don't move. Since da2ae15 (25 Sep 2026), our code leaves Head Start and Early Head Start out of household net income, and the published sweep (#38, 30 Jul 2026) counted them. We split the revenue, Gini, poverty and decile changes with
diagnose_sweep_changeinto four steps: Head Start, the stored labor inputs, TANF counted once and health coverage. Its runs wait for disk space.The comparison report at 54a85f3
Largest chart identity residual: 0.0000B
Every identity holds.
Rerun by shift ($B, 2026). The health and TANF columns flip the sign of the spending change, as the charts do.
Lowest total: -2,011.3B at a 100% shift.
Change from the earlier run ($B):
CA chart ($B):
Distribution measures: the rerun, and its change from the earlier run:
Decile impacts at a 10% shift: change in mean net income (%), and each decile's baseline mean net income:
The earlier run's code left the order of ties in baseline market income to NumPy's default sort.
baseline_decilesnow breaks those ties by row order. It also puts a household whose cumulative weight lands exactly on a decile edge in the lower decile, where the earlier code put it in the upper one. An edge that falls inside a tie, such as the households with no market income, can split it differently from the earlier run.Run metadata that differs (earlier → rerun):
metadata.branch_arrays: null → {"shared_until_first_read": true, "port_of": "policyengine-core#556 (…metadata.code_commit: null → 706a58cmetadata.description: Positive employment and self-employment income are reduced by the sel… → Positive employment and self-employment income fall by the selected s…policyengine_bundle.legacy_input_renames: null → {}policyengine_bundle.net_income_excluded_benefits: null → ["early_head_start", "head_start"]Fitting the sweep in memory
Our job runner stopped two reruns in their baseline stage. The first passed the runner's default ceiling of 20,480 MB at a 21,504 MB footprint, and the second reached 39,936 MB against a ceiling of 36,864 MB. policyengine-core 3.30.3, the core the pinned runtime runs, copies every cached array into each branch
Simulation.get_branchmakes. policyengine-us 1.764.6 makes branches inside its formulas (itemizing,not_itemizing,no_salt, and Delaware's, Virginia's and New York's credit branches) and keeps them for the life of the simulation. In a profile of the baseline stage, the footprint passed 34 GB asctc_limiting_tax_liabilitycopied the cache into itsno_saltbranch.policyengine-core#556 fixed this upstream on 2 Oct 2026 and first shipped in core 3.32.12. That release also carries every change since 3.30.3, and some of those can change results. For example, 3.30.4 makes
Microsimulation.calculate_addandcalculate_dividehonormap_to, and 3.32.8 uprates an input from its latest known earlier period. So we keep the study on 3.30.3, andanalysis/branch_arrays.pyports #556 onto it and refuses any other core.run_shift_sweepturns the port on for its whole run:freeze_sources, so making a branch also marks the parent's shared arrays read-only. A formula that writes through a reference it took before the branch then raisesValueErrorinstead of changing a result.Simulation.clone(), such asderivativemakes, still copies every array.With the port, the rerun peaked at a 16,521 MB footprint against a ceiling of 24,576 MB. It ran to the end, so no write raised
ValueError. Each output records the setting undermetadata.branch_arrays.Review rounds and what changed
Rounds 1 to 3 read the branch before its rebase onto #51; the tables give the rebased hashes.
Round 1 (at 277d496):
tanf_in_state_benefitssums the state TANF programshousehold_state_benefitslists, ungated, sotanf_chargeturns negative for Massachusetts families who skip TANF, and Massachusetts's line keeps the TAFDC the model pays. An engine test with a Massachusetts family that skips TANF fails on the old code. The generators draw the copy independently oftanf.compare_shift_sweepsreports them beside the earlier run's.Round 2 (at e99ac6f), all in e5c099e:
assigned_healthProgramStatestripsassigned_, and a test maps all 11STATE_HEALTHprograms to their states. 09fb09a moves that list into a fixture that the Python and JS tests both read.Nonedecile cell would crash the reportRound 3 (at 821f46d, then 7f97441 and db91f80):
map_tochange and 3.32.8's uprating change.STATE_HEALTHby handanalysis/tests/fixtures/state_health_programs.json, which a Python test checks againstSTATE_HEALTH.Round 4 (at db26550 and 54a85f3): in progress.
Disclosures
Deciles. The published sweep's code left the order of ties in baseline market income to NumPy's default sort. It also put a household whose cumulative weight lands exactly on a decile edge in the upper decile.
baseline_decilesbreaks those ties by row order and puts that household in the lower decile. So an edge that falls inside a tie, such as the households with no market income, can split it differently from the published decile impacts.Massachusetts families who skip TANF. Until the follow-up PR drops
ma_tafdcfromhousehold_state_benefits, their TAFDC stays in household net income, as in the published sweep, so Gini, poverty and the deciles still count it. The revenue totals here don't.The California run (Add a California breakdown of the labor-to-capital shift #49).
analysis/outputs/california_shift_pe6.16.0.jsonpredates the TAFDC fix in round 1's finding 1.household_state_benefitslists no California TANF program, so its California figures don't depend on the fix; its national figures with health and TANF could move on a rerun.CHIP and DACA. policyengine-us 1.764.6 enrolls DACA children in CHIP, and the sweep charges their federal share to the state. The pinned microdata hold 220 DACA records, one of them under 19 (age 17, weight 8,917), so the charge reaches about 8,900 children at most.
Formulas that write into arrays they read. policyengine-us#9739's scan finds five such formulas in policyengine-us 1.764.6, and none writes into a shared array that anything else holds:
basic_income_phase_inwrites only underinclude_ss_benefits_as_earnings, which the sweep leaves off;agi_surtaxreform off;nj_gross_incomeadds into the new arrayaddbuilds from its eight sources;ga_deductionssubtracts an adjustment the pinned microdata don't carry, so it subtracts zero;ut_personal_exemption, the only formula that readsut_total_dependents, reads and writes it in one call, and nothing makes a branch in between.policyengine-us#9739 made
ga_deductionsandut_personal_exemptionbuild new arrays on 2 Oct 2026. In its 2026 baseline run, no tax, benefit or net-income total moved, only the reportedut_total_dependents. Both writes ran in the published sweep too.The paper.
paper/values_generated.tex,paper/tables_generated.texand the figures still come from the published sweep. On the rerun's data,emit_paper_values.pychanges 7 macros invalues_generated.texand 11 lines oftables_generated.tex. For example, the paper's revenue change at a full shift goes from -790 to -676 ($B), and its baseline Gini from 0.470 to 0.475. In the follow-up PR, we regenerate both files and the figures and decide how the paper counts health coverage.Invariants (Hypothesis property tests)
healthcare_benefit_valueprices, less the Healthier Oregon overlap. Moving state-only coverage to the state never moves more than the federal share it comes from.compare_shift_sweepschecks 11 fields on the rerun).household_state_benefits− health spending.tanf− the copy, so the copy plus the charge equalstanf, state by state.set_input, calculating, deleting, in-place writes and cache purges, 150 withfreeze_sourcesand 150 without. It compares every value read and every array held with a run on full copies.itemizing,not_itemizingandno_saltbranches exist and that some arrays stay shared.python -m analysis.check_branch_arraysrunsrun_shift_sweepboth ways at 0 and 10% on households it draws from the pinned microdata, and compares every array in every branch, the result, the household microdata and the decile impacts. Its run waits for disk space.src/utils/sweepRevenue.test.js, a seeded generator over 500 cases): the federal chart, every state's chart and the taxes outside them add up to the total, with and without health coverage and TANF. Each state's chart total equals thestate_net_change_bthe sweep writes.Tests
At 54a85f3:
pytest analysis/tests -qwith numpy, pandas, pytest and hypothesis, as CI installs them: 280 passed, 53 skipped.test_branch_arrays.py,test_health_tanf.py): 59 passed.bun run test:run: 12 files, 50 tests passed.bun run lint -- --max-warnings=0: clean.🤖 Generated with Claude Code