Skip to content

Remove retired dataset names from the study site - #50

Draft
MaxGhenis wants to merge 10 commits into
mainfrom
remove-retired-dataset-names
Draft

MaxGhenis wants to merge 10 commits into
mainfrom
remove-retired-dataset-names

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Removes Microcosm's pre-August name, and the name of the dataset it replaced as the default, from everything the study site renders at policyengine.org/us/ai-inequality. The study's own pages also stop linking pages that show either name. The shared PolicyEngine header still links three policyengine.org pages that show the earlier dataset's name: Model, Python and Research.

Changes

  • Income-shift footer: reads "Microcosm US 2024 (build p, 28 July 2026)". microcosmBuildLabel builds that label from the published release id and returns null for an id whose timestamp names no calendar date. JSON keys and ids stay as published.
  • US home page: drops the card for the earlier dataset and its image.
  • Budget Lab memo: says Microcosm in the stability and reproducibility paragraphs. The stability paragraph now says the rerun also moved policyengine.py from 4.22.3 to 5.0.1, so it no longer attributes the differences to the data or credits them to a recalibration. It drops the claim that build p revised capital gains calibration. Between the two builds, 28 Microcosm commits revise targets for mortgage interest, taxable interest and other income, along with SSI take-up, but none changes a capital gains calibration target. It also names Rapid / proportional as the variant behind the New York figure. bun run build:memo regenerated src/data/memoContent.js.
  • Slide deck: cuts the slide whose figures describe the earlier dataset.
  • Model links: the model links on the policy analysis, income-shift and research pages now open the policyengine-us documentation at policyengine.github.io/policyengine-us. They opened policyengine.org/us/model, whose comparison with TRIM3, NBER TAXSIM or mRelief names the earlier dataset seven times. UK pages keep policyengine.org/uk/model, which shows neither name. The home page's PolicyEngine card now opens the country's home page. The sweep data's model_url and MODEL_URL in compute_shift_sweep.py and compute_ai_scenarios.py move to the documentation too, so the next run writes the same link. The committed analysis outputs keep the old link, since the site serves none of them.
  • Other links: the approach section's "open source (AGPL-3.0)" link and the unmounted capabilities section's GitHub link now open PolicyEngine/policyengine-core. They opened the org page, whose profile README lists the earlier dataset's repository. The "multi-agent AI" link now opens the multi-agent workflows post on policyengine.org in place of the policyengine-us repository, whose README shows a retired name.
  • analysis/: renames prose in docstrings, comments and AI_SCENARIOS.md but keeps frozen ids, keys, fixtures and committed outputs. By default compare_data_builds.py now compares build o with the published build p results. It prints each run's policyengine-us and policyengine.py versions, and it warns when the two runs differ in country model version or in the benefits they keep out of net income.

Invariants

  • No route in src/App.jsx shows either retired name in its text, title or attribute values in any state the sweep below reaches, and no route's content links a page listed in PAGES_SHOWING_RETIRED_NAMES. No file a page hands over as a download, such as the references page's BibTeX, shows either name.
  • Every page the site serves comes from a route in src/App.jsx, since app/ holds only the catch-all page that renders the router app.
  • The shared header and footer from @policyengine/ui-kit, which wrap every page, show neither name in any state the sweep reaches. Of the listed pages, they link only the three in SHELL_LINKS_TO_RETIRED_PAGES: policyengine.org/us/model, /us/python and /us/research.
  • The generated memo, the site's code, copy and settings, and every text file public/ serves contain neither name, once the test folds compatibility characters and drops invisible ones. Markup files also contain neither name once the test strips their tags and decodes their entities. src/utils/modelMetadata.js may spell the old name only in the release-id prefix it has to recognize.
  • public/, src/ and app/ hold no PDF, Office file or archive unless someone has read it in full and listed it in CHECKED_DOCUMENTS, which starts empty.
  • No URL in site code, data or public files points at a page in PAGES_SHOWING_RETIRED_NAMES. The test resolves root-relative and protocol-relative paths, and link targets in served markup, as policyengine.org would.
  • Site code asks the network only for files that public/ serves, through fetch(withBasePath(...)). The call names the path itself or a string constant that site code defines with one value.
  • Site code names a data key that holds a frozen id only where ALLOWED_READS lists it, and only inside the call listed with it. AIScenarios names certifiedDataBuildId inside microcosmBuildLabel(...) and nowhere else. Code that prints a whole metadata object names no key, so only the sweep can catch it.
  • microcosmBuildLabel returns a label for a Microcosm US build id, in either spelling, whose timestamp names a real calendar date, and null for anything else.

src/retiredDatasetNames.test.jsx checks all but the last: it reads each route from src/App.jsx, renders it in each country and sweeps it. The sweep works every button, native disclosure, checkbox and radio, and every element with an inline pointer cursor or a role or state that marks it as a control. It also works any element whose React props carry a press or hover handler, such as a card that a CSS class styles as clickable. It hovers each element that handles a hover before pressing it, and after each step it records the page's text, title, attribute values and links.

A click can hide a control before the pass reaches it, as a menu that closes on a choice hides its other options. The sweep then renders the page again, replays the steps back to that control, works it and runs a pass from there, up to 1,000 replays per page. The sweep also gives each option of every radio group, tab list, list box and select a fresh render and a pass that keeps that option chosen. A last phase tries each option of one group with each option of every other group. The sweep leaves the country switch alone, because the test renders each country's page separately. It reaches 239 distinct states of the US policy-analysis page, 221 of income-shift and 42 of research.

The sweep never tries three options together, presses a key or types, and a page that adds a text field fails until the sweep learns to fill it. The option passes and the pairing phase don't replay controls they hide. The test never loads the uprating explorer's table, since the public-file checks read that file directly, and the charts draw nothing in jsdom, so the sweep never reads chart labels. A link to an outside page the list doesn't name passes, so someone has to read each new outside page by eye. The fetch check looks constants up by name, so it can't tell a served constant from a local variable that shadows it. src/utils/modelMetadata.test.js checks the last invariant, with month 13, 32 July, day 0, 31 February, 29 February 2027 and a leap day.

Verification

  • Vitest passes all 73 tests, eslint reports zero warnings, and pytest passes 188 tests in analysis/tests and skips 13. bun run build succeeds, and Prettier flags the same nine files here as on main.
  • We broke the site 55 ways, one at a time, and every break fails at least one test. They include a plain button, a hover-only tooltip, a new route, a root-relative link, a live Hugging Face fetch, a menu that closes on a choice and a BibTeX download. Six harmless edits pass. A retired word used as an English noun in a reference title fails until someone lists the phrase in ALLOWED_PHRASES. Turning off the option passes or the replays, or cutting replays to one level, lets through the breaks each phase targets.
  • Regenerate the uprating table from policyengine-us and count the site's uprating cards from it #54 fetches the uprating table through a named constant, and the fetch check passes on a tree that merges it. On that tree six breaks fail the check: a missing file, a constant with two values, a remote address in a constant, an undefined name, a template path and a fetch without the base path. Renaming the constant passes, and so does defining it twice with one value.
  • Fix broken source and research links on the study site #52 and Regenerate the uprating table from policyengine-us and count the site's uprating cards from it #54 each merge cleanly with this branch. Describe the income shift and the state revenue measure accurately #51 and Count health coverage and TANF in the income-shift sweep (d2671) #53 each conflict on src/data/shiftSweepData.json, where they rewrite the description line next to the model_url line this branch changes. To resolve it, keep their description and this branch's model_url. Vitest passes on each merged tree and pytest on each that changes analysis/, and both pass on a tree that merges all four.
  • On the Vercel preview of ebe60e8, a production build, we worked every control and select option on every route in both countries in Chrome. We read the text, title and attribute values of each state, 95 and 97 of them on the US policy-analysis and income-shift pages, and searched the slide deck's full HTML. Neither name appears. On the preview of f1edea9, /budget-lab shows the memo's new stability and reproducibility paragraphs, and neither name appears in its text, title or attribute values.
  • We opened the 12 policyengine.org pages the shared header and footer link, two of them under its About menu, and worked each page's controls. Research shows a retired name in a featured card for an August 2025 post, Python in its example code, and Model in its comparison with TRIM3, NBER TAXSIM or mRelief, while the other nine show neither. We worked the other policyengine.org pages the study links the same way, and none shows either name: the UK and Canada home pages, four US research posts, /us/ai, /us/obbba-households, /uk/api, and /uk/model, including its comparison of all seven UK models. The policyengine-us documentation's home page, which the model links now open, shows neither name too.
  • The site holds 158 outside addresses, and 140 of them load and show neither name. Two ScienceDirect pages sit behind a bot check, eight PDFs refuse scripted downloads, and two hosts serve fonts. Six no longer load: Fix broken source and research links on the study site #52 replaces the UK autumn-budget post's link, and ai-objectives.org, dontloseyour.job, two windfallclause.org links and the Tax Policy Center's microsimulation primer still need new targets.
  • The build o and published build p runs reproduce the memo's stability figures. Between them, no scenario's revenue estimate moves by more than $5.92B (Rapid / expansive): Moderate and Rapid estimates move by at most 2.76% and Slow estimates by up to 11.6%, and no poverty change moves by more than 0.108pp. The baseline top-1% net income share rises from 8.77% to 9.00%, and New York's state take under Rapid / proportional from $3.8B to $5.8B.
  • An independent review of f1edea9 stopped on a usage limit after finding nine breaks the test let through, and 7a7a1af catches all nine. A new independent review of 0c6ee17 has started.
  • Merge and repeat the rendered check on the live site

Left for other repositories

These pages still show a retired name, and the study's own pages no longer link them:

  • policyengine.org/us/model, whose comparison with TRIM3, NBER TAXSIM or mRelief names the earlier dataset; /us/python, whose example loads it; /us/model/data/pipeline, whose diagram names it; and /us/research, whose featured card for an August 2025 post names it (PolicyEngine/policyengine-app-v2). PolicyEngineShell from @policyengine/ui-kit renders the same header on every page here as on policyengine.org. Its US navigation links Model, Python and Research, so the site links those three pages while they show the name.
  • github.com/PolicyEngine: PolicyEngine/.github's profile README lists the earlier dataset's repository, and nine repositories, two of them archived, show a retired name in their name or description on the repository list.
  • the PolicyEngine/policyengine-us README
  • the PolicyEngine/microcosm README, microcosm-dynamics' README and description, and microcosm.institute

Left for the study refresh

  • src/data/incomeDistributionData.json still comes from the earlier dataset, and the refresh should rerun it. Its metadata doesn't render, and no page names a source for the charts it feeds (IncomeDistributionBreakdown, BaselineDistributionSummary), so nothing labels them Microcosm.
  • The two build-stability runs differ in policyengine.py version as well as data. Rerunning build o on policyengine.py 5.0.1 would isolate the data change.
  • src/data/shiftSweepData.json and src/data/incomeDistributionData.json record local file paths (runtime_dataset_source and baseline_microdata_path), and the site bundles both files. The refresh should drop those paths or record published addresses.
  • paper/ prose still uses the old names, though no page on the site links to it.
  • Match the Budget Lab's 9 Oct 2026 revision: rerun scenarios, regenerate site, memo and paper #45 conflicts with main in .gitignore, and with this branch in five more files: the memo, memoContent.js, compare_data_builds.py, compute_ai_scenarios.py and reconcile_budget_lab.py. Its side of the memo's stability and reproducibility paragraphs uses Microcosm's old name and prints its release id, so the new test will fail on it until the resolved memo drops both.

🤖 Generated with Claude Code

Microcosm's pre-August name and the dataset it replaced as the default
still appeared on pages the site renders. The income-shift footer printed
a frozen dataset key, the US home page carried a card for the earlier
dataset, the Budget Lab memo named the old build, and the unlinked slide
deck described the earlier dataset's construction.

The footer now shows a label built from the build id, the memo says
Microcosm and names the build by its commit, and the card, its image and
the slide go, since no clean page exists to link and the slide's figures
describe the earlier dataset. A regression test renders every view and
scans site code, public files and the generated memo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
ai-inequality Ready Ready Preview Nov 10, 2026 8:35pm UTC

Request Review

MaxGhenis and others added 2 commits November 9, 2026 08:15
Cut a clause that repeated "identical model version".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The names test now clicks every control a page shows, including any a
  click reveals, and re-renders to click any control an earlier click hid.
  It leaves the country switch alone: clicking it swapped the page to the
  other country, so the US scenario picker went unswept.
- Escape the invisible characters in the test's character class so eslint
  passes.
- Point the approach and capabilities links at pages that show no retired
  name.
- Return no build label for a timestamp that names no calendar date.
- compare_data_builds.py compares build o with the published build p by
  default, prints both runs' versions, and warns when they differ in country
  model version or net-income exclusions.
- The memo says the build p rerun kept policyengine-us at 1.764.6 and moved
  policyengine.py from 4.22.3 to 5.0.1, in place of calling it a data-only
  change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… read

The sweep now clicks elements styled with a pointer cursor, which reaches
the research page's expandable cards and the uprating explorer's toggle.
For each radio group, tab list and select on the starting page, it also
tries each option with every option of each other group shown alongside
it, so it reaches every AI adoption scenario with every wage variant.

Site code may name a data key that holds a frozen id only where
ALLOWED_READS lists it, and AIScenarios may name certifiedDataBuildId
only inside its call to microcosmBuildLabel. A new test renders the
shared header and footer from @policyengine/ui-kit and lists the two
policyengine.org pages they link that show a retired name: Research and
Python. The research listing joins the pages that show one, and archives
join the documents public/ may not serve unread.

The memo's stability paragraph no longer attributes the differences
between builds to the data or credits them to a recalibration, drops the
capital gains claim that no Microcosm commit between the builds supports,
and names Rapid / proportional as the variant behind the New York figure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits November 9, 2026 22:53
The sweep now returns a route to each control that a click hid before the
pass reached it, such as the other options of a list that closes once
someone chooses. It replays each route from a fresh render, works whatever
the replayed control reveals, and fails once it needs more than 1,000
routes.

For each option of every radio group, tab list, list box and select on the
starting page, a pass starts from that option and leaves the group alone.
Native radios that share a name form one group, and the pairing step now
draws its pairs from the same groups.

The test now counts the org's repository list, the profile README's pages,
microcosm-dynamics, and the listed repositories' branch roots and README
files among the pages that show a retired name.

In a mutation run the test fails on all 55 deliberate breaks and passes on
7 harmless edits. Turning off the option passes or the routes, or cutting
the routes to one level, lets through the breaks that each phase targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine.org/us/model names the earlier dataset seven times in its
comparison with TRIM3, NBER TAXSIM or mRelief, and a click or a ?compare=
link opens that comparison. The study linked the page from its home,
research, policy analysis and income shift pages.

The model documentation links in PolicyAnalysis, IncomeShift and
ShiftSweep and the research page's model link now open
https://policyengine.github.io/policyengine-us/, which the model page
itself links as the policyengine-us documentation. UK pages keep
policyengine.org/uk/model, which shows neither name. The PolicyEngine card
on the home page describes the whole project, so it now opens the
country's home page, and the unmounted capabilities component gets the
docs link.

The sweep data's model_url and MODEL_URL in compute_shift_sweep.py and
compute_ai_scenarios.py move to the docs together, so the next run writes
the same link. The committed analysis outputs keep the old one, since the
site serves none of them.

The test now lists the model page among pages that show a retired name and
expects the shared header to link it. Against the old links, it fails on
all four pages and on the sweep data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On #54's branch, useUpratingTable fetches
withBasePath(UPRATING_TABLE_PATH), a constant that upratingTable.js
defines. The fetch check counted only a path written in the call, so it
failed on a tree that merges #54, though the fetch still asks for a file
public/ serves.

The check now also accepts a fetch that names a constant, and looks up the
value site code gives that constant. It fails when no site code defines
the name, or when site code gives it more than one value. A unit test
checks fetchedFile with a written path, a constant, a constant with two
values and an undefined name.

On a tree that merges #54, six breaks fail the check: a missing file, a
constant with two values, a remote address in a constant, an undefined
name, a template path and a fetch without the base path. Renaming the
constant passes, and so does defining it twice with one value.

The header-and-footer test and the BibTeX download test now get the
120-second limit the page tests have. Under load the header sweep took 6.5
seconds and failed Vitest's default 5-second limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits November 10, 2026 15:12
#54 adds a comment that uses the word fetch, and
the check counted that word as a request with no served fetch to match it.
The check now blanks every comment that espree finds before it counts.
espree, the parser ESLint already runs on this code, leaves a URL inside a
string whole. A request word in a string or in page copy still counts, and a
file espree can't parse, such as a TypeScript file, keeps its comments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The names test imports espree directly, so package.json now lists it beside
eslint, which already installs the same version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 8448ff77 Deployed Nov 10, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant