[MaxGhenis]

What this changes

The policyengine-data skill steered agents to the old data stack. PolicyEngine archived microimpute on 2026-11-08 (PolicyEngine/microimpute#224), and Microcosm imputes PolicyEngine's microdata with microcosm-fit. This PR corrects skills/policyengine-data/SKILL.md to match.

• UK paragraph (lines 164-167 on main). It said policyengine-uk-data's Enhanced FRS "feeds the UK Microcosm build" and that UK imputations "land" there. The new text says that:
  • uk-data builds the Enhanced FRS and still imputes with microimpute 1.8.1;
  • policyengine.py defaults UK simulations to Microcosm's microcosm_uk_2024_25 and serves the Enhanced FRS only by name;
  • a release-blocking Microcosm gate compares UK candidates with a pinned Enhanced FRS;
  • Microcosm runs its own UK imputations with microcosm-fit in uk_runtime, where new UK imputation work goes; the text names each stage's source survey.
• microimpute's archive. A new bullet beside the policyengine-us-data one says PolicyEngine archived microimpute on 2026-11-08, tells agents to open no PRs or issues there, and notes that PyPI keeps all of its releases, including the 1.8.1 that uk-data pins.
• Line 29. Takes microdf off the list of packages Microcosm replaced, since policyengine-core depends on it. Adds one sentence: build new imputation on microcosm-fit and new calibration on microcosm-calibrate, never on microimpute, microcalibrate or L0.
• Package table. microcosm-fit now succeeds microimpute as well as ad hoc scripts, microcosm-calibrate succeeds microcalibrate and L0 (l0-python), and microcosm-frame succeeds microdf only inside Microcosm.
• "Institutional knowledge". The old text said the old stack's algorithms "are exactly what Microcosm's fit and calibrate operators do". The new text says Microcosm implements the same families of methods with its own code and defaults, contrasts the two QRFs' defaults for leaf samples and weights, and tells agents not to let a previous-stack fit stand in for a Microcosm one. The heading now reads "previous-stack concepts", since PolicyEngine has not archived microcalibrate or L0.
• QRF paragraph. "Quality is scored with quantile loss" presented microimpute's metric as current practice. The paragraph now credits that metric to microimpute and names microcosm-fit's held-out scorer, which covers categorical models only. Its last sentence drops the claim that the fit reads the frame's weights "by construction", since a caller can opt out with weights="none".
• Description. Adds microimpute and microcalibrate to the triggers, so an agent about to reach for either one loads this skill. To stay under the 1,024-character Codex limit (now 1,019), "conditional models" became "models", "build/release gates" became "release gates", and "calibration diagnostics dashboard" became "calibration dashboard".

Evidence from code

Claim	Where
PolicyEngine archived microimpute on 2026-11-08	gh api repos/PolicyEngine/microimpute: "archived": true, updated_at 2026-11-08T06:31:54Z; PolicyEngine/microimpute#224 merged 2026-11-08T06:27:46Z
PyPI keeps all of microimpute's releases, including 1.8.1	https://pypi.org/pypi/microimpute/json lists 61 releases and none yanked; uk-data's uv.lock:889-891 pins 1.8.1 from https://pypi.org/simple
Microcosm runs its UK imputations with microcosm-fit	Microcosm main 3db65154b, packages/microcosm-build/src/microcosm/build/uk_runtime/: was_wealth.py:900,935, lcfs_consumption.py:1935,1990, etb_vat.py:371,373 and etb_services.py:481,489 call RegimeGatedQRF; spi_income.py:3566 and salary_sacrifice.py:880 load microcosm.fit.QRF (an alias of RegimeGatedQRF, microcosm/fit/__init__.py:118)
The WAS stage imputes student-loan balances	was_wealth.py:205-222 (student_loan_balance among the outputs); student_loans.py:93
Microcosm's salary-sacrifice stage trains on the FRS respondents with pay whom the survey asked about salary sacrifice	uk/source_stages.json stage note 47 ("The QRF trains only on the 2024-25 FRS asked subset with pay"); uk_runtime/salary_sacrifice.py:95-98 (SALSAC_TRAINING_POPULATION: FRS channel, salary_sacrifice_asked == 1 and employment_income > 0)
Microcosm's WAS, LCFS and ETB stages port uk-data's models	uk/source_stages.json stage notes 22, 30, 31, 32 ("Ports incumbent WAS round-8 wealth imputation", "Ports the incumbent LCFS consumption QRF", "Ports ETB VAT imputation", "Ports ETB public-services QRF")
A release-blocking gate compares UK candidates with a pinned Enhanced FRS	uk/gates.json uk_input_mass_parity (terminal, release_blocking, enhanced_frs_2024_25.h5 at revision a9e52499); uk_runtime/terminal_gates.py:81
policyengine.py defaults UK simulations to Microcosm and serves the Enhanced FRS only by name	policyengine.py main 57b587036 (6.16.0), src/policyengine/data/bundle/manifest.json:106 (default_dataset: microcosm_uk_2024_25), :261-271 (dataset_overlays.uk.enhanced_frs_2024_25); src/policyengine/provenance/manifest.py:756-776 (an overlay "can never alter default resolution")
uk-data's build imputes with microimpute 1.8.1	policyengine-uk-data main ed19e336: policyengine_uk_data/utils/qrf.py:9 (from microimpute.models import QRF), used by the income, wealth, vat, salary_sacrifice, consumption, frs_only and services/etb imputations; uv.lock:889-890 (microimpute 1.8.1)
microdf stays	PolicyEngine/policyengine-core master 7c84d941b, pyproject.toml:26 (microdf_python>=1.0.0); no Microcosm pyproject.toml names microdf, and Microcosm's uv.lock pulls it in only through policyengine-core, policyengine-uk and policyengine-us
L0 ships as l0-python	PolicyEngine/L0 pyproject.toml:6; uk-data pyproject.toml:42
microimpute's QRF keeps one sample per leaf by default and fits unweighted unless the caller passes weight_col	microimpute 1.8.1 (9051a53): microimpute/models/imputer.py:222 (weight_col defaults to None; when set, fit resamples the training rows by it, :222-300) and qrf.py:167 (no max_samples_leaf); main e4cbe025d: imputer.py:247 takes the same weight_col and hands it to the forest as sample_weight (qrf.py:178-212); quantile-forest 1.4.1, which uk-data locks (uv.lock:1783-1784), and 1.4.2 both default max_samples_leaf to 1
microcosm-fit keeps every leaf sample, up to 4,096 per leaf when a forest's target takes 2 to 32 distinct values, and weights every fit unless the caller passes weights="none"	Microcosm main 3db65154b, packages/microcosm-fit/src/microcosm/fit/qrf.py: the default max_samples_leaf=None (:1740) keeps all leaf samples (:1712-1719); _fit_forest raises it to _DISCRETE_Y_LEAF_BOUND = 4_096 (:925) when the forest's pre-bootstrap target has 2 to _DISCRETE_Y_UNIQUE_MAX = 32 (:917) distinct values and the caller left max_samples_leaf unset (:1006-1030), and packages/microcosm-fit/tests/engine_free/shared/test_qrf.py:548-568 asserts the cap; the fit reads a Frame's typed weights by default, treats "none" as its only unweighted path, and raises on a DataFrame without explicit weights (qrf.py:1767-1772)
microimpute scores numeric imputations with quantile loss	microimpute microimpute/comparisons/metrics.py:33-56 (quantile_loss for numeric variables, log_loss for categorical ones)
microcosm-fit's one held-out scorer covers categorical models	microcosm/fit/held_out.py (module docstring; the package defines no scoring function besides categorical_scores)

Not in this PR

• This PR leaves dashboard/src/data/manifest.json alone. A rebuild would change the data skill's inferred target_repos (adding policyengine-core, dropping policyengine.py) and reshuffle about 1,600 lines of overlap scores. Require PyYAML for the dashboard manifest, check it on Linux in CI, and keep overlaps in artifact order #107 and the repo-coverage branch regenerate the manifest wholesale, and the dashboard's prebuild hook rebuilds it anyway.
• The analyze-policy pipeline routes UK INVESTIGATE issues to policyengine-uk-data (targets/claude/agents/report-logger.md:61,200,203,273-277, tests/pipeline/test_country_routing.py, .github/workflows/analyze-policy.yml), although policyengine.py defaults UK simulations to Microcosm. skills/policyengine/SKILL.md:250-253 says the pre-2026 datasets "are gone", though policyengine.py still serves enhanced_frs_* by name. Route UK data issues to the repo that builds the run's dataset #110 fixes both.

uv run pytest --ignore=tests/test_skill_examples.py: 208 passed. build_claude_wrapper.py copies the file unchanged.

🤖 Generated with Claude Code