Require every reported number to come from a model run

bundles/analysis-tools.json

@@ -2,7 +2,7 @@
   "name": "analysis-tools",
   "description": "Policy analysis and research - impact studies, dashboards, notebooks, and visualizations",
   "category": "analysis",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "analysis",
     "research",

bundles/api-development.json

@@ -2,7 +2,7 @@
   "name": "api-development",
   "description": "API development - Flask endpoints, caching, services, and integration patterns",
   "category": "development",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "api",
     "flask",

bundles/app-development.json

@@ -2,7 +2,7 @@
   "name": "app-development",
   "description": "React app development - app-v2, standalone tools, Vercel deployment, and PolicyEngine-specific patterns",
   "category": "development",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "react",
     "frontend",

bundles/complete.json

@@ -2,7 +2,7 @@
   "name": "complete",
   "description": "Complete PolicyEngine knowledge base - all agents, commands, and skills for users, analysts, and contributors",
   "category": "complete",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "complete",
     "all",

bundles/country-models.json

@@ -2,7 +2,7 @@
   "name": "country-models",
   "description": "Complete multi-agent workflow for implementing government benefit programs in PolicyEngine country packages",
   "category": "development",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "tax",
     "benefits",

bundles/dashboard-builder.json

@@ -2,7 +2,7 @@
   "name": "dashboard-builder",
   "description": "Orchestrated AI workflow for creating PolicyEngine dashboards from natural-language descriptions",
   "category": "development",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "dashboard",
     "calculator",

bundles/data-science.json

@@ -2,7 +2,7 @@
   "name": "data-science",
   "description": "Data analysis, survey enhancement, imputation, calibration, and microdata utilities",
   "category": "data",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "data",
     "microdata",

bundles/essential.json

@@ -2,7 +2,7 @@
   "name": "essential",
   "description": "Essential PolicyEngine knowledge for all users - understanding the platform, using the web app, and basic concepts",
   "category": "getting-started",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "keywords": [
     "beginner",
     "users",

skills/policyengine-tools/SKILL.md

@@ -127,7 +127,11 @@ multizone. Register in `policyengine-app-v2/website/src/data/apps.json` (`type`,
 ## Data and backend patterns
 
 Never paste ad-hoc computed numbers into source. Always: Python script → data file
-(JSON/CSV) → frontend imports it. Pick by need:
+(JSON/CSV) → frontend imports it. The data file holds model outputs for the scenarios
+shown (and arithmetic on those runs, such as reform minus baseline), plus clearly labelled
+external comparators; the pipeline and the frontend never scale, interpolate or extrapolate
+results to cover scenarios that were not run (see "Every reported number is a
+model run" in the `policyengine` skill). Pick by need:
 
 - **A — Precomputed JSON/CSV**: finite scenarios (legislative trackers, static analyses). A
   `scripts/*.py` microsimulation writes to `public/data/`; the frontend imports it. Zero

skills/policyengine-uk/SKILL.md

@@ -25,8 +25,9 @@ the canonical `import policyengine as pe` interface, the population flow
 and dataset handling — this skill only carries what is *different* about the UK.
 
 Verified against policyengine 4.21.0 + policyengine-uk 2.89.2 + policyengine-core 3.30.0
-(2026-07). UK examples are not in the `[us]`-only CI verify set, so they carry no
+(2026-07). UK examples are not in the `[us]`-only CI verify set, so they carry no fast
 `<!-- verify -->` marker; each was run manually in a `policyengine[uk]` environment.
+Population examples may carry `<!-- verify: slow -->`.
 
 ## Setup
 
@@ -213,6 +214,53 @@ bundle. The pre-Microcosm UK datasets are superseded. See the `policyengine-data
 Microcosm UK is built (FRS + WAS imputation) and calibrated, and the `policyengine` skill for the
 `ensure_datasets` / `Simulation` population flow, which is identical across countries.
 
+## Macro scenarios (a different CPI or earnings path)
+
+A macro path goes in through the growth series in `gov.economic_assumptions.yoy_growth.obr`
+(`consumer_price_index`, `average_earnings`; move `rpi` and `cpih` with CPI if prices move).
+Pass it to `managed_microsimulation` as a `Scenario` applied **before the data load**, as
+below, so the dataset is projected on the new path along with the parameters derived from
+it. Don't send a macro path through the `pe.uk` `Simulation`, `Policy`/`Dynamic` or
+`economic_impact_analysis` interfaces: they hand parameter values to policyengine-uk as a
+reform when it builds the simulation, on each year's data as the dataset was already
+projected.
+
+<!-- verify: slow -->
+```python
+from policyengine.tax_benefit_models.uk import managed_microsimulation
+from policyengine_uk.utils.scenario import Scenario
+
+OBR = "gov.economic_assumptions.yoy_growth.obr"
+base = managed_microsimulation()
+p = base.tax_benefit_system.parameters
+changes = {
+    f"{OBR}.{series}": {
+        f"year:{y}-01-01:1": float(p.get_child(f"{OBR}.{series}")(f"{y}-01-01")) + 0.01
+        for y in range(2027, 2034)
+    }
+    for series in ("consumer_price_index", "average_earnings")
+}
+macro = managed_microsimulation(
+    scenario=Scenario(parameter_changes=changes, applied_before_data_load=True)
+)
+for year in (2029, 2034):  # inside and beyond the dataset's own years
+    assert macro.calculate("employment_income", year).sum() > base.calculate("employment_income", year).sum()
+```
+
+This example last ran on policyengine 5.3.0 / policyengine-uk 2.90.2 (2026-09), on the
+then-default `enhanced_frs_2024_25`: +1pp CPI and earnings in 2027-2033 raised 2034 employment
+income from £1,662.6bn to £1,779.4bn. policyengine-uk has since changed how reforms and
+scenarios reach the parameters (`policyengine_uk.utils.dated_reforms`), so check any other
+route, period key or series horizon against the installed version before relying on it, and
+measure again on the current bundle before you quote a figure. `policyengine` downloads the
+dataset to `./data/` under the working directory, so run parallel processes from separate
+working directories.
+
+To combine a macro path with a policy, build a macro baseline and a macro reform on the same
+macro `Scenario`, check on the installed version that the policy reaches the reform's
+parameters for every year you report, and compare the two. Don't compare against
+`sim.baseline`, which policyengine-uk builds on the default macro path.
+
 ## Country-model development notes (policyengine_uk directly)
 
 Everything above uses the managed `pe.uk` surface. When you work *inside* the UK country model

skills/policyengine/SKILL.md

@@ -8,7 +8,7 @@ description: |
   cost of a policy, revenue estimate, poverty rate, child poverty, winners and losers, decile,
   Gini, inequality, CTC, EITC, SNAP, income tax, universal credit, state-level analysis,
   congressional district, constituency, Microcosm dataset, MicroSeries, ensure_datasets,
-  economic_impact_analysis, managed_microsimulation.
+  economic_impact_analysis, managed_microsimulation, scenarios, uncertainty, Monte Carlo.
   NOT for: implementing new variables/parameters inside country models (use
   policyengine-model-development) or calling the REST API from JS (use policyengine-api).
 metadata:
@@ -25,6 +25,33 @@ Originally verified against policyengine 4.21.0 (2026-07); the marked examples r
 against the latest release (5.0.1 at 2026-08). Re-verify the bundle when precision matters
 (see "Checking what you're running" below).
 
+## Every reported number is a model run
+
+PolicyEngine is a microsimulation model, and its outputs are the product. Every fiscal,
+distributional, poverty or household figure an analysis presents as a PolicyEngine estimate
+must come from running the model for that scenario and year, never derived from the output of
+runs for other scenarios. None of these is allowed, however well it is validated:
+
+- scaling one run's totals by a ratio of indices, levels or shares (for example
+  `cost = spend * (index_a - index_b) / index_central` applied to thousands of Monte Carlo
+  draws, with a handful of full runs as a "check");
+- interpolating or extrapolating between runs, years or parameter values, or applying an
+  elasticity or share taken from one run to another scenario;
+- a side model (closed form, spreadsheet, regression on model output) that stands in for the
+  model on some scenarios;
+- changing one parameter in a scenario that should move the rest of the system. A macro or
+  price path goes in as model inputs, so the model's uprating carries it through (UK: see
+  "Macro scenarios" in the `policyengine-uk` skill).
+
+When the design needs more scenarios than you can run, run fewer: choose a small set you can
+explain, run each one fully, and state how many full runs stand behind each figure. A
+distribution comes from full runs over sampled scenarios, with the sampling error stated.
+Arithmetic on model *inputs* is fine (building a parameter path from an uprating rule, say),
+and so is arithmetic that combines outputs of the runs for the scenario being reported: reform
+minus baseline, sums over years, ratios, percentage changes, averages, deflation, and sampling
+error across full runs. Arithmetic that produces a figure for a scenario nobody ran is not. If
+the model cannot produce a figure, do not publish it.
+
 ## Setup
 
 Country models are extras — bare `policyengine` installs neither:
@@ -39,7 +66,8 @@ Analysis always runs on the **latest released** `policyengine` (`>=5.0.1`; resol
 from PyPI as described in "Checking what you're running"). Each release pins exactly-matched
 country-model versions and the certified data bundle, which is what makes results
 reproducible. Directly-imported country packages (`policyengine_us` / `policyengine_uk`) are
-for model development and tests, not for analysis compute.
+for model development and tests, not for analysis compute (the exception: a UK macro path needs
+`policyengine_uk`'s `Scenario`; see "Macro scenarios" in the `policyengine-uk` skill).
 
 ## Household calculations (fast, ~2 GB RAM)
 

skills/review-program/references/workflow.md

@@ -493,7 +493,13 @@ Classify confirmed findings:
   an explicit repository completion requirement (such as source corroboration or functional
   tests). Identify that requirement and the concrete failure. A plausible output difference,
   missing optional test, unknown source or preferred implementation pattern alone is not
-  a confirmed critical.
+  a confirmed critical. A figure published as a PolicyEngine estimate (analysis, report,
+  dashboard data or blog) that was derived from other runs' output instead of a run of that
+  scenario (scaling by an index or share ratio, interpolation or extrapolation between
+  runs, a side model standing in for the model) is always critical, whatever validation
+  accompanies it: this is an explicit repository requirement; see "Every reported number
+  is a model run" in the `policyengine` skill. Hand-calculated expected values in tests are
+  not covered.
 - **SHOULD ADDRESS**: missing boundary coverage on tested behavior, maintainability or
   repo-standard issues. Missing rounding/flooring/capping is normally here unless a
   demonstrated case changes eligibility/category or materially changes results.

targets/claude/agents/reform-comparator.md

@@ -157,7 +157,7 @@ Build a `benchmark_agreement` block in the output:
 
 The stage does NOT fire only when the reform has a full external cluster within ±25% (Step 2b already CORROBORATED).
 
-**Rule of thumb — replace verbal reasoning with model runs.** If the natural comparator section is written as "TCJA scored X, our reform is 1/6 the size, so linear scaling suggests Y" — that's a mirror candidate. Run TCJA's shape through our model and check we reproduce X. Verbal linear-scaling should not appear in a final report; it should either be corroborated by a mirror run or explicitly labeled as an unvalidated sanity check.
+**Rule of thumb — replace verbal reasoning with model runs.** If the natural comparator section is written as "TCJA scored X, our reform is 1/6 the size, so linear scaling suggests Y" — that's a mirror candidate. Run TCJA's shape through our model and check we reproduce X. Linear scaling must not appear in a report, labeled or not: either run the comparator's shape through the model or leave the comparison out (see "Every reported number is a model run" in the `policyengine` skill).
 
 When the trigger fires, invoke `model-corroborator` with the original reform context and the benchmark cluster. The corroborator picks 1-2 closest-shape candidates, builds mirror reform-dicts (and a baseline policy if the source uses a different baseline schedule like TCJA-extension vs OBBBA-current-law), submits them to the PE API, polls for completion, and computes per-candidate corroboration:
 

targets/claude/marketplace.template.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://claude.ai/schemas/plugin-marketplace.json",
   "name": "policyengine-claude",
-  "version": "4.1.0",
+  "version": "4.1.1",
   "description": "Complete PolicyEngine knowledge base - for users, analysts, and contributors across the entire PolicyEngine ecosystem",
   "owner": {
     "name": "PolicyEngine",