Skip to content

Point the two-child limit validation page at Scorecard's rows - #2685

Merged
MaxGhenis merged 5 commits into
mainfrom
docs/tcl-page-to-scorecard
Oct 28, 2026
Merged

MaxGhenis merged 5 commits into
mainfrom
docs/tcl-page-to-scorecard

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #2704

Refs #1398, which stays open because it tracks baseline child-poverty benchmarks.

axiom: n/a: documentation only; no parameter, variable or rule changes.

PolicyEngine/policyengine-scorecard#225 merged on 2026-10-26 as eb1bb45a; the links below point at the claim ids it mints or attaches results to.

What changes

docs/book/validation/child-poverty-tcl.md hard-coded PolicyEngine's two-child limit estimates beside the DWP, HM Treasury and Resolution Foundation figures (corrected in #1724). Max asked on 2026-10-23 whether that belonged in Scorecard instead. It now does: PolicyEngine/policyengine-scorecard#225 recomputes the comparison on policyengine.py 6.11.0's certified pair. Each row states whether the two concepts match (relative line convention, absolute line reference year, AHC or BHC, child count, financial-year mapping, static or post-behavioural cost) and keeps the earlier runs in its history.

This page keeps the method and drops the numbers:

  • the reform (current law against current law with the limit set back to 2);
  • the variables for each measure;
  • a snippet on pe.uk.managed_microsimulation(), with the certified pair it loads (policyengine.py 6.11.0 pins policyengine-uk 2.132.6);
  • a table linking each benchmark to its Scorecard row, plus the decomposition of the September-to-October change (now also split by policyengine-uk release).

reform-impacts-dashboard-plan.md's two references to the page, and two-child-limit-full-rollout.md's one, now describe what it holds.

The page also names in_absolute_poverty_fye2011_ahc and _bhc, which policyengine-uk 2.155.0 added. That is the FYE 2011 line in DWP's note. Scorecard's certified release (2.132.6) predates them, so Scorecard still constructs that line.

Checks

  • Every mechanism the page states comes from policyengine-uk 2.132.6, read during the Scorecard work:
    • the limit parameter's values;
    • in_relative_poverty_*: 60% of the person-weighted median of the world being calculated;
    • in_poverty_*: the FYE 2025 reference;
    • would_claim_uc has no formula, and the labour-supply elasticities are zero;
    • year Y is FY Y-Y+1, from convert_to_fiscal_year_parameters.
  • Claim ids: all nine are in the Scorecard database built from Speedup #225's branch and in its app feed (data/populations.json), with the values and statuses the page's notes describe.
  • uvx --from ruff==0.17.0 ruff format --check . (the Lint job's pinned ruff, which also formats the page's Python block) and ruff check . pass.
  • Independent review (GPT-6.1 Sol): r1 at 6b98161 asked for Fixes with a dedicated issue and the full-rollout backlink; both are fixed here, along with its two optional wording points. Its two out-of-scope findings on two-child-limit-full-rollout.md predate this PR (exceptions described as unmodelled; FYE 2021 constants from an older parameter file) and go to their own task.
  • make documentation not run locally: the change is markdown text and links only; CI's Test documentation builds runs.

🤖 Generated with Claude Code

The page hard-coded PolicyEngine's estimates beside the DWP, HM Treasury
and Resolution Foundation figures. Scorecard now holds that comparison,
recomputed on a certified release with a status and notes per row
(PolicyEngine/policyengine-scorecard#225). The page keeps the method:
the reform, the measures and variables, and a snippet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits October 26, 2026 11:37
…imate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 26, 2026 20:27
policyengine-uk 2.155.0 added in_absolute_poverty_fye2011_ahc and _bhc.
Scorecard's certified release predates them, so it still constructs the line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Program review

Base repository: PolicyEngine/policyengine-uk
PR number: 2685
Reviewed head SHA: 6b98161
Merge base SHA: 5b94537
Mode: full
Scope: Three changed documentation/changelog paths, the shown comparison/API contracts and linked Scorecard/benchmark source claims; no underlying model or private population audit.
Source manifest: /private/tmp/policyengine-command-runs/881387a1b745/pr2685-external-review-20261026-review-sources.json
Review status: PARTIAL

Source Documents

Critical

None.

Should Address

None.

Suggestions

None.

Evidence Gaps

  • The DWP comparison note references the FYE2024 HBAI methodology, but that underlying primary methodology original was not acquired to corroborate the precise FYE2011 absolute-poverty reference-year premise. Current UK parameter/code support and author/internal reports do not substitute for the benchmark's primary methodology.
  • Role checks incomplete: /private/tmp/policyengine-command-runs/881387a1b745/pr2685-external-review-20261026-review-policy.md; see recovered report.

Notes

  • No confirmed critical, should-address or optional defect was established. The unresolved POLICY/source check is distinct from a demonstrated wrong value or citation; the auto-listed incomplete-policy-role gap refers to that same missing source premise.
  • CODE independently qualified the unexecuted snippet's syntax/API forwarding, reform range/sign, FY/entity/weighted-return and current poverty-variable contracts, all nine unique link labels and relative documentation targets. POLICY separately qualified linked public claim rows and benchmark definitions within its source window; neither role reproduced private population estimates.
  • POLICY preserved24fullpublicoriginals and corroborated all9deployed/merged225feedrows, relevant officialbenchmark tables/definitions, static-versus-behavioural costing and API/release distinctions. Three selected pages from2PDFs were rendered and visually audited. Manifestchecker24valid/0rejected. Originalsourcewindow20:46:26.767633–20:49:05.270474 lasted158.503/180seconds;20:49:18 was a reporting timestamp. No subsequent externalacquisition.
  • Local model/Core/country imports, collection, tests, documentation build, environment installation and private dataset/impact validation were NOT RUN. The documentation build is matching-head hosted CI evidence. Existing internal reviews and PR claims were context, not proof.
  • The raw CI rollup remains FAILURE with6SUCCESS/5workflow-qualified code-inapplicableSKIPPED/1FixCANCELLED. Docs121828844635/Lint121828845013/Changelog121828844468/Result121828941980 succeeded; skipped code jobs are not executed tests. Fresh Root metadata acquired actual Fix timestamps: canceled121828846244 ran20:28:29–20:28:55; successful121829134484 ran20:29:01–20:29:25. This establishes a later exact-head successful Fix, without cancellation-cause, attempt, all-jobs-reran, effective-universe or merge inference.
  • Source acquisition used one original3minute window and is closed; the missing primary-methodology source was not fetched through another role after its deadline. All originals and rejected/unusable derivatives remain preserved, with only qualified evidence admitted to the final source manifest.
  • The separate2687 qualification stopped before any substantive review after fullcomments revealed prior own-account public review and current-head implementation replies. Its internal PARTIAL/STOP and normal cleanup are preserved; no publicreview/guard workaround occurred. PR2685 is a new independent scope with its own original20:41:03 clock, not a2687retry.
  • Scope/diff briefing took101.414seconds after setup, exceeding the60second aim. POLICY finished22.849seconds after its preferred20:56 role target, before20:58:03 checkpoint. Original canonical/source clocks unchanged; Root normal cleanup and final timing follow.
  • The exact detached snapshot was removed normally after both reviewers completed (Root toola66d8e exit0), with no force. Reports and originals remain preserved; this is the sole timing finalization rerun.

Validation Summary

Exact6b98161832287c9ae5217020223d08c78b44dc0d clean detached snapshot; three docs/changelog files. Static API/reform/entities/FY/reference-year/current-variable/link contracts and scoped public source evidence checked. Matching-head docs build, Lint, Changelog and Result SUCCESS; raw12contexts6S5inapplicableSKIP1historicalFixCANCELLED, rawFAILURE retained. No local model/docs build/tests or private numerical reproduction. One specific DWP benchmark-methodology source premise unresolved.

Timing

setup seconds: 97.59s; scope seconds: 101.41s; parallel review seconds: 822.26s; policy role seconds: 720.48s; code role seconds: 565.36s; adjudication seconds: 0.00s; consolidation cleanup seconds: 83.15s; elapsed seconds: 1043.98s

Review Severity

COMMENT. Open findings: 0 critical, 0 should address, 0 suggestions.

…r, static scope

Fixes #2704. The changelog fragment now carries #2704, the issue this PR closes.
#1398 stays open. two-child-limit-full-rollout.md now describes what the TCL
page holds. The snippet note names policyengine.py 6.11.0's pinned
policyengine-uk 2.132.6. The static note says the labour-supply elasticities are
zero, not that no behavioural response runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis added this pull request to the merge queue Oct 28, 2026
@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Queued for merge at head a75c57b954cfcd85ba321e370554a29cf5ef5ae5.

  • Independent review.
  • Program review. The program review at 6b98161 found 0 critical, 0 should-address and 0 suggestions. Its evidence gap is that DWP's note uses the FYE 2011 absolute line. Scorecard's DWP source records that from the GOV.UK note, and the absolute rows are marked constructed, not comparable.
  • Checks. gh pr checks passes, including the required Lint and Test / Result, and the PR is MERGEABLE.
  • Out of scope here. Two findings on two-child-limit-full-rollout.md predate this PR: it describes the exceptions as unmodelled, which they are not since Model the two-child limit exceptions for Universal Credit and Child Tax Credit #2631, and it uses stale FYE 2021 poverty-line constants. A separate change will correct them.

Merged via the queue into main with commit f737953 Oct 28, 2026
11 of 12 checks passed
@MaxGhenis
MaxGhenis deleted the docs/tcl-page-to-scorecard branch October 28, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move the two-child limit child-poverty comparison from the validation page to Scorecard

2 participants