Repository navigation
Point the two-child limit validation page at Scorecard's rows - #2685
Conversation
The page hard-coded PolicyEngine's estimates beside the DWP, HM Treasury and Resolution Foundation figures. Scorecard now holds that comparison, recomputed on a certified release with a status and notes per row (PolicyEngine/policyengine-scorecard#225). The page keeps the method: the reform, the measures and variables, and a snippet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…imate Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk 2.155.0 added in_absolute_poverty_fye2011_ahc and _bhc. Scorecard's certified release predates them, so it still constructs the line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22
left a comment
There was a problem hiding this comment.
Program review
Base repository: PolicyEngine/policyengine-uk
PR number: 2685
Reviewed head SHA: 6b98161
Merge base SHA: 5b94537
Mode: full
Scope: Three changed documentation/changelog paths, the shown comparison/API contracts and linked Scorecard/benchmark source claims; no underlying model or private population audit.
Source manifest: /private/tmp/policyengine-command-runs/881387a1b745/pr2685-external-review-20261026-review-sources.json
Review status: PARTIAL
Source Documents
- source-1.original — cached original.
- source-2.original — cached original.
- source-3.original — cached original.
- source-4.original — cached original.
- source-5.original — cached original.
- source-6.original — cached original.
- source-7.original — cached original.
- source-8.original — cached original.
- source-9.original — cached original.
- source-10.original — cached original.
- source-11.original — cached original.
- source-12.original — cached original.
- source-13.original — cached original.
- source-14.original — cached original.
- source-15.original — cached original.
- source-16.original — cached original.
- source-17.original — cached original.
- source-18.original — cached original.
- source-19.original — cached original.
- source-20.original — cached original.
- source-21.original — cached original.
- source-22.original — cached original.
- source-23.original — cached original.
- source-24.original — cached original.
Critical
None.
Should Address
None.
Suggestions
None.
Evidence Gaps
- The DWP comparison note references the FYE2024 HBAI methodology, but that underlying primary methodology original was not acquired to corroborate the precise FYE2011 absolute-poverty reference-year premise. Current UK parameter/code support and author/internal reports do not substitute for the benchmark's primary methodology.
- Role checks incomplete: /private/tmp/policyengine-command-runs/881387a1b745/pr2685-external-review-20261026-review-policy.md; see recovered report.
Notes
- No confirmed critical, should-address or optional defect was established. The unresolved POLICY/source check is distinct from a demonstrated wrong value or citation; the auto-listed incomplete-policy-role gap refers to that same missing source premise.
- CODE independently qualified the unexecuted snippet's syntax/API forwarding, reform range/sign, FY/entity/weighted-return and current poverty-variable contracts, all nine unique link labels and relative documentation targets. POLICY separately qualified linked public claim rows and benchmark definitions within its source window; neither role reproduced private population estimates.
- POLICY preserved24fullpublicoriginals and corroborated all9deployed/merged225feedrows, relevant officialbenchmark tables/definitions, static-versus-behavioural costing and API/release distinctions. Three selected pages from2PDFs were rendered and visually audited. Manifestchecker24valid/0rejected. Originalsourcewindow20:46:26.767633–20:49:05.270474 lasted158.503/180seconds;20:49:18 was a reporting timestamp. No subsequent externalacquisition.
- Local model/Core/country imports, collection, tests, documentation build, environment installation and private dataset/impact validation were NOT RUN. The documentation build is matching-head hosted CI evidence. Existing internal reviews and PR claims were context, not proof.
- The raw CI rollup remains FAILURE with6SUCCESS/5workflow-qualified code-inapplicableSKIPPED/1FixCANCELLED. Docs121828844635/Lint121828845013/Changelog121828844468/Result121828941980 succeeded; skipped code jobs are not executed tests. Fresh Root metadata acquired actual Fix timestamps: canceled121828846244 ran20:28:29–20:28:55; successful121829134484 ran20:29:01–20:29:25. This establishes a later exact-head successful Fix, without cancellation-cause, attempt, all-jobs-reran, effective-universe or merge inference.
- Source acquisition used one original3minute window and is closed; the missing primary-methodology source was not fetched through another role after its deadline. All originals and rejected/unusable derivatives remain preserved, with only qualified evidence admitted to the final source manifest.
- The separate2687 qualification stopped before any substantive review after fullcomments revealed prior own-account public review and current-head implementation replies. Its internal PARTIAL/STOP and normal cleanup are preserved; no publicreview/guard workaround occurred. PR2685 is a new independent scope with its own original20:41:03 clock, not a2687retry.
- Scope/diff briefing took101.414seconds after setup, exceeding the60second aim. POLICY finished22.849seconds after its preferred20:56 role target, before20:58:03 checkpoint. Original canonical/source clocks unchanged; Root normal cleanup and final timing follow.
- The exact detached snapshot was removed normally after both reviewers completed (Root toola66d8e exit0), with no force. Reports and originals remain preserved; this is the sole timing finalization rerun.
Validation Summary
Exact6b98161832287c9ae5217020223d08c78b44dc0d clean detached snapshot; three docs/changelog files. Static API/reform/entities/FY/reference-year/current-variable/link contracts and scoped public source evidence checked. Matching-head docs build, Lint, Changelog and Result SUCCESS; raw12contexts6S5inapplicableSKIP1historicalFixCANCELLED, rawFAILURE retained. No local model/docs build/tests or private numerical reproduction. One specific DWP benchmark-methodology source premise unresolved.
Timing
setup seconds: 97.59s; scope seconds: 101.41s; parallel review seconds: 822.26s; policy role seconds: 720.48s; code role seconds: 565.36s; adjudication seconds: 0.00s; consolidation cleanup seconds: 83.15s; elapsed seconds: 1043.98s
Review Severity
COMMENT. Open findings: 0 critical, 0 should address, 0 suggestions.
…r, static scope Fixes #2704. The changelog fragment now carries #2704, the issue this PR closes. #1398 stays open. two-child-limit-full-rollout.md now describes what the TCL page holds. The snippet note names policyengine.py 6.11.0's pinned policyengine-uk 2.132.6. The static note says the labour-supply elasticities are zero, not that no behavioural response runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Queued for merge at head
|
Fixes #2704
Refs #1398, which stays open because it tracks baseline child-poverty benchmarks.
axiom: n/a: documentation only; no parameter, variable or rule changes.
PolicyEngine/policyengine-scorecard#225 merged on 2026-10-26 as eb1bb45a; the links below point at the claim ids it mints or attaches results to.
What changes
docs/book/validation/child-poverty-tcl.mdhard-coded PolicyEngine's two-child limit estimates beside the DWP, HM Treasury and Resolution Foundation figures (corrected in #1724). Max asked on 2026-10-23 whether that belonged in Scorecard instead. It now does: PolicyEngine/policyengine-scorecard#225 recomputes the comparison on policyengine.py 6.11.0's certified pair. Each row states whether the two concepts match (relative line convention, absolute line reference year, AHC or BHC, child count, financial-year mapping, static or post-behavioural cost) and keeps the earlier runs in its history.This page keeps the method and drops the numbers:
pe.uk.managed_microsimulation(), with the certified pair it loads (policyengine.py 6.11.0 pins policyengine-uk 2.132.6);reform-impacts-dashboard-plan.md's two references to the page, andtwo-child-limit-full-rollout.md's one, now describe what it holds.The page also names
in_absolute_poverty_fye2011_ahcand_bhc, which policyengine-uk 2.155.0 added. That is the FYE 2011 line in DWP's note. Scorecard's certified release (2.132.6) predates them, so Scorecard still constructs that line.Checks
in_relative_poverty_*: 60% of the person-weighted median of the world being calculated;in_poverty_*: the FYE 2025 reference;would_claim_uchas no formula, and the labour-supply elasticities are zero;Yis FYY-Y+1, fromconvert_to_fiscal_year_parameters.data/populations.json), with the values and statuses the page's notes describe.uvx --from ruff==0.17.0 ruff format --check .(the Lint job's pinned ruff, which also formats the page's Python block) andruff check .pass.Fixeswith a dedicated issue and the full-rollout backlink; both are fixed here, along with its two optional wording points. Its two out-of-scope findings ontwo-child-limit-full-rollout.mdpredate this PR (exceptions described as unmodelled; FYE 2021 constants from an older parameter file) and go to their own task.make documentationnot run locally: the change is markdown text and links only; CI's Test documentation builds runs.🤖 Generated with Claude Code