Repository navigation
Preserve supplied formula inputs in DataFrame exports - #2498
Merged
Merged
Conversation
juaristi22
marked this pull request as ready for review
October 18, 2026 11:28
vahid-ahmadi
left a comment
Collaborator
There was a problem hiding this comment.
Looks right, fine to approve.
@juaristi22, the default export now keeps supplied values for variables with formulas. The state_pension_reported round trip in #2297 keeps £9,500, and include_computed_variables=True still goes through Core unchanged. Core 3.33.2's own to_input_dataframe excludes formula variables by design, so overriding it in UK is the right place for the fix.
- should:
policyengine_uk/utils/input_dataframe_export.py:8-31, 39, 62adds new reads of Core private state:holder._user_input_storage,_memory_storage,_disk_storage,_stores_user_input,simulation._get_visible_branch_namesand, when present,simulation._get_recorded_input_array. #2175 (and #2495 for the parameter side) is moving the UK package off exactly these fields. Please either add this module to #2175's checklist, or raise the Core floor to 3.33.2 and drop_legacy_recorded_array. Otherwise the fallback will copy Core internals that can change underneath it. - nit:
_legacy_recorded_arraymirrors Core 3.33.2's tier precedence by hand (memory then disk, skipping derived values). A test that runs both paths on the same fixture and asserts the same arrays would catch drift. If point 1 drops the fallback, this goes too.
What I checked
- Read the full diff and issue #2297, and compared the code with Core 3.33.2's
_get_recorded_input_arrayandto_input_dataframe. pytest test_input_dataframe_export.py test_dataframe_dataset.py: 103 passed (locked Core 3.33.2).- Reverted
simulation.pyto main.test_input_dataframe_export.pythen fails 14 of 25. ruff checkandruff format --checkon the changed Python files: clean.- Hosted CI on this head: all green.
- Not checked: the Core 3.32.27 fallback path, since my environment has only the locked 3.33.2.
🤖 Generated with Claude Code
vahid-ahmadi
approved these changes
Oct 18, 2026
vahid-ahmadi
left a comment
Collaborator
There was a problem hiding this comment.
Approving, per my review above. The shoulds and nits can follow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2297
Default input exports dropped dataset-supplied values for variables with formulas. In the reported example, £9,500 of State Pension became £0 after export and reload. Export surviving supplied arrays, including formula and adds inputs, while excluding calculated caches and model-generated values. Preserve period/entity handling, identifiers and the existing
include_computed_variables=Truepath.Reproducer:
policyengine_uk/tests/test_input_dataframe_export.py::test_dataset_formula_input_round_trip_preserves_final_pensionValidation: 51 distinct export, DataFrame and input-reset cases pass independently on each Core version: installed 3.32.27 and an official-wheel 3.33.2 package overlay. The pension regression reproduces £9,500 → £0 before the fix and £9,500 → £9,500 afterward. Native formatting/lint passes; independent code and engineering/source reviews completed with no findings or material gaps. Runtime: Python 3.13.14 and microdf 1.2.1; full-lock environment validation was not run.
axiom: n/a — input serialization/provenance engineering; policy formulas and parameters are unchanged. No encoded-parity claim.