Skip to content

Track the provenance of parsed values - #975

Merged
mauvilsa merged 1 commit into
mainfrom
value-provenance
Sep 15, 2026
Merged

mauvilsa merged 1 commit into
mainfrom
value-provenance

Conversation

@mauvilsa

Copy link
Copy Markdown
Owner

What does this PR do?

A parse combines defaults, config files, environment variables and command line arguments, but afterwards it was not known where each value came from. Now every value's source is recorded, and shown in errors and by a new provenance flag of the print config argument:

Parser key "val":
  Expected a <class 'int'>. Got value: abc
  Source: config file configs/exp.yaml:2
    2 | val: abc

Sources are kept per leaf in a __provenance__ slot of each Namespace, also for namespaces in lists and dicts, so equality and dumps are unaffected and there is no new public API. A context variable holds the source being applied. Code that processes already parsed values, e.g. type adaptation or merging, runs without one, so values keep their sources. Line numbers are only for YAML, computed lazily.

Also fixed: list adaptation deep copied the whole list for every item, making parsing quadratic, e.g. a minute for 10000 numbers. Parsing is now within noise of main, and faster for lists.

Behavior changes: errors due to a value end with a Source: line, the comments flag is always accepted, and assigning __class__ to convert an argparse.Namespace no longer works.

Not addressed: single source for plain lists like list[int], errors for nested config values point to the top level key, and repeated --x+= re-adapting previous items.

Before submitting

  • Did you read the contributing guideline?
  • If you used a coding agent, did you fully understand and validate all generated code and ensure it follows the contributing guidelines?
  • Did you update the documentation? (readme and public docstrings)
  • Did you write unit tests such that there is 100% coverage on related code? (required for bug fixes and new features)
  • Did you verify that new and existing tests pass locally?
  • If this is a bug fix, did you verify that the tests fail without the code fix?
  • Did you make sure that all changes preserve backward compatibility?
  • Did you update the CHANGELOG including a pull request link? (not for typos, docs, test updates, or minor internal changes/refactors)

@mauvilsa mauvilsa added bug Something isn't working enhancement New feature or request labels Sep 15, 2026
greptile-apps[bot]

This comment was marked as off-topic.

@codecov

codecov Bot commented Sep 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (473f7cb) to head (f5d50db).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##              main      #975    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           27        27            
  Lines         8777      9043   +266     
==========================================
+ Hits          8777      9043   +266     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@sonarqubecloud

Copy link
Copy Markdown

@mauvilsa
mauvilsa merged commit 945c917 into main Sep 15, 2026
32 checks passed
@mauvilsa
mauvilsa deleted the value-provenance branch September 15, 2026 05:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant