✨ Use Pydantic for parameter validation

[svlandeg]

Meta: marked as a "feature", but definitely also "breaking" !

Description

Make Pydantic a required dependency and rely on it for parameter validation. Picked >=2.5.3 which is the current pin for github-actions.

Click's ParamType hierarchy is now replaced by a three-layer design:

• TypeDescriptor objects store static facts about a parameter's type (in the new module coercion.py)
• Pydantic TypeAdapter objects are defined in the new module adapters.py
• RuntimeParam subclasses own coercion (also defined in coercion.py)

Further type-specific functionality is bundled in the new module param_types.py.

How to review this

• Probably read all of the below first 👇
• Note that _click/types.py and typer/_types.py are deleted
• First look at typer/param_types.py, then typer/adapters.py, then typer/coercion.py
• Look at the changes in typer/core.py
• Check all other changes in typer/*.py
• Check all changes in typer/_click/*.py
• Check all changes in the test suite & documentation

Extended/improved functionality

• More date time formats are now supported by default, instead of just the three [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]. Unix timestamps as seconds or milliseconds since the Unix epoch are now also supported.
• Started moving towards a TyperParameter, not quite finished yet with some ugly imports from within _click but we'll deal with those in follow-up work

Tests with same behaviour on master

• Added test_bool_convert_valid to ensure that the "bool" conversion has remained the same for all kinds of inputs.
• Rewrote several tests to use Typer public API instead of internals
• test_path_resolves, cf point 1) in the Fable review 👇

Breaking changes

(more or less in order of breakingness from most to least)

• Removed support for click_type and open bounds through min_open and max_open. This greatly simplifies the code base while still providing an alternative by setting parser instead.
• A list of choices is now shown as <list[Eggs|Bacon|Cheese]> (before there was no visual distinction between a single choice, or a list of them).
• When not providing a default for DateTime, we just show "<datetime>". Before, it would show the (only) 3 options, but those are not exhaustive anymore. When the user provides actual formats, those are shown (as before).
• Validation error messages have changed from Click's to Pydantic's phrasing. This mostly requires updating test suites and otherwise shouldn't impact users too badly.
• Removed repr functionality of param.type. This wasn't really used except for the tests that were recently added in ➖ Vendor Click and streamline Typer's functionality and code base #1774.
• A parameter typed as Any does NOT raise RuntimeError: Type not yet supported anymore, but instead just falls through and gets a generic TypeAdapter(annotation)

Note that as before, nothing in the module typer/_click (a remnant of the recent vendoring) should be used directly by users and all of it is subject to change in the near future. Any changes in those modules are not considered to be "breaking".

Bug fixes (also breaking bwd compat)

• def main(age: int = typer.Option(15.3)) will now throw a validation error by Pydantic instead of int(15.3) converting it to 15. I consider this a bug fix instead of a regression, and have added a test test_int_rejects_float_default for it.
• Tuples with types that aren't allowed to be None (e.g. tuple[str, int, bool]), will fail validation as soon as one of those elements is None. Our documented example of setting that tuple's default to (None, None, None) is not valid anymore, it should be set to None instead. cf. change to ‎docs_src/multiple_values/options_with_multiple_values/tutorial001_an_py310.py.
• Types are now inferred from the default if the parameter is untyped. Fixes Default value of `False` cast to `'False'` and is thus True #942, cf. new test test_default_infers_param_type.
• With Rich disabled, types were always shown for options, but for arguments only when they were "informative" like a tuple or datetime. This PR makes everything internally consistent, and similar to how it looks with Rich enabled, by always showing the types also for arguments.

Decision points

• _parse_cli_bool was created to ensure we still parse "" as False and to strip whitespace from something like " True ". This is just to mimic old Click behaviour. If we don't do this preprocessing (cleaner code base), the empty string and the non-stripped strings won't pass Pydantic validation and it would be slightly breaking.
• How to display the datetime type if no formats are provided by the user (cf ☝️ "Breaking changes")
• Should we allow an int or float as datetime input, to present Unix timestamps as seconds or milliseconds since the Unix epoch? Cf also Fable's concern in point 4 of its review below 👇
• Should we try to do less with Pydantic to avoid runtime increase? (cf point 7 in Fable's review)

AI Disclaimer

Cursor was used as a micro-managed junior. Every edit was reviewed & understood by me.

TODO

• Mention Pydantic dependency in the docs
• Update the tutorial, focus on (changed) error messages
• Fix coverage
• Merge in the fix from 🏷️ Make Parameter.name typed as str instead of str | None #1878
• I added tests for the change.
• Coverage stays at 100%.
• Metavar printing that came up in this PR has already been fixed in tests & documentation in 💥 Update metavar printing #1863.
• Self review with Fable, after which: "This v2 is a substantial improvement — most of what we discussed landed, and several of the fixes are better than what I suggested." 😎

Follow-up work

• Continue cleaning up code in typer/_click
• Expand code/docs/tests to support many more data types (should be straightforward?!)
• Test/document Pydantic models for re-usable CLI Options

[svlandeg]

This test currently mimics master behaviour 100%.

[github-actions]

📝 Docs preview

Last commit fc7fb16 at: https://3f7aeae1.typertiangolo.pages.dev

Modified Pages

• https://3f7aeae1.typertiangolo.pages.dev/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/arguments/default/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/arguments/envvar/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/arguments/help/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/arguments/optional/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/commands/arguments/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/commands/help/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/install/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/multiple-values/arguments-with-multiple-values/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/multiple-values/options-with-multiple-values/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/options/help/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/options/required/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/parameter-types/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/parameter-types/datetime/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/parameter-types/enum/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/parameter-types/number/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/parameter-types/path/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/parameter-types/uuid/ - (before)
• https://3f7aeae1.typertiangolo.pages.dev/tutorial/typer-app/ - (before)

[svlandeg]

Removed the note that "most are tiny" as that feels a bit untrue now that Pydantic is added to the list.

[svlandeg]

I've self reviewed this with Fable, and it's got opinions 🥲 And so do I! 😎

Review by Fable 5, transcribed & commented by me

Positives

• The direction is right and the destination is good
• Threading ctx/param through Pydantic's validation context is elegant
• the parity tests (test_bool_convert_valid, the new envvar-list-splitting tests, test_coercion_tuple_files) show real care about preserving behavior

Issues

1) Path-specific kwargs for non-Path annotations

Fable got confused and then confused me as well, so allow me to recap the issues involved here.

• on master, a str parameter with something like resolve_path=True would silently ignore the Path-specific setting and just resolve as str, as types.STRING would be returned before checking anything Path-like.

  • As a consequence, we have the test pytest.param(Annotated[str, typer.Option(..., resolve_path=True)], "<str>") which means this resolves to str, unchanged on this PR. So far so good, IMO.
  • One thing to note is that those Path-like args are being ignored silently, which is something to be reconsidered, but not in this PR.

• on master, a parameter typed as Any WOULD get parsed as a Path if it has something like allow_dash orso (cf here). And as a consequence, something like tmp_path / "first_dir" / "second_dir" / ".." / "file.txt" (notice the ..) will be resolved correctly to tmp_path / "first_dir" / "file.txt". I've edited this PR to also succeed here to avoid breaking behaviour, but it's something we can reconsider in the future. Some of the necessary functions for this are marked as "defined as such for bwd-compat".

• Both the str and Any behaviour is now captured in test_path_resolves, which behaves exactly the same on master as on this PR. Note that the test test_path_coerced should have actually tested this properly but doesn't, and I kept it unchanged in this PR for now (i.e. future work).

2) Heterogeneous tuples

Fable had a comment about heterogeneous tuples, especially with paths in them, not being processed correctly. There were indeed some edge cases not covered correctly, so I wrote some additional unit tests for them: test_coercion_tuple_file_and_str and test_coercion_tuple_file_modes, that both succeed on master. The code on this PR was fixed so both tests now succeed here as well.

3) None parsing in a list/tuple

Fable pointed out at an issue in the code where None values in a list or tuple would bypass Pydantic validation. That didn't seem right, so I fixed the code and wrote up two tests to ensure this can't happen: test_list_rejects_none and test_tuple_rejects_none.

As a consequence, we should use just None for the default value of a tuple, rather than (None, None, None), which would now go through coercion. Cf. The edits in docs_src\multiple_values\options_with_multiple\values\tutorial001_*_py310.py.

4) datetime parsing

Fable found an issue with the datetime type not showing up for Arguments, while it would for Options. That was actually coming from an inconsistency on master that is now fixed, by always showing the type, whether Rich is enabled or not, and whether it's an argument or an option.

Further, Fable worries about something like --timestamp 1969 automatically being interpreted as a datetime near the epoch instead of parsed as a year. In other words, Pydantic allowing int's and float's might produce subtle errors for users. Then again, I personally kind of like enabling this feature. Maybe it should be opt-in?

5) Assert statements

Fable flags some changes from TypeError to assert statements, e.g. in TyperParameter's constructor. These are intentionally though, as the code should only be called internally and you wouldn't expect TyperArgument objects to be created directly. As such, an assert feels more natural. This also falls into the category of trying to have all tests only use public-facing API behaviour to truly test user code and not internals.

6) Showing "raw" Pydantic messages

get_error_msg doesn't have a "translation" layer anymore and just passes Pydantic error messages through. I think this is fine and reduces code on our end. Fable worries about minor changes in Pydantic's wordings breaking our test suite, which is a valid concern. Maybe something to reconsider if this does come up in the future.

7) Startup-cost structure

Fable flagged the added startup cost for importing Pydantic and creating TypeAdapters up front.

So I ran a benchmark to benchmark this on a mid-sized app (~30 commands × 5 params):

On master:

   9.3 ms  python startup
  32.0 ms  import typer
  42.5 ms  import pydantic
  33.8 ms  import app (register cmds)
  38.9 ms  build command tree
  92.0 ms  full --help

Before more edits (aka at time of Fable's review):

   9.8 ms  python startup
  81.5 ms  import typer
  45.5 ms  import pydantic
  86.6 ms  import app (register cmds)
 110.4 ms  build command tree
 153.6 ms  full --help

Then, I implemented:

• Lazy definition of RunTimeParam to avoid defining TypeAdapter objects (too) early
• Lazy import of Pydantic
• Let bool flags use a pass-through RuntimeParam so they bypass Pydantic

After that:

  10.0 ms  python startup
  33.4 ms  import typer
  42.8 ms  import pydantic
  35.8 ms  import app (register cmds)
  40.8 ms  build command tree
  94.2 ms  full --help

Which is noice!

Then, comparing actual tool call:

master:

  40.7 ms  CLI call

this PR:

  96.5 ms  CLI call

This is a substantial relative increase, and mostly dominated by the actual import (~43 ms of the 56ms difference).

We could remedy some of this by e.g. only putting complex types through Pydantic and use a fall through path for int etc, like we do now for bool. That would make the cost less for simpler commands if they end up not needing Pydantic at all. But it would kind of go against the philosophy of doing this PR in the first place, and it would make the code more complex & less nice to maintain. So I'll defer this and add a note into "decision points" ☝️

Good things to keep as-is

• RuntimeParam.coerce faithfully ports the old type_cast_value contract (None → ()/None, string-rejection for multi-value)
• enum/Literal choice handling now returns original members through one code path with correct token_normalize_func and case-folding
• arguments finally show their range in help (ID [0<=x<=1000; required])
• list[bool] split-flag detection is a thoughtful edge case
• scalar-in-default_map → single-item list is a nice fix
• type-inference-from-default fixes Default value of `False` cast to `'False'` and is thus True #942 with good parametrized coverage
• rewriting tests against the public CLI surface instead of constructing TyperOption by hand is the right long-term call

[svlandeg]

This module ended up being relatively small, we could also fold it in somewhere else.