Close the receipt stack's outstanding review Lows

changelog.d/receipt-review-lows.fixed.md

@@ -0,0 +1 @@
+Receipts no longer refuse a measure helper whose quoted `Annotated` metadata contains `|` (for example `List['Annotated[float, "weekly | monthly"]']`), still refuse a union hidden under a name a helper rebinds (`from typing import Tuple as Annotated`), and inspect deeply quoted annotations without a recursion error. A parameter path such as `'gov.hmrc.income_tax.rates.uk[0].threshold'` no longer counts toward the inspection limit, and a lazy loader whose module `__getattr__` raises no longer crashes the receipt. Receipts now refuse a typing generic over a local import of a module not yet loaded (for example `Optional[statistics.NormalDist]`), an attribute of something other than a module inside a typing generic or subscripted with a union (`List[pd.DataFrame.dtypes]`, `HTTPStatus.OK.phrase[0 | 0]`), a function holding a string too deeply nested to parse, and a function whose forward references exceed 100,000 inspected nodes. The `Literal` and `Annotated` exemption is withheld from a function when code anywhere in the module could rebind typing names by a route the receipt can see (`sys.modules`, `setattr` through an alias, a forward reference that calls code or stores to an attribute or subscript). The docs now say which uses of `id` and the other refused names are refused, and which routes the exemption does not follow.

docs/receipts.md

@@ -83,7 +83,7 @@ The accepted module shape also refuses these constructs:
 
 Anywhere in the module, code may not:
 - tell equal values apart by identity: `is` is allowed only against `None`, `True`, `False`, `...` or `type(...)`, and `type` may not be rebound. The compiler may make equal immutable constants one object, and a digest of values cannot tell one object from two equal ones.
-- use `id`, `operator.is_` or `is_not`, `globals`, `vars`, `locals`, `exec`, `eval`, `compile`, `__import__` or `breakpoint`, however imported, or import `builtins`;
+- use `id`, `operator.is_` or `is_not`, `globals`, `vars`, `locals`, `exec`, `eval`, `compile`, `__import__` or `breakpoint`, however imported, or import `builtins`. Any use of one of these names as a variable is refused: a read, an assignment target or a `del`, even when it is a parameter or local variable (reading a household identifier parameter named `id` is refused; name it `household_id`), because telling a local from the builtin would need scope analysis. Other positions are not variable uses and are allowed: a parameter or function of that spelling that the code never reads, an attribute (`simulation.id`, though `.is_` and `.is_not` are refused) and a keyword argument (`id=3`);
 - read annotation values through `typing.get_type_hints`, `inspect.signature`, `inspect.getfullargspec`, `get_annotations` or `from_callable`; these explicit metadata readers remain outside the accepted module shape.
 - declare a name `global`.
 - read dunder attributes other than `__init__` and `__name__`, or dunder names other than `__name__` and `__file__`. Reading a function's or module's own metadata is introspection.
@@ -95,7 +95,7 @@ Put measure values in literal constants or in default arguments.
 - **Versions.** At receipt build time, any recorded version of policyengine, policyengine-uk or policyengine-core must still match its installed version. Independently, each package's distribution metadata must not have been written after this process started. The builder compares the latest modification or change time of its `.dist-info` directory and of the existing `METADATA`, `RECORD`, `INSTALLER` and `direct_url.json` files against `psutil.Process().create_time()`, and refuses metadata written more than two seconds after it. On Linux psutil builds that start time from the boot time in whole seconds plus the start offset in clock ticks, so it can read up to about a second early (one second plus one tick). Without the margin, a process started just after an install finished would be refused. The margin is a blind window: metadata written within two seconds after the reported start is accepted, so an install that lands in that window is not detected. Past it, metadata is refused regardless of import order or whether a version was recorded; an unavailable metadata location is refused too. Restart the process and run again. This also refuses installing a package and then importing it in the same process, such as a notebook `pip install` without a restart, even when that import loaded the new code, unless the install finishes within two seconds of the process starting.
 - **Functions.** The source is compiled as a whole, so each function compiles in its module's context, imports included. The modifier and every other function defined in that file must use the module's own globals and match that code, nested code and constants included. Constants are compared by exact type, by IEEE 754 bytes (so a NaN's sign counts) and by how they share objects. Annotation functions that Python 3.14 compiles (`__annotate__`) are the compiler's, not the module's, and a generic function (`def helper[T](...)`, Python 3.12+) is checked through the wrapper the compiler makes to bind its type parameters.
 - **Function names.** A `def` name must still be bound to that function, not to another function of the same file or any other object.
-- **Code inside functions.** It runs after import, so a type it builds is made then: an expression in a function body, or the annotations, type-parameter bounds and class annotations of a helper defined there (quoted or not). typing caches its generic aliases (`List`, `Dict`, `Optional`, `Annotated`) and user generics (`Generic` subclasses, Pydantic models) by argument equality, and a union equals one holding its members in any order. So `List[Union[Decimal, float]]` built there can return an alias cached for `List[Union[float, Decimal]]`, and Pydantic tries a union's members in order. A function whose code subscripts such a generic with a union or `Literal` is refused. Builtin generics, `collections.abc` classes and classes whose `__class_getitem__` is implemented in C (NumPy's `ndarray` and `dtype`) build a new alias in source order and are allowed (`list[Decimal | float]`), as is a bare `Union[Decimal, float]`. Resolution is conservative. A name stands for every object the module, or any import anywhere inside the function, binds it to, so a helper importing `typing.Sequence` is caught even when a sibling imports `collections.abc.Sequence`. Every string in the function's code that parses as an expression is inspected as a type, since a forward reference such as `Optional['List[Union[Decimal, float]]']` is evaluated when something resolves it. `Literal` values and `Annotated` metadata are left as strings. Local aliases of typing objects (`L = List`) are not followed. The module's own function signatures are compared against their source instead (below).
+- **Code inside functions.** It runs after import, so a type it builds is made then: an expression in a function body, or the annotations, type-parameter bounds and class annotations of a helper defined there (quoted or not). typing caches its generic aliases (`List`, `Dict`, `Optional`, `Annotated`) and user generics (`Generic` subclasses, Pydantic models) by argument equality, and a union equals one holding its members in any order. So `List[Union[Decimal, float]]` built there can return an alias cached for `List[Union[float, Decimal]]`, and Pydantic tries a union's members in order. A function whose code subscripts such a generic with a union or `Literal` is refused. Builtin generics, `collections.abc` classes and classes whose `__class_getitem__` is implemented in C (NumPy's `ndarray` and `dtype`) build a new alias in source order and are allowed (`list[Decimal | float]`), as is a bare `Union[Decimal, float]`. Resolution is conservative. A name stands for every object the module, or any import anywhere inside the function, binds it to, so a helper importing `typing.Sequence` is caught even when a sibling imports `collections.abc.Sequence`. An import is resolved as Python binds it (`import a.b as c` binds the package's attribute `b`). One whose target is not loaded when the receipt is built, and an attribute of anything but a loaded module or one the module lacks then, could stand for any object: subscripting one with a union is refused, as is a typing generic subscripted with one, and the message says the receipt cannot resolve the name. Every string in the function's code that parses as an expression subscripting something the receipt can name, or holding another string, is inspected as a type, since a forward reference such as `Optional['List[Union[Decimal, float]]']` is evaluated when something resolves it (a label, or a parameter path such as `'gov.hmrc.income_tax.rates.uk[0].threshold'`, subscripts nothing the receipt can name and is skipped); past 100,000 such nodes in one function, or for a string too deeply nested to parse, the function is refused rather than left uninspected. `Literal` values and `Annotated` metadata are left as strings, but only where every object the name can stand for is that form and no code the receipt can see could change what the name, or any attribute in its chain, stands for. Any of these makes the strings under the name inspected as types: an assignment, a parameter, a `:=` target or lambda parameter inside a forward reference, an attribute store or deletion of that name (`typing.Annotated = Tuple`, `del t.Annotated`) or `setattr` or `delattr` with it, in the function or in any other function of the module (a function can call a sibling that does it), or a local import of a name or module path that code stores to. A function gets no such exemption at all when it or any other function of the module uses `modules` as an attribute, an imported name or a string (`sys.modules`, `from sys import modules`, `getattr(sys, 'modules')`), or uses `setattr`, `delattr` or `getattr` other than as a direct call with a literal name (`put = setattr`, `map(setattr, ...)`, a computed name); or when a forward reference in the function that calls something or stores to an attribute or subscript (and so runs code: `[0 for t.__dict__['Annotated'] in ...]` runs `__setitem__`) does either, or names one of those builtins, `import_module`, a refused name or a dunder. A string that does neither, such as a label `'id'` or `'__main__'`, runs no code; apart from the string `'modules'` and the inspection limit, it never costs the exemption, though a store or binding in it still counts. Like the rest of the receipt, this guards an honest producer; it is not a sandbox. Some routes are not followed: local aliases of typing objects (`L = List`, or `Amount = Union[Decimal, float]` and then `List[Amount]`), `getattr(typing, 'List')`, `importlib.import_module('typing')` results, `operator.getitem` or `reduce` over typing objects, what a call inside a forward reference does beyond the names above, forward references in other functions of the module, forward references built at run time (f-strings, concatenation), and changes made from other modules. A function using them can build such an alias unnoticed, so use a builtin generic there. The module's own function signatures are compared against their source instead (below).
 - **Function metadata.** Each function's name, qualified name and docstring must still be what its `def` made, compared as the compiler stores them. A generic function's `__type_params__` must still be bare type parameters of the names and kinds its source declares, in order, with no bound, constraints, default or declared variance. Each annotation must still hold what its source names, resolved against the module's globals and builtins (and those type parameters): named objects are compared by identity; reconstructed generics, unions and `ParamSpec` argument proxies must have the same origin and arguments, recursively and in order. Source arguments are resolved separately so typing's cache cannot hide a reordered union. Parameterised typing aliases are refused when their source order cannot be checked independently of that cache. Subscripted annotations that do not expose a typing origin and arguments, such as generated Pydantic model classes, are also refused. Quoted annotations and strings stored by `from __future__ import annotations` must match their source, and their resolvable type expressions must also match source order. This includes annotations consumed implicitly by libraries such as `functools.singledispatch`. It must use the builtins module and carry no other attributes (`helper.amount = 240` is state).
 - **Defaults.** Literal default arguments in the source, such as `rate=0.21`, must still be the function's defaults, with the same type.
 - **Literal constants.** Every literal constant must still hold its value, compared exactly: the same types throughout, floats by their bits, dictionary keys in order, for every literal kind including bytes, sets and complex numbers.