Repository navigation
fix(hashing): a type hashes as its logical type before and after registration (ITL-1137) #391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
eywalker
merged 7 commits into
main
from
eywalker/itl-1137-a-pipeline-hash-over-a-user-class-depends
Nov 14, 2026
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
2711ba1
docs(plans): ITL-1137 cause report and plan
2c8f493
test(hashing): a type hashes the same before and after registration (…
b036cca
fix(hashing): a type hashes as its logical type before and after regi…
c977ab6
fix(hashing): declare get_logical_type_for_python_type on the convert…
44fa71f
Merge remote-tracking branch 'origin/main' into eywalker/itl-1137-a-p…
19e7fa2
Merge remote-tracking branch 'origin/main' into eywalker/itl-1137-a-p…
3aef99b
fix(hashing): a parameter annotation resolves its class before and af…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| ### Fixed | ||
|
|
||
| - A schema that holds a user class (an ordinary dataclass, an `orcapod.dataclass` or a pydantic model, also nested in another class, inside `list[...]` or as `T | None`) hashes the same whether or not anything has registered the class yet (ITL-1137). A class gets its logical type when a function pod names it or a source is built from its values; before that, hashing a type looked the class up only and hashed it by module path and qualified name, with no kind and no fields, so two different classes of one name hashed alike and a pipeline hash taken before registration differed from the same hash taken after. Such a hash is kept by the node, so one pipeline could store one node at two paths, depending on what the process built first: a job looked at (a node's hash, path or preview) before its source existed, over a `SourceNode` slot followed by operators or by function pods that never name the class. Hashing a type now registers a class that a factory covers, so the hash before registration is the registered one; a registered class hashes exactly as before, and no golden value or stored path of a registered class moves. The same holds for a parameter annotated with such a class when a function is hashed (a function used as a value, and the function signature hash a function pod stores with every result row as `__pf_var_function_signature_hash`): a pod defined before its parameter's class was registered stored the name-based signature hash, one defined after it the registered one; now both store the registered one. The signature hash is a recorded column, not a lookup key, so stored results are found as before. A class Orcapod has no logical type for still hashes by its name. A schema hashed on its own no longer raises `TypeError` for a class not registered yet. A store written by a development build of `main` before this fix may hold, for such a node, function pipeline tables (`pdb_v2`) and operator caches (`odb_v1`) under a `schema:<hash>` path component computed from the unregistered class; those tables are not read again (the node now reads its tables at the registered path), and function results (`rdb_v2`, keyed by the function) are not affected. No release wrote such tables. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| ## `src/orcapod/hashing/semantic_hashing/` | ||
|
|
||
| ### H6 — A user class hashed before registration hashed by its name only | ||
| **Status:** resolved | ||
| **Severity:** high | ||
| **Issue:** ITL-1137 | ||
|
|
||
| `TypeObjectHandler` only looked a class up in the logical type registry. A user class (an | ||
| ordinary dataclass, an `orcapod.dataclass`, a pydantic model, also nested or inside `list[...]` | ||
| or `T | None`) gets its logical type when something registers it: a function pod naming it, a | ||
| source built from its values. Before that, the class hashed as `type:<module>.<qualname>`, with | ||
| no kind and no fields, so two different classes of one qualified name hashed alike, and the same | ||
| schema hashed differently before and after registration. A `Schema` nested in an identity | ||
| structure (a node's `(tag_schema, data_schema)`) is expanded as a mapping and never reaches | ||
| `SchemaHandler`, so nothing registered the class first; a `Schema` hashed on its own raised | ||
| `TypeError` instead. A pipeline hash taken before registration (a job looked at before its | ||
| source exists, a slot followed only by operators or by pods that never name the class) was | ||
| memoized, so one node wrote function pipeline tables (`pdb_v2`) and operator caches (`odb_v1`) | ||
| at two paths depending on what the process built first. | ||
|
|
||
| **Fix:** `TypeObjectHandler` resolves the type through | ||
| `type_converter.get_logical_type_for_python_type`, which registers a class a factory covers, and | ||
| `SchemaHandler` hashes each field type before its translatability check. A registered class | ||
| hashes exactly as before and no golden value moves; a class no factory covers still hashes as | ||
| `type:<module>.<qualname>`. Tests: `tests/test_hashing/test_type_hash_registration.py`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Script executed:
Repository: metamorphic-labs/orcapod-python
Length of output: 14732
Script executed:
Repository: metamorphic-labs/orcapod-python
Length of output: 5360
Script executed:
Repository: metamorphic-labs/orcapod-python
Length of output: 3256
Use the registering lookup for parameter annotations, so their hash does not depend on registration state.
canonical_annotation_strreads registered logical types withget_logical_type, which never registers. A covered class that has not been registered yet therefore hashes as its module and qualified name. Once any other hash registers the class, the same parameter annotation hashes as its logical identifier. A function such asdef f(p: Point) -> Nonecan get a different hash on a second call, or in a process that hashedPointfirst. This is the registration dependence that ITL-1137 targets. The return-annotation path already avoids it, becauseTypeObjectHandlerregisters before lookup.Switch the lookup to
get_logical_type_for_python_type. The method is already onTypeConverterProtocol, andTypeObjectHandleralready calls it for arbitrary type objects.Suggested fix
Registered classes and primitives keep their current strings. Unsupported classes still fall back to the name. The change affects only covered but unregistered classes in function annotations, and their function hashes change once. Check the golden suite and any stored
pdb_v2orodb_v1keys for such functions before merging.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Accepted and fixed in 3aef99b.
Reproduced first:
def f(p: Point) -> inthashed as a value gave5b523b91…beforePointwas registered andc8bfc858…after. It also reaches stored data.PythonDataFunction.__init__computes the function signature hash beforeFunctionPod.__init__registers the annotation types, so the__pf_var_function_signature_hashstored with every result row was9c5496fa…for a pod defined in a fresh process ande0f57781…when a source ofPointhad been built first. It is a recorded column, not a lookup key.canonical_annotation_strnow usesget_logical_type_for_python_type, asTypeObjectHandlerdoes. New tests: a function with a user-class parameter (also inlist[...] | None) hashes the same before and after registration, and two pods over one function store one signature hash whatever was registered first. Both failed without the fix. No golden value moves:semantic.jsonpins only functions with native annotations. The changelog fragment now covers the signature hash.I also searched for every other caller of the non-registering lookup (
get_logical_type(/get_by_python_type() to check for hash paths:hashing/hash_utils.py:78: this finding, fixed.core/operators/index.py:177,core/operators/pick.py:185: operator validation (which element typeIndex/Pickgive), not a hash path. Not changed.semantic_types/universal_converter.py(lines 431, 841, 1020, 1029, 1283, 1646): the converter's own registration and conversion internals and the lookups behindget_logical_type/get_logical_type_for_python_type. None hash a type.No other hash path reads the registry without registering.
Posted by a Claude Code agent on behalf of @eywalker.