Skip to content

fix: round-trip serialization of Ellipsis in variadic tuple and Callable types#12062

Merged
anakin87 merged 2 commits into
deepset-ai:mainfrom
winklemad:fix-variadic-tuple-type-serialization
Jul 21, 2026
Merged

fix: round-trip serialization of Ellipsis in variadic tuple and Callable types#12062
anakin87 merged 2 commits into
deepset-ai:mainfrom
winklemad:fix-variadic-tuple-type-serialization

Conversation

@winklemad

Copy link
Copy Markdown
Contributor

serialize_type rendered the ... in types like tuple[int, ...] and Callable[..., X] as the literal string "Ellipsis" (it fell through to str(Ellipsis)), which deserialize_type then rejected — the "Ellipsis" token hits the builtin-type gate and is correctly refused since Ellipsis isn't a type.

The result is a serialization round-trip break: any component given a variadic-tuple or Callable[..., X] type — e.g. OutputAdapter(output_type=tuple[int, ...]), ConditionalRouter/MetadataRouter route types, ListJoiner/BranchJoiner, LLMEvaluator, an Agent State schema — produces a pipeline that Pipeline.dumps() writes but Pipeline.loads() hard-rejects.

Fix: serialize_type emits "..." (the literal Python itself uses) for Ellipsis, and deserialize_type resolves the non-type singletons (None, NoneType, ..., plus the legacy "Ellipsis") up front — before the module-path branch, since the dots in "..." would otherwise be read as a module path. Covers both variadic tuples and Callable[..., X]; legacy "Ellipsis" payloads still load. Added round-trip regression tests for variadic tuples and Callable-with-Ellipsis.

…ble types

serialize_type rendered the `...` in types like tuple[int, ...] and
Callable[..., X] as the literal string "Ellipsis", which deserialize_type
then refused as a non-type builtin. So a component using such a type (for
example OutputAdapter(output_type=tuple[int, ...])) could be written by
Pipeline.dumps() but failed to load back with Pipeline.loads().

Serialize Ellipsis to "..." (the literal Python itself uses when rendering
these types) and resolve it, together with None and NoneType, before the
generic and module-path handling in deserialize_type. The legacy "Ellipsis"
literal is still accepted so pipelines serialized by older versions keep
loading.
@winklemad
winklemad requested a review from a team as a code owner July 18, 2026 19:06
@winklemad
winklemad requested review from anakin87 and removed request for a team July 18, 2026 19:06
@vercel

vercel Bot commented Jul 18, 2026

Copy link
Copy Markdown

@winklemad is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Jul 18, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/core/pipeline
  pipeline.py
  haystack/utils
  type_serialization.py 219
Project Total  

This report was generated by python-coverage-comment-action

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, the fix looks good!

I only found two minor opportunities to improve this PR...

Comment thread haystack/utils/type_serialization.py Outdated
# "..." would otherwise be read as a module path, and the None/Ellipsis singletons would be refused
# by the builtin type gate. "Ellipsis" is also accepted so that types serialized by older versions
# (which emitted the literal "Ellipsis") can still be read back.
special_literals: dict[str, Any] = {"None": None, "NoneType": NoneType, "...": ..., "Ellipsis": ...}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: I'd use a _SPECIAL_LITERALS constant defined on the module

Comment thread test/utils/test_type_serialization.py Outdated
Comment on lines +253 to +272
def test_output_type_serialization_ellipsis():
# The `...` (Ellipsis) singleton is serialized to the literal "..." Python itself uses when rendering
# variadic tuples and Callable, not to str(Ellipsis) == "Ellipsis" (which is not a type).
assert serialize_type(tuple[int, ...]) == "tuple[int, ...]"
assert serialize_type(Tuple[int, ...]) == "typing.Tuple[int, ...]"
assert serialize_type(Callable[..., int]) == "typing.Callable[..., int]"


def test_output_type_round_trip_variadic_tuple_and_callable_ellipsis():
# Regression: `...` used to be serialized as "Ellipsis", which deserialize_type then rejected as a
# non-type builtin, so a type Haystack serialized itself could not be read back.
for type_ in [
tuple[int, ...],
tuple[str, ...],
tuple[dict[str, int], ...],
Tuple[int, ...],
Callable[..., int],
Callable[..., str],
]:
assert deserialize_type(serialize_type(type_)) == type_

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def test_output_type_serialization_ellipsis():
# The `...` (Ellipsis) singleton is serialized to the literal "..." Python itself uses when rendering
# variadic tuples and Callable, not to str(Ellipsis) == "Ellipsis" (which is not a type).
assert serialize_type(tuple[int, ...]) == "tuple[int, ...]"
assert serialize_type(Tuple[int, ...]) == "typing.Tuple[int, ...]"
assert serialize_type(Callable[..., int]) == "typing.Callable[..., int]"
def test_output_type_round_trip_variadic_tuple_and_callable_ellipsis():
# Regression: `...` used to be serialized as "Ellipsis", which deserialize_type then rejected as a
# non-type builtin, so a type Haystack serialized itself could not be read back.
for type_ in [
tuple[int, ...],
tuple[str, ...],
tuple[dict[str, int], ...],
Tuple[int, ...],
Callable[..., int],
Callable[..., str],
]:
assert deserialize_type(serialize_type(type_)) == type_

Let's add

pytest.param("typing.Callable[..., int]", Callable[..., int]),
pytest.param("typing.Callable[..., str]", Callable[..., str]),

to TYPING_AND_TYPE_TESTS, so that the above-mentioned tests become fully redundant and we can remove them.

…a round-trip params

Move the special-literal lookup table in deserialize_type to a module-level
_SPECIAL_LITERALS constant, and add typing.Callable[..., int] / [..., str] to
TYPING_AND_TYPE_TESTS so the variadic-tuple and Callable ellipsis cases are
exercised by the shared serialize/deserialize round-trip tests. The two
standalone ellipsis tests are now redundant and removed.
@winklemad

Copy link
Copy Markdown
Contributor Author

Done, thanks for the review!

  • Extracted the lookup into a module-level _SPECIAL_LITERALS constant and reference it from deserialize_type.
  • Added pytest.param("typing.Callable[..., int]", Callable[..., int]) and pytest.param("typing.Callable[..., str]", Callable[..., str]) to TYPING_AND_TYPE_TESTS. The variadic-tuple cases were already in that list, so with Callable added the two standalone tests were fully redundant — removed both. I kept test_output_type_deserialization_legacy_ellipsis_literal, which covers reading back the legacy "Ellipsis" literal (not exercised by the round-trip params).

@anakin87
anakin87 self-requested a review July 21, 2026 07:37

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@anakin87
anakin87 enabled auto-merge (squash) July 21, 2026 07:37
@github-actions github-actions Bot added the type:documentation Improvements on the docs label Jul 21, 2026
@anakin87
anakin87 merged commit 04fe770 into deepset-ai:main Jul 21, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants