From 03243f29fd9bf47b31e100c58f3ee4e32efb6d2f Mon Sep 17 00:00:00 2001 From: Alex Mazzeo Date: Wed, 2 Sep 2026 14:19:52 -0700 Subject: [PATCH 1/3] Add generated spec based on Python, .NET, and Java prototype behavior for collaborative review --- features/transfer_types/spec.md | 172 ++++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 features/transfer_types/spec.md diff --git a/features/transfer_types/spec.md b/features/transfer_types/spec.md new file mode 100644 index 00000000..3ea246a1 --- /dev/null +++ b/features/transfer_types/spec.md @@ -0,0 +1,172 @@ +# Transfer Type Converter Behavioral Specification + +Status: Draft + +Last updated: 2026-09-02 + +## Purpose + +This document defines the portable behavior of Temporal transfer type converters. It specifies observable conversion semantics without prescribing SDK-specific APIs, generic type mechanisms, or converter construction strategies. + +## Terminology + +- **Model type:** The user-facing type associated with a transfer type converter. +- **Transfer type:** The serializer-facing type produced from a model value. +- **Transfer value:** A value of the transfer type. +- **Payload conversion:** Conversion between an in-memory value and a Temporal `Payload`. +- **Payload codec:** A byte-to-byte transformation applied to a payload, such as encryption or compression. +- **Failure details:** Typed payloads stored in application failures, cancellations, timeout heartbeat details, reset failures, or nested failure causes. + +## Normative behavior + +### Conversion order + +For outbound values, an SDK MUST apply operations in this order: + +1. Locate the transfer converter from the top-level model value's runtime type. +2. Invoke the transfer converter when one is present. +3. Pass the resulting transfer value to the configured payload converter. +4. Apply external storage processing and payload codecs according to the SDK's existing data-converter contract. + +For inbound values, an SDK MUST apply the inverse order: + +1. Apply payload codecs and external storage retrieval according to the SDK's existing data-converter contract. +2. Locate the transfer converter from the requested model type. +3. Ask the configured payload converter to decode the payload as the transfer type. +4. Invoke the transfer converter to reconstruct the requested model value. + +The transfer layer MUST NOT replace or bypass the configured payload converter, failure converter, payload codecs, external storage provider, or serialization context. + +### Top-level values only + +Transfer conversion MUST apply only to values directly handed to the SDK payload-conversion boundary. It MUST NOT recursively inspect fields, properties, collection elements, or other nested values. + +Nested serialization remains the responsibility of the configured payload converter. + +### Exactly one transfer step + +An SDK MUST perform at most one transfer conversion for a top-level value. + +If model type `A` converts to transfer value `B`, and `B` also has a transfer converter, the SDK MUST pass `B` directly to the configured payload converter. It MUST NOT invoke the converter associated with `B`. + +On decode, the payload converter MUST decode directly to the selected transfer type. Only the converter associated with the requested model type may reconstruct the final value. + +### Encode and decode type selection + +Encoding MUST select a converter using the top-level value's runtime type. A null outbound value has no runtime model type and MUST pass directly to the configured payload converter. + +Decoding MUST select a converter using the requested user-facing type. If the SDK has no requested type information, it MUST return the configured payload converter's normal result without applying transfer reconstruction. + +The SDK MUST provide the requested model type information available in its type system when selecting the transfer type and reconstructing the model value. This includes generic arguments when the SDK's conversion API preserves them. + +### Converter declaration and lookup + +A transfer converter declaration applies only to the exact model type on which it is declared. Declarations MUST NOT be inherited from a base type. + +A subclass or other derived type MAY declare its own converter independently of its base type. + +Runtime encoding MUST look only for a converter declared by the exact runtime type. Inbound decoding MUST look only for a converter declared by the exact requested type. + +Every selected converter MUST provide a concrete, non-null transfer type for inbound payload conversion. The SDK MUST fail clearly if the converter does not provide a valid transfer type. It MUST NOT fall back to payload metadata inference after selecting a converter. + +### Transfer value nullability + +When a converter has been selected for an inbound model type, the SDK MUST invoke its reconstruction hook even when the decoded transfer value is null. + +This permits a non-null model value to use null as its transfer representation. + +An outbound null model value MUST pass through without converter lookup because no runtime model type is available. + +### Failure conversion + +Failure semantics MUST remain owned by the configured failure converter. + +The failure converter MUST use the same transfer-aware payload-conversion path used for ordinary arguments and results when encoding or decoding typed failure payloads. This includes: + +- Application failure details. +- Cancellation details. +- Timeout last-heartbeat details. +- Reset workflow failure details. +- Details contained in nested failure causes. + +Payload codecs and external storage transformations MUST be applied exactly once to failure payloads and in their normal order. + +Transfer conversion does not apply to untyped failure metadata such as messages, stack traces, failure types, retry state, or encoded failure attributes. Existing payload-codec behavior for encoded failure attributes remains unchanged. + +### Raw payloads + +A raw payload wrapper MUST preserve its existing pass-through behavior. A transfer converter MUST NOT reinterpret a raw payload wrapper as a model value. + +### Configured converter authority + +The configured payload converter remains authoritative over the transfer representation's wire format. Transfer conversion MUST NOT add transfer-specific payload metadata or choose a wire encoding independently. + +The configured failure converter remains authoritative over the mapping between language exceptions and Temporal failure protos. + +### Serialization context + +Transfer conversion MUST preserve serialization context propagation to payload converters, failure converters, codecs, and external storage components. + +### Converter lifetime and concurrency + +Converter construction, lookup, and invocation MUST be safe under concurrent serialization. An SDK MAY reuse converter instances, so converter implementations MUST satisfy the concurrency requirements documented by that SDK. + +The number of converter instances and the time at which they are constructed are not part of the portable behavior. + +### Invalid declarations and callback failures + +An invalid converter declaration MUST fail clearly before conversion completes. Examples include an incompatible converter, a converter that cannot be constructed according to the SDK's API, or an invalid transfer type. + +Exceptions raised by transfer callbacks MUST remain observable as conversion failures. The SDK MUST NOT silently fall back to ordinary model serialization after selecting a converter. + +## Conformance requirements + +An implementation conforms to this specification when all of the following hold: + +- Ordinary arguments and results satisfy the normative conversion order. +- Failure details use transfer conversion without bypassing a configured failure converter. +- Payload codecs and external storage transformations are applied exactly once. +- A selected converter reconstructs from a null transfer value. +- Top-level-only and one-step behavior have explicit regression tests. +- Raw payload pass-through remains intact. +- Context propagation remains intact. +- Exact converter lookup is implemented and documented as non-inherited behavior. +- Workflow history containing transfer-represented values can be replayed. + +## Required conformance tests + +### Payload conversion + +- Annotated argument and result round trip. +- Mixed annotated and ordinary values preserve position and type. +- Missing payloads still produce language-default values. +- Available requested type information reaches converter selection and reconstruction. +- Null outbound model passes through. +- Non-null model represented by null reconstructs through its hook. +- Raw payload wrapper passes through unchanged. + +### Conversion boundaries + +- Annotated `A` converts to annotated `B` without invoking `B`'s converter. +- Annotated nested values do not invoke their converters. +- An unannotated subclass does not inherit an annotated base converter. +- A subclass can declare its own converter independently. + +### Failures + +- Application failure details round trip. +- Cancellation details round trip. +- Timeout last-heartbeat details round trip. +- Nested failure causes round trip their details. +- Custom failure converter remains authoritative. +- Payload codecs transform each failure payload exactly once. + +### Integration surfaces + +- Workflow client and worker. +- Workflow history replay. +- Activity stubs and standalone activity client. +- Local activities and activity heartbeats. +- Nexus client and worker. +- Schedule client create and describe operations. +- Test activity environment. From 4cb382ccbb36d5b642f61f4da91c6da940858da4 Mon Sep 17 00:00:00 2001 From: Alex Mazzeo Date: Wed, 2 Sep 2026 17:30:56 -0700 Subject: [PATCH 2/3] More WIP spec --- .../transfer_types/spec.md | 149 +++++++++++++++--- 1 file changed, 130 insertions(+), 19 deletions(-) rename features/{ => data_converter}/transfer_types/spec.md (60%) diff --git a/features/transfer_types/spec.md b/features/data_converter/transfer_types/spec.md similarity index 60% rename from features/transfer_types/spec.md rename to features/data_converter/transfer_types/spec.md index 3ea246a1..3415ea73 100644 --- a/features/transfer_types/spec.md +++ b/features/data_converter/transfer_types/spec.md @@ -1,55 +1,166 @@ # Transfer Type Converter Behavioral Specification -Status: Draft - Last updated: 2026-09-02 -## Purpose +## Motivation + +Transfer Type Conversion provides a way to translate to and from a Payload serialization friendly format, e.g. a protobuf object, to a more ergonomic or language idiomatic type. +TransferTypeConverters are added to the API facing type and specify the target "transfer type" which is then used by the configured Payload Converter serialize to/from a Payload. + +For example, given the protobuf definition + +```proto3 + message WorkflowExecution { + string workflow_id = 1; + string run_id = 2; + } + + message PauseRequest { + WorkflowExecution execution = 1; + string reason = 2; + } +``` + +Using a generated protobuf object in Python may force an awkward interface: + +```python +request = pause_pb2.PauseRequest(reason="maintenance") + +# This does not test presence. Protobuf returns an empty message. +assert request.execution is not None +assert request.execution.workflow_id == "" + +# You must use the protobuf specific inspection for presence. +if request.HasField("execution"): + print(request.execution.workflow_id) +else: + print("No workflow execution provided") +``` -This document defines the portable behavior of Temporal transfer type converters. It specifies observable conversion semantics without prescribing SDK-specific APIs, generic type mechanisms, or converter construction strategies. +Adding a TransferTypeConverter allows you to construct a more idiomatic type: + +```python +@dataclass(frozen=True, slots=True) +class WorkflowExecution: + workflow_id: str + run_id: str + + +# Add the PauseRequestConverter that converts to and from pause_pb2.PauseRequest +@transfer_type_convertible(PauseRequestConverter) +@dataclass(frozen=True, slots=True) +class PauseRequest: + reason: str + execution: WorkflowExecution | None = None + + +request = PauseRequest(reason="maintenance") + +# Can use normal Python optional-value semantics: +if request.execution is not None: + pause_workflow(request.execution.workflow_id) +else: + print("No workflow execution provided") +``` ## Terminology - **Model type:** The user-facing type associated with a transfer type converter. +- **Model value:** A value of the model type. - **Transfer type:** The serializer-facing type produced from a model value. - **Transfer value:** A value of the transfer type. -- **Payload conversion:** Conversion between an in-memory value and a Temporal `Payload`. -- **Payload codec:** A byte-to-byte transformation applied to a payload, such as encryption or compression. -- **Failure details:** Typed payloads stored in application failures, cancellations, timeout heartbeat details, reset failures, or nested failure causes. +- **Transfer type converter:** The conversion logic to map between the model type and the transfer type. + +## Behaviors + +### Conversion wraps configured payload converters -## Normative behavior +#### Encoding -### Conversion order +When encoding a value, the transfer type converter is applied to the model type **before** the payload converter so the payload converter receives the transfer type. The external storage processing and codec are applied as normal to the resulting payload. -For outbound values, an SDK MUST apply operations in this order: +```mermaid +sequenceDiagram + actor App as Application + participant Client + participant Converter as Transfer type converter + participant Serializer as Payload converter + participant Codec as Codec & ExtStore + participant Server as Temporal Server + + App->>+Client: Send request containing model value + + Note over Client,Converter: The SDK finds the converter
associated with the model type + + Client->>+Converter: to_transfer_type(model value) + Converter-->>-Client: Transfer value + + Client->>+Serializer: to_payload(transfer value) + Serializer-->>-Client: payload + + Client-->+Codec: process(payloads) + Codec-->-Client: encoded payloads + + Client->>-Server: Send outgoing request + +``` 1. Locate the transfer converter from the top-level model value's runtime type. 2. Invoke the transfer converter when one is present. 3. Pass the resulting transfer value to the configured payload converter. 4. Apply external storage processing and payload codecs according to the SDK's existing data-converter contract. -For inbound values, an SDK MUST apply the inverse order: +#### Decoding + +When decoding a value, external storage processing and codec are applied as normal to the payload. Then the transfer type converter is applied to the transfer type **after** the payload converter so. + +```mermaid +sequenceDiagram + actor App as Application + participant Client + participant Converter as Transfer type converter + participant Serializer as Payload converter + participant Codec as ExtStore & Codec + participant Server as Temporal Server -1. Apply payload codecs and external storage retrieval according to the SDK's existing data-converter contract. + Server->>+Client: Send incoming response + + Client->>+Codec: process(encoded payloads) + Codec-->>-Client: payloads + + Note over Client,Converter: The SDK finds the converter
associated with the model type + + Client->>+Serializer: from_payload(payload, transfer type) + Serializer-->>-Client: Transfer value + + Client->>+Converter: from_transfer_type(transfer value, model type) + Converter-->>-Client: Model value + + Client-->>-App: Return response containing model value +``` + +1. Apply payload codecs and external storage retrieval. 2. Locate the transfer converter from the requested model type. 3. Ask the configured payload converter to decode the payload as the transfer type. 4. Invoke the transfer converter to reconstruct the requested model value. -The transfer layer MUST NOT replace or bypass the configured payload converter, failure converter, payload codecs, external storage provider, or serialization context. - ### Top-level values only -Transfer conversion MUST apply only to values directly handed to the SDK payload-conversion boundary. It MUST NOT recursively inspect fields, properties, collection elements, or other nested values. +Transfer conversion only applies to top-level values and does not recursively inspect fields for transfer types. -Nested serialization remains the responsibility of the configured payload converter. +Nested serialization is the responsibility of the configured payload converter. ### Exactly one transfer step -An SDK MUST perform at most one transfer conversion for a top-level value. +Transfer type converters are applied at most once. For example, given: + +- type `A` has transfer type converter `converterA` +- `converterA` produces the transfer type `B` +- type `B` has transfer type converter `converterB` -If model type `A` converts to transfer value `B`, and `B` also has a transfer converter, the SDK MUST pass `B` directly to the configured payload converter. It MUST NOT invoke the converter associated with `B`. +When encoding type `A`, the SDK does not inspect the output type `B` for a transfer type converter and thus does not apply `converterB`. -On decode, the payload converter MUST decode directly to the selected transfer type. Only the converter associated with the requested model type may reconstruct the final value. +Similarly when decoding, the SDK will convert using only the transfer type encoder registered on the requested model type. ### Encode and decode type selection From 2aae7cf74a0629211b7f5195a13191f405022ba4 Mon Sep 17 00:00:00 2001 From: Alex Mazzeo Date: Thu, 3 Sep 2026 16:05:20 -0700 Subject: [PATCH 3/3] Edit the transfer_types spec --- .../data_converter/transfer_types/spec.md | 133 ++++-------------- 1 file changed, 24 insertions(+), 109 deletions(-) diff --git a/features/data_converter/transfer_types/spec.md b/features/data_converter/transfer_types/spec.md index 3415ea73..3d96042d 100644 --- a/features/data_converter/transfer_types/spec.md +++ b/features/data_converter/transfer_types/spec.md @@ -105,7 +105,7 @@ sequenceDiagram ``` -1. Locate the transfer converter from the top-level model value's runtime type. +1. Locate the transfer converter from the declared model type. 2. Invoke the transfer converter when one is present. 3. Pass the resulting transfer value to the configured payload converter. 4. Apply external storage processing and payload codecs according to the SDK's existing data-converter contract. @@ -150,7 +150,7 @@ Transfer conversion only applies to top-level values and does not recursively in Nested serialization is the responsibility of the configured payload converter. -### Exactly one transfer step +### At most one transfer step Transfer type converters are applied at most once. For example, given: @@ -162,122 +162,37 @@ When encoding type `A`, the SDK does not inspect the output type `B` for a trans Similarly when decoding, the SDK will convert using only the transfer type encoder registered on the requested model type. -### Encode and decode type selection +### TransferTypeConverter Selection -Encoding MUST select a converter using the top-level value's runtime type. A null outbound value has no runtime model type and MUST pass directly to the configured payload converter. +Converting to the transfer type should use the type hint for the target execution rather than the value's runtime type. For example: -Decoding MUST select a converter using the requested user-facing type. If the SDK has no requested type information, it MUST return the configured payload converter's normal result without applying transfer reconstruction. +```java + // Given this workflow + @WorkflowMethod + void run(Animal animal); -The SDK MUST provide the requested model type information available in its type system when selecting the transfer type and reconstructing the model value. This includes generic arguments when the SDK's conversion API preserves them. - -### Converter declaration and lookup - -A transfer converter declaration applies only to the exact model type on which it is declared. Declarations MUST NOT be inherited from a base type. - -A subclass or other derived type MAY declare its own converter independently of its base type. + // And this invocation + Animal value = new Dog(); + workflow.run(value); +``` -Runtime encoding MUST look only for a converter declared by the exact runtime type. Inbound decoding MUST look only for a converter declared by the exact requested type. +Here the transfer type converter associated with `Animal` should be used to +even if `Dog` also has a transfer type converter ensure decoding can match. +If the type hint does not have an associated transfer type converter, the +value is passed directly to the configured payload converter. -Every selected converter MUST provide a concrete, non-null transfer type for inbound payload conversion. The SDK MUST fail clearly if the converter does not provide a valid transfer type. It MUST NOT fall back to payload metadata inference after selecting a converter. +Converting from the transfer type should also use the type hint for the target execution. If the type hint does not have an associated transfer type converter, +the result of the configured payload converter should be used directly. -### Transfer value nullability +A transfer converter declaration applies only to the exact model type on which it is declared. Declarations are not inherited from a base type. -When a converter has been selected for an inbound model type, the SDK MUST invoke its reconstruction hook even when the decoded transfer value is null. +A subclass or other derived type can declare its own converter independently of its base type. -This permits a non-null model value to use null as its transfer representation. +### TransferType Nullability Converter -An outbound null model value MUST pass through without converter lookup because no runtime model type is available. +TransferTypeConverters should specify a non-null transfer type. ### Failure conversion -Failure semantics MUST remain owned by the configured failure converter. - -The failure converter MUST use the same transfer-aware payload-conversion path used for ordinary arguments and results when encoding or decoding typed failure payloads. This includes: - -- Application failure details. -- Cancellation details. -- Timeout last-heartbeat details. -- Reset workflow failure details. -- Details contained in nested failure causes. - -Payload codecs and external storage transformations MUST be applied exactly once to failure payloads and in their normal order. - -Transfer conversion does not apply to untyped failure metadata such as messages, stack traces, failure types, retry state, or encoded failure attributes. Existing payload-codec behavior for encoded failure attributes remains unchanged. - -### Raw payloads - -A raw payload wrapper MUST preserve its existing pass-through behavior. A transfer converter MUST NOT reinterpret a raw payload wrapper as a model value. - -### Configured converter authority - -The configured payload converter remains authoritative over the transfer representation's wire format. Transfer conversion MUST NOT add transfer-specific payload metadata or choose a wire encoding independently. - -The configured failure converter remains authoritative over the mapping between language exceptions and Temporal failure protos. - -### Serialization context - -Transfer conversion MUST preserve serialization context propagation to payload converters, failure converters, codecs, and external storage components. - -### Converter lifetime and concurrency - -Converter construction, lookup, and invocation MUST be safe under concurrent serialization. An SDK MAY reuse converter instances, so converter implementations MUST satisfy the concurrency requirements documented by that SDK. - -The number of converter instances and the time at which they are constructed are not part of the portable behavior. - -### Invalid declarations and callback failures - -An invalid converter declaration MUST fail clearly before conversion completes. Examples include an incompatible converter, a converter that cannot be constructed according to the SDK's API, or an invalid transfer type. - -Exceptions raised by transfer callbacks MUST remain observable as conversion failures. The SDK MUST NOT silently fall back to ordinary model serialization after selecting a converter. - -## Conformance requirements - -An implementation conforms to this specification when all of the following hold: - -- Ordinary arguments and results satisfy the normative conversion order. -- Failure details use transfer conversion without bypassing a configured failure converter. -- Payload codecs and external storage transformations are applied exactly once. -- A selected converter reconstructs from a null transfer value. -- Top-level-only and one-step behavior have explicit regression tests. -- Raw payload pass-through remains intact. -- Context propagation remains intact. -- Exact converter lookup is implemented and documented as non-inherited behavior. -- Workflow history containing transfer-represented values can be replayed. - -## Required conformance tests - -### Payload conversion - -- Annotated argument and result round trip. -- Mixed annotated and ordinary values preserve position and type. -- Missing payloads still produce language-default values. -- Available requested type information reaches converter selection and reconstruction. -- Null outbound model passes through. -- Non-null model represented by null reconstructs through its hook. -- Raw payload wrapper passes through unchanged. - -### Conversion boundaries - -- Annotated `A` converts to annotated `B` without invoking `B`'s converter. -- Annotated nested values do not invoke their converters. -- An unannotated subclass does not inherit an annotated base converter. -- A subclass can declare its own converter independently. - -### Failures - -- Application failure details round trip. -- Cancellation details round trip. -- Timeout last-heartbeat details round trip. -- Nested failure causes round trip their details. -- Custom failure converter remains authoritative. -- Payload codecs transform each failure payload exactly once. - -### Integration surfaces - -- Workflow client and worker. -- Workflow history replay. -- Activity stubs and standalone activity client. -- Local activities and activity heartbeats. -- Nexus client and worker. -- Schedule client create and describe operations. -- Test activity environment. +Failure conversion should use the transfer converter aware payload converter before +applying the configured payload converter to failure details.