Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ robots directives, and the sitemap.
Engineering design docs (not in the public sidebar):

* [Dex Design](design/Dex-Design.md)
* [Compact workflow-history wire names](design/compact-workflow-history-names.md)
* [IDL renames (OpenAPI → dex.proto)](design/idl-renames.md)
* [ContinueAsNew in Temporal (or Cadence)](design/ContinueAsNew-in-Temporal-(or-Cadence)-workflow.md)
* Plans under [`design/plan/`](design/plan/)
Expand Down
91 changes: 91 additions & 0 deletions docs/design/compact-workflow-history-names.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Compact workflow-history wire names

Status: accepted.

## Decision

Dex uses compact names for Server-internal values persisted in Temporal and
Cadence workflow history. The Go identifiers remain descriptive where practical,
but their history-facing wire values are deliberately short.

| History record | Previous persisted name | Current persisted name |
| --- | --- | --- |
| Request ID memo key | `__DexSystem_WorkflowRequestId` | `ReqId` |
| InvokeRPC Update | `InvokeRpc` | `IRPC` |
| InvokeRPC fallback Signal | `__DexSystem_ExecuteRpc` | `ERPC` |
| WaitFor Activity Type | SDK-derived name ending in `InvokeWaitForMethod` | `IWaitForM` |
| Execute Activity Type | SDK-derived name ending in `InvokeExecuteMethod` | `IExecuteM` |
| Worker RPC Activity Type | SDK-derived name ending in `InvokeWorkerRPC` | `IWRPC` |
| WaitFor Activity ID prefix | `__DexSystem_StepWaitFor_` | `StepWaitFor_` |
| Execute Activity ID prefix | `__DexSystem_StepExecute_` | `StepExecute_` |
| Skip-timer Signal | `__DexSystem_SkipTimerChannel` | `SkipTimer` |
| Stop-workflow Signal | `__DexSystem_StopWorkflowChannel` | `StopWorkflow` |
| Update-config Signal | `__DexSystem_UpdateWorkflowConfig` | `UpdateConfig` |
| Continue-as-new Signal | `__DexSystem_TriggerContinueAsNew` | `TriggerContinueAsNew` |
| SubFlow-completion Signal | `__DexSystem_SubFlowCompletion` | `SubFlowCompletion` |

`DexSystemConstPrefix` is removed. The Go memo constant changes from
`WorkflowRequestId` to `ReqId`. The Activity implementation methods change from
`InvokeWaitForMethod`, `InvokeExecuteMethod`, and `InvokeWorkerRPC` to
`IWaitForM`, `IExecuteM`, and `IWRPC`. WorkerService retains its original gRPC
method names.

Temporal and Cadence workers explicitly register the three compact Activity
Types. Scheduling, history decoding, reset lookup, and replay therefore use the
same exact values on both backends.

## Context

Temporal and Cadence persist Activity Types, Activity IDs, Update names, Signal
names, and memo keys in durable history. Step Activities occur frequently, so
their Type and ID strings are repeated throughout a Flow's lifetime. The request
memo is also copied to child Flows, while RPC and internal control operations add
more named records.

The former `__DexSystem_` prefix and complete Go method names added bytes without
adding useful information. History event kinds already distinguish Activities,
Updates, Signals, and memos. These values are private to the interpreter, so Dex
also controls their namespace and decoding.

Compact persisted values reduce workflow-history storage, transfer, and replay
input size. This optimization is intentionally limited to history-facing Server
internals. Public SDK names, FlowService and WorkerService gRPC methods, protobuf
messages, and Web API results keep their descriptive names.

## Compatibility

This is an intentional Server-side breaking change. Workflows and histories
written with the former names are not supported by the new Server. The change
does not add Temporal or Cadence version gates, register old Activity aliases,
accept old Signal or Update names, or increment `GlobalVersioner`.

Replay baselines are histories captured from the new implementation. A rollout
must not expect existing workflows recorded with the former names to continue.

## Server protocol version

The Server protocol version is not incremented. That version negotiates
compatibility between Server, Workers, SDKs, and dexcli. This decision does not
change either gRPC service, any protobuf payload, or application-visible
behavior, so existing clients and Workers remain compatible.

Increasing the minimum protocol version would reject compatible SDK releases.
Increasing only the current version would add no negotiated behavior and would
not detect or repair an old workflow history. The breaking rollout requirement
belongs in Server release notes instead.

## Consequences

- Workflow histories become smaller, especially for Flows with many Steps.
- Temporal and Cadence persist the same exact Activity Type names.
- Server history conversion, reset, and replay recognize only the compact names.
- Web clients continue to receive the same `FlowHistoryEvent` representation.
- Future persisted internal names should remain compact; public APIs and ordinary
code identifiers should remain descriptive.

## Verification

Integration tests inspect real Temporal and Cadence histories for the compact
memo key, Activity Types, Activity ID prefixes, Update name, and Signal names.
Replay tests use newly captured Temporal histories, and Web API integration tests
verify that the decoded history result is unchanged on both backends.
2 changes: 1 addition & 1 deletion docs/design/plan/design-web.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ message GetFlowSummaryResponse {
}
```

`request_id` 从 `__DexSystem_WorkflowRequestId` memo 转换得到。`flow_type` 从 Dex `FlowType` search attribute 转换得到。原始 memo 不返回。
`request_id` 从 `ReqId` memo 转换得到。`flow_type` 从 Dex `FlowType` search attribute 转换得到。原始 memo 不返回。

`run_id` 为空时,server 让 Temporal/Cadence 解析 current/latest run,并在 response 中返回实际 run ID。Web 随后替换为包含 run ID 的 canonical URL。

Expand Down
2 changes: 1 addition & 1 deletion docs/design/plan/go-sdk-rewrite.md
Original file line number Diff line number Diff line change
Expand Up @@ -1226,7 +1226,7 @@ long-poll reattachments. When a previous Update using that ID completed with a
handler timeout, the server appends an increasing `-N` suffix until it can start
a new Update.

InvokeRPC uses the ID for external-value ownership and as the `InvokeRpc`
InvokeRPC uses the ID for external-value ownership and as the `IRPC`
Temporal Update ID. Durable waits use their server-selected generation ID. The
ID is not exposed in results.

Expand Down
2 changes: 1 addition & 1 deletion docs/design/plan/server-rewrite-plan-phase4.md
Original file line number Diff line number Diff line change
Expand Up @@ -461,7 +461,7 @@ compile only after S5 migrates updater/query/CAN files.

### InvokeRPC update

- Full attributes; no LoadingPolicy. Temporal uses one synchronous `InvokeRpc`
- Full attributes; no LoadingPolicy. Temporal uses one synchronous `IRPC`
Update for non-empty `lock_attribute_keys` or when all RPCs opt into Updates.
- The API owns a WorkerService connection pool built from
`InterpreterActivityConfig`. The non-locking path queries `PrepareRpc`, invokes
Expand Down
4 changes: 2 additions & 2 deletions docs/design/plan/server-rewrite-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ flowchart LR
- **Attribute locking:** retain WaitFor/Execute lock keys on `StepOptions`. For
RPC, add a minimal lock surface — `repeated string
lock_attribute_keys` on `InvokeRPCRequest`. Temporal uses one synchronous
`InvokeRpc` Update for non-empty lists and may opt all RPCs into Updates. Keeps
`IRPC` Update for non-empty lists and may opt all RPCs into Updates. Keeps
the retained
`RPC_ACQUIRE_LOCK_FAILURE` + `skip_writes_reapply` meaningful. Because the
locking path uses `SynchronousUpdateWorkflow`, a **non-empty `lock_attribute_keys`
Expand Down Expand Up @@ -442,7 +442,7 @@ short-circuit these two RPCs — and any `InvokeRPC` with non-empty
`lock_attribute_keys` — to `Unimplemented` before dialing.

**RPC locking (`InvokeRPC`):** driven by the new `InvokeRPCRequest.lock_attribute_keys`
(Phase 0). Temporal uses the synchronous `InvokeRpc` Update for locking RPCs or
(Phase 0). Temporal uses the synchronous `IRPC` Update for locking RPCs or
when `api.useTemporalSynchronousUpdateForAllRPCs` is enabled. The validator accepts
an empty list and returns `RPC_ACQUIRE_LOCK_FAILURE` / `Aborted` when a requested
key is locked. The handler calls `LoadAttributes`; an empty list reads all attributes
Expand Down
2 changes: 1 addition & 1 deletion server/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Interpreter workflows and activities use constructor injection. Do not add mutab
package-global environments or registries.

`WaitForStepCompletion` and `WaitForAttribute` are Temporal-only synchronous
updates. Locking Temporal InvokeRPC uses an `InvokeRpc` synchronous Update.
updates. Locking Temporal InvokeRPC uses an `IRPC` synchronous Update.
Non-locking InvokeRPC uses query, WorkerService, and optional signal unless
`api.useTemporalSynchronousUpdateForAllRPCs` opts Temporal into Updates.

Expand Down
5 changes: 2 additions & 3 deletions server/integ/async_retry_budget_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ package integ

import (
"context"
"strings"
"sync"
"testing"
"time"
Expand Down Expand Up @@ -650,7 +649,7 @@ func temporalRegularFallbackRetryPolicy(
continue
}
attributes := historyEvent.GetActivityTaskScheduledEventAttributes()
if !strings.Contains(attributes.GetActivityType().GetName(), "InvokeWaitForMethod") {
if attributes.GetActivityType().GetName() != "IWaitForM" {
continue
}
var input dexpb.InvokeWaitForMethodActivityInput
Expand Down Expand Up @@ -689,7 +688,7 @@ func cadenceRegularFallbackRetryPolicy(
continue
}
attributes := historyEvent.GetActivityTaskScheduledEventAttributes()
if !strings.Contains(attributes.GetActivityType().GetName(), "InvokeWaitForMethod") {
if attributes.GetActivityType().GetName() != "IWaitForM" {
continue
}
var input dexpb.InvokeWaitForMethodActivityInput
Expand Down
17 changes: 8 additions & 9 deletions server/integ/from_step_execution_id_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ package integ
import (
"context"
"fmt"
"strings"
"testing"
"time"

Expand Down Expand Up @@ -327,11 +326,11 @@ func recordTemporalScheduledLineage(
attributes := event.GetActivityTaskScheduledEventAttributes()
activityType := attributes.GetActivityType().GetName()
switch {
case strings.Contains(activityType, "InvokeWaitForMethod"):
case activityType == "IWaitForM":
var input dexpb.InvokeWaitForMethodActivityInput
require.NoError(t, dataConverter.FromPayloads(attributes.GetInput(), &input))
recordStepContext(t, lineage, input.GetRequest().GetContext())
case strings.Contains(activityType, "InvokeExecuteMethod"):
case activityType == "IExecuteM":
var input dexpb.InvokeExecuteMethodActivityInput
require.NoError(t, dataConverter.FromPayloads(attributes.GetInput(), &input))
recordStepContext(t, lineage, input.GetRequest().GetContext())
Expand Down Expand Up @@ -360,11 +359,11 @@ func recordTemporalLocalLineage(
var marker temporalLocalActivityMarkerData
require.NoError(t, dataConverter.FromPayloads(markerPayload, &marker))
switch {
case strings.Contains(marker.ActivityType, "InvokeWaitForMethod"):
case marker.ActivityType == "IWaitForM":
var output dexpb.InvokeWaitForMethodActivityOutput
require.NoError(t, dataConverter.FromPayloads(resultPayload, &output))
recordLocalActivityMetadata(t, lineage, output.GetResponse().GetLocalActivityMetadata())
case strings.Contains(marker.ActivityType, "InvokeExecuteMethod"):
case marker.ActivityType == "IExecuteM":
var output dexpb.InvokeExecuteMethodActivityOutput
require.NoError(t, dataConverter.FromPayloads(resultPayload, &output))
recordLocalActivityMetadata(t, lineage, output.GetResponse().GetLocalActivityMetadata())
Expand Down Expand Up @@ -418,12 +417,12 @@ func recordCadenceScheduledLineage(
attributes := event.GetActivityTaskScheduledEventAttributes()
activityType := attributes.GetActivityType().GetName()
switch {
case strings.Contains(activityType, "InvokeWaitForMethod"):
case activityType == "IWaitForM":
var input dexpb.InvokeWaitForMethodActivityInput
var localInput *dexpb.InternalLocalActivityInput
require.NoError(t, dataConverter.FromData(attributes.GetInput(), &input, &localInput))
recordStepContext(t, lineage, input.GetRequest().GetContext())
case strings.Contains(activityType, "InvokeExecuteMethod"):
case activityType == "IExecuteM":
var input dexpb.InvokeExecuteMethodActivityInput
var localInput *dexpb.InternalLocalActivityInput
require.NoError(t, dataConverter.FromData(attributes.GetInput(), &input, &localInput))
Expand All @@ -450,7 +449,7 @@ func recordCadenceLocalLineage(
return
}
switch {
case strings.Contains(marker.ActivityType, "InvokeWaitForMethod"):
case marker.ActivityType == "IWaitForM":
var output dexpb.InvokeWaitForMethodActivityOutput
require.NoError(
t,
Expand All @@ -460,7 +459,7 @@ func recordCadenceLocalLineage(
[]byte(marker.ResultJSON),
)
recordLocalActivityMetadata(t, lineage, output.GetResponse().GetLocalActivityMetadata())
case strings.Contains(marker.ActivityType, "InvokeExecuteMethod"):
case marker.ActivityType == "IExecuteM":
var output dexpb.InvokeExecuteMethodActivityOutput
require.NoError(
t,
Expand Down
6 changes: 3 additions & 3 deletions server/integ/rpc_external_storage_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ func assertTemporalRpcBlobHistory(
string(rpcStorage.TestInput.GetObjValue().GetPayload()),
)
}
require.Equal(t, service.InvokeRpcUpdateType, accepted.GetInput().GetName())
require.Equal(t, "IRPC", accepted.GetInput().GetName())
var request dexpb.InvokeRPCRequest
require.NoError(t, dataConverter.FromPayloads(accepted.GetInput().GetArgs(), &request))
acceptedInput = request.GetInput()
Expand All @@ -363,7 +363,7 @@ func assertTemporalRpcBlobHistory(
completedCount++
case temporalenums.EVENT_TYPE_WORKFLOW_EXECUTION_SIGNALED:
signalName := event.GetWorkflowExecutionSignaledEventAttributes().GetSignalName()
require.NotEqual(t, service.ExecuteRpcSignalChannelName, signalName)
require.NotEqual(t, "ERPC", signalName)
}
}
require.Equal(t, 1, acceptedCount)
Expand Down Expand Up @@ -466,7 +466,7 @@ func assertTemporalRpcSignalHistory(
require.NotEqual(t, requestID, accepted.GetMeta().GetUpdateId())
case temporalenums.EVENT_TYPE_WORKFLOW_EXECUTION_SIGNALED:
attributes := event.GetWorkflowExecutionSignaledEventAttributes()
if attributes.GetSignalName() != service.ExecuteRpcSignalChannelName {
if attributes.GetSignalName() != "ERPC" {
continue
}
var request dexpb.ExecuteRpcSignalRequest
Expand Down
4 changes: 2 additions & 2 deletions server/integ/subflow_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ func doTestSubFlowCondition(
)
require.NoError(t, err)
require.Len(t, parentDescription.Memos, 1)
require.Equal(t, parentRequestID, string(parentDescription.Memos[service.WorkflowRequestId].GetObjValue().GetPayload()))
require.Equal(t, parentRequestID, string(parentDescription.Memos["ReqId"].GetObjValue().GetPayload()))

firstResult, err := flowClient.WaitForFlow(ctx, &dexpb.WaitForFlowRequest{
FlowId: parentFlowID,
Expand Down Expand Up @@ -199,7 +199,7 @@ func doTestSubFlowCondition(
require.Equal(
t,
startResponse.GetRunId()+subFlowParentStep+"-1",
string(childDescription.Memos[service.WorkflowRequestId].GetObjValue().GetPayload()),
string(childDescription.Memos["ReqId"].GetObjValue().GetPayload()),
)
objectCount, err := globalBlobStore.CountWorkflowObjectsForTesting(ctx, childFlowID)
require.NoError(t, err)
Expand Down
2 changes: 2 additions & 0 deletions server/replayTests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ Dex uses the [global versioning design pattern](https://medium.com/@qlong/how-to

After the gRPC interpreter rewrite, the global-version scheme **restarted at v1**.
Pre-rewrite histories were deleted; do not keep baselines for old global versions.
The current baseline uses the shortened internal Activity, Update, Signal, memo,
and Step Activity ID names. Histories recorded with the former names are unsupported.

* For every new global version, add at least one new history under [`history/`](./history).
* Each version may need multiple histories for different code paths.
Expand Down
Loading
Loading