Skip to content

Compact Flow-scoped Blob refs and make ASYNC input snapshots opt in - #480

Merged
longquanzheng merged 8 commits into
mainfrom
codex/contextual-blob-references
Sep 16, 2026
Merged

longquanzheng merged 8 commits into
mainfrom
codex/contextual-blob-references

Conversation

@longquanzheng

@longquanzheng longquanzheng commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace verbose Blob references with compact Flow-contextual references such as p1|260913/ab3de7kp2x
  • preserve the storage ID and six-digit UTC write date while deriving the owning Flow ID from trusted context
  • use deterministic lowercase Base36 object IDs and make their length server-configurable
  • store the complete EncodedObject in Object Blobs, removing the encoding suffix from references
  • enforce destination-Flow ownership by copying or inlining Blob-backed Values at every cross-Flow boundary
  • make successful ASYNC local Step input snapshots opt-in and disabled by default
  • distinguish unavailable Step method input snapshots from individual Value Blob load failures in Dex Web
  • decode the Value null arm as an ordinary application value while preserving Attribute-delete and Flow-output behavior at their specific boundaries

Blob reference and storage behavior

  • String and Object references share <storageId>|<YYMMDD>/<objectId>; their Value arms distinguish the types.
  • Physical objects use <namespace>/<YYMMDD>$<base64url(flowId)>/<objectId> with no version directory.
  • Object Blobs contain the deterministic protobuf encoding of the complete EncodedObject, including json, raw, or custom encoding metadata.
  • blobStore.thresholdInBytes defaults to 100: payloads through 100 bytes stay inline and payloads from 101 bytes are offloaded.
  • blobStore.objectIdLength defaults to 10. Zero selects the default, negative values are rejected, and any positive value is accepted. Lengths above 50 add leading zero padding after the full SHA-256 value is represented.
  • IDs use lowercase Base36, so Local BlobStore remains safe on case-insensitive filesystems.
  • retries deterministically overwrite the same key without conditional create, read-before-write, or collision fallback.
  • cross-Flow StartFlow, SubFlow, RPC, result, and side-effect boundaries hydrate source-owned references before inlining or rewriting them under the destination Flow.
  • LoadBlobs carries the owning Flow ID per entry, and SDK caches key entries by Flow ID plus Blob reference.

ASYNC Step input snapshots

blobStore.asyncStepInputSnapshotsEnabled defaults to false. When disabled, successful ASYNC local Step invocations do not write separate semantic-history input snapshots. This does not affect Step execution, retries, fallback, or Flow recovery.

When a snapshot is unavailable, semantic history returns input.unavailable=true. Dex Web explains that ASYNC snapshots are disabled by default, identifies blobStore.asyncStepInputSnapshotsEnabled, and distinguishes a missing whole-method snapshot from an unavailable individual Value Blob. Terminal ASYNC failures that exhaust their short retry policy before regular fallback retain their more specific explanation.

SYNC method inputs and ASYNC regular-fallback inputs continue to come from backend history.

Breaking changes

The project is pre-launch, so this directly replaces the previous wire and storage formats. There is no compatibility parser, dual read, migration shim, manifest, /v2/ prefix, encoding suffix, eight-digit date, or old reference support.

Public APIs continue to reject client-supplied internal Blob references.

Validation

  • complete GitHub Actions matrix, including Temporal and Cadence partitions
  • Server unit, Blob Store, Attribute Store, Web API, client, and Worker integration suites
  • all five SDK test and user-contract matrices
  • Dex Web typecheck, 90 component/library tests, and production build
  • generated-code freshness, format, Clippy, license, release, and coverage checks
  • English and Simplified Chinese documentation builds and smoke checks

@longquanzheng
longquanzheng force-pushed the codex/contextual-blob-references branch from 5a127cb to 6ac5c0a Compare September 16, 2026 00:16
@longquanzheng
longquanzheng marked this pull request as ready for review September 16, 2026 04:00
@longquanzheng longquanzheng changed the title Compact and scope Blob references Compact Flow-scoped Blob refs and make ASYNC input snapshots opt in Sep 16, 2026
@longquanzheng
longquanzheng merged commit bcb954c into main Sep 16, 2026
51 checks passed
@longquanzheng
longquanzheng deleted the codex/contextual-blob-references branch September 16, 2026 04:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant