Skip to content

.NET: Harden durable mailbox delivery and workflow output provenance - #94

Draft
Tamir Dresher (tamirdresher) wants to merge 3 commits into
tamirdresher-microsoft-stack-1-dotnet-schema-2from
tamirdresher-microsoft-harden-durable-delivery-ttl
Draft

.NET: Harden durable mailbox delivery and workflow output provenance#94
Tamir Dresher (tamirdresher) wants to merge 3 commits into
tamirdresher-microsoft-stack-1-dotnet-schema-2from
tamirdresher-microsoft-harden-durable-delivery-ttl

Conversation

@tamirdresher

@tamirdresher Tamir Dresher (tamirdresher) commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Stacked draft and release gates

Layer 2, dependent on schema contract #92 and .NET schema implementation #93.

  • Base: tamirdresher-microsoft-stack-1-dotnet-schema-2 at 6b5410e0afa6adf5037ee2a9b42340440bf69196
  • Head: tamirdresher-microsoft-harden-durable-delivery-ttl at da6a9c6752b34b9839f95a26e8c54d8ea0a8238c
  • Layer-2 commits: 30f821d, ef960455, da6a9c6
  • Shared contract reviewed against Propose durable agent state schema 2.0 contract #92 head eff12f41d3923d0123310a178b9dd6a0df61fb56

Keep this PR draft. Both merge and release remain blocked until the required isolated
Durable Task backend test passes and shared contract/rollout/rollback agreement is complete.
Producer activation and receipt-deleting entity TTL remain separate internal, default-disabled
gates. There is no public production schema-2 activation API and no shared schemas/ change.
Passing local tests or obtaining draft review acceptance does not prove backend atomicity.

Mailbox delivery, compatibility and expiry

  • Entity execution and polling share dual-layout outcome resolution. Legacy reads retained transcript;
    schema 2 reads immutable terminal results and permanent completion receipts authoritatively.
    Pruning or payload expiry never makes a completed correlation new or permits transcript fallback.
  • Successful outer invocation stages results/receipts with all entity-local working state. External
    provider/tool effects remain outside. Validation, cancellation, provider/model, serialization,
    capacity and failed-commit errors remain retryable unless an explicit durable terminal contract commits them.
  • Legacy conversion is evidence-only, idempotent and model/tool-free. Existing legacy state requires
    independent complete-history authorization; truncation/compaction blocks promotion. Evicted
    completions are never fabricated. Promotion preserves declared and unknown response metadata.
  • Canonical metadata and value survive GetDurableResult() and durable transport, preserving
    absence, null, false, zero, empty and object values. Committed failures remain failures through the
    actual SDK exception transport, never success-shaped responses.
  • Historical 1.0/1.1/1.2 boundaries stay strict. Developer roles, string function arguments and URI
    content without media type remain v2-only. Schema-aware cold replay/delivery keeps valid media-less
    URI content opaque. Optional historyBinding, unknown-content markers and terminal diagnostics remain intact.
  • Result retention is configurable and separate from entity TTL; there is no implicit 60-second
    retention. Cleanup durably removes payloads while retaining receipts/outcomes, without invoking
    agents or refreshing TTL. Polling reports logical expiry without mutating state.
  • An optional namespaced root extensionData profile persists one logical cleanup deadline,
    entity identity and generation token. Runs reuse pending checks; replacements invalidate old
    signals. Duplicate/stale checks, including legacy/deleted/recreated/null cases, perform no state
    setter or outgoing signal. Recovery of idle imports requires explicit cleanup or a later new run.
    Scheduling backoff is not retention. Compatible writers must honor the profile, not merely preserve JSON.

Workflow trust boundary and typed child results

The IcM trust-boundary fix uses structural provenance: the dispatcher and runner exchange typed
framework output. Agent/model and request-port text remains exact result-only text even when it
matches a control envelope. Only trusted regular activities and typed subworkflows provide controls.
Legacy plain activity text remains supported; invalid/duplicate known fields reject an entire
trusted envelope to text with no partial controls. Routed messages require valid nonblank typeName
and data; serialized null/false/zero/empty-string payloads remain valid.

Final child workflow results now carry an explicit CLR string type hint while retaining exact text.
Typed successors do not accidentally choose an earlier JSON-compatible handler. Empty-result,
halt and superstep behavior remains unchanged. No HMAC or new wire discriminator is introduced.
Entity fixture interoperability is separate; no C#/Python workflow-output wire parity is claimed.

Consumer behavior

HTTP preserves text negotiation and native response, adding canonical result for full metadata.
Statuses are 202 pending, 200 available success, 500 supported committed failure, and 410
completed-but-unavailable with retained outcome in completion_outcome and
x-ms-agent-completion-outcome. MCP keeps successful text by default; opt-in responseFormat=json
returns structured metadata/value. Invalid formats fail, cancellation propagates, and terminal
failures/unavailable results remain exceptions. Entity deletion ends duplicate protection.

Provider/history ownership, replay filtering, transcript compaction and later layers are out of scope.
Rollout must cover workers, readers, pollers, hosting consumers, rollback writers and dashboard tooling.

Validation and independent review

Fresh independent OpenAI gpt-5.6-sol review ACCEPTED, for draft readiness only, exact tree
4cda596a84543fc1e31aff9653a871337baf540a; commit da6a9c6752b34b9839f95a26e8c54d8ea0a8238c
was created only afterward. New corrections: six files, +433/-3; full layer: 56 files, +8,629/-297.

  • 1073 DurableTask + 223 Hosting + 11 local orchestration tests passed.
  • Fifty new production-path metadata/routing cases passed after reproducing 45 genuine RED failures.
    They cover authorized promotion, repeated cold duplicates/polling, typed child/parent routing,
    actual dispatcher/activity execution and recorded semantic replay after serialized reconstruction.
  • All 96 shared schema cases, four fixtures with date-time/local-reference checks, and the
    44-case runtime-envelope test passed; unchanged base schema-only legacy exceptions remain.
  • Full Release solution build/package, all 49 changed-C# format/style/analyzer checks and diff
    checks passed. The reviewer independently reran 1073 + 223 + 11 + 50 tests and verified 49 source hashes.
  • Post-commit affected builds, complete solution packaging and 1073 + 223 + 11 tests passed again.
    All four final package archives and their DLL/PDB data reference the accepted SHA; packaged DLLs
    match the tested binaries byte-for-byte.

Real backend: NOT RUN, one explicit gated skip, zero passed. Public-registration integration
coverage exists for entity state/outbox commit, rollback, delayed cleanup, duplicate delivery and
worker restart, but no safe isolated endpoint was configured. Local semantic workflow replay is
not actual worker-restart proof. Both merge and release remain blocked pending the required pass.

Earlier review rejections were corrected by different exclusive owners before fresh acceptance;
the scheduler/SDK-failure protections remain covered. No rejected tree was committed.
The only permitted baseline exception is command-scoped WarningsNotAsErrors=NU1902 for transitive
Microsoft.Build.Tasks.Git 8.0.0 (GHSA-23fw-v26w-5fgq).
Audit and all other warnings remain enforced; no dependency or repository suppression changes.
Existing assets required no restore; any required restore uses Microsoft CFS.

@tamirdresher
Tamir Dresher (tamirdresher) added this pull request to stack #96 September 11, 2026 19:24
@tamirdresher
Tamir Dresher (tamirdresher) force-pushed the tamirdresher-microsoft-harden-durable-delivery-ttl branch from 2d5aa78 to 6f515fd Compare September 12, 2026 01:49
Copilot AI lite review requested due to automatic review settings September 12, 2026 01:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

One or more issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review tier: Lite
Findings: 2 High severity · 1 Medium severity

Open findings (3)
What changed in this PR

This PR hardens durable agent delivery, preserves terminal outcome metadata, and enforces workflow-output provenance across entity, orchestration, polling, HTTP, and MCP paths.

Changes:

  • Adds authoritative mailbox outcome resolution, duplicate handling, failure transport, and retention controls.
  • Preserves canonical response metadata through durable serialization and hosting endpoints.
  • Restricts workflow controls to trusted activity/subworkflow outputs and expands regression coverage.
File Description
dotnet/​tests/​Microsoft.Agents.AI.Hosting.AzureFunctions.UnitTests/​DurableAgentFunctionMetadataTransformerTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.Hosting.AzureFunctions.UnitTests/​BuiltInFunctionsSessionIdAliasTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.Hosting.AzureFunctions.UnitTests/​BuiltInFunctionsAgentOutcomeTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​Workflows/​WorkflowExecutionTestHelper.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​Workflows/​DurableWorkflowRunnerTrustBoundaryTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​Workflows/​DurableExecutorDispatcherTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​State/​DurableAgentStateMailboxTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​DurableAIAgentProxyTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​DurableAgentStateOutcomeResolverTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​DurableAgentResponseSerializationTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​DurableAgentFailureDeliveryTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​AgentRunHandleTests.cs Updated as part of this pull request.
dotnet/​tests/​Microsoft.Agents.AI.DurableTask.UnitTests/​AgentEntityTimeToLiveTests.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.Hosting.AzureFunctions/​README.md Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.Hosting.AzureFunctions/​DurableAgentFunctionMetadataTransformer.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.Hosting.AzureFunctions/​CHANGELOG.md Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.Hosting.AzureFunctions/​BuiltInFunctions.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​Workflows/​DurableWorkflowRunner.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​Workflows/​DurableExecutorOutput.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​Workflows/​DurableExecutorDispatcher.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​State/​README.md Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​State/​DurableAgentStateTerminalResponse.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​State/​DurableAgentStateResponse.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​State/​DurableAgentStateRequest.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​State/​DurableAgentStateOutcomeResolver.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​State/​DurableAgentStateJsonConverter.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​State/​DurableAgentState.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​RunRequest.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​README.md Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​Microsoft.Agents.AI.DurableTask.csproj Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​Logs.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableDataConverter.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAIAgent.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentTerminalException.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentStateCorruptionException.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentsOptions.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentRunOutcome.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentResultUnavailableException.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentResponseExtensions.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentJsonUtilities.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentFailureMetadataException.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DurableAgentFailure.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​DefaultDurableAgentClient.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​CHANGELOG.md Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​AgentRunHandle.cs Updated as part of this pull request.
dotnet/​src/​Microsoft.Agents.AI.DurableTask/​AgentEntity.cs Updated as part of this pull request.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread dotnet/src/Microsoft.Agents.AI.DurableTask/AgentEntity.cs
Comment thread dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableExecutorOutput.cs Outdated
Copilot AI review requested due to automatic review settings September 12, 2026 05:40
@tamirdresher
Tamir Dresher (tamirdresher) force-pushed the tamirdresher-microsoft-harden-durable-delivery-ttl branch from 6f515fd to 30f821d Compare September 12, 2026 05:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Three unresolved moderate findings remain involving expiry cleanup complexity, missing cleanup-signal metadata, and legacy response metadata preservation.

Get a fresh assessment by requesting another Copilot review.

Review tier: Lite
Findings: 2 Medium severity

Open findings (2)
Resolved findings (2)

Copilot AI review requested due to automatic review settings September 12, 2026 08:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

One or more issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review tier: Lite
Findings: 1 High severity · 1 Medium severity

Open findings (2)
Resolved findings (3)

Copilot AI review requested due to automatic review settings September 12, 2026 10:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

A critical typed-routing validation gap remains, and real-backend atomicity is still unvalidated.

Review tier: Lite
Findings: 1 High severity

Open findings (1)
Resolved findings (2)

{
Result = workflowResult.Result,
Events = workflowResult.Events ?? [],
SentMessages = workflowResult.SentMessages ?? [],
Tamir Dresher and others added 3 commits September 12, 2026 15:17
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Persist one logical expiry schedule with entity identity and generation token. Make stale checks side-effect-free across revised, legacy, missing and recreated state. Add gated real-backend atomicity coverage and locally executable SDK entity-failure handling regressions.

Independent gpt-5.6-sol review accepted tree 0652be2 for draft readiness only. Actual isolated-backend atomicity validation and shared rollout agreement remain merge/release gates.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain declared and unknown legacy response metadata in detached mailbox snapshots. Route final child workflow output with an explicit string type hint. Add 50 production-path metadata, duplicate, typed routing and cold replay regressions.

Independent gpt-5.6-sol review accepted exact tree 4cda596 for draft readiness only. Actual backend restart and atomicity remain unproven; merge and release remain gated on isolated-backend validation and shared rollout agreement.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@tamirdresher
Tamir Dresher (tamirdresher) force-pushed the tamirdresher-microsoft-harden-durable-delivery-ttl branch from da6a9c6 to 352a8fa Compare September 12, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants