Skip to content

feat: add Dynamic Workflow task execution policies - #170

Draft
Tsuyoshi Ushio (TsuyoshiUshio) wants to merge 35 commits into
mainfrom
tsuyoshiushio-workflow-task-execution-policy
Draft

feat: add Dynamic Workflow task execution policies#170
Tsuyoshi Ushio (TsuyoshiUshio) wants to merge 35 commits into
mainfrom
tsuyoshiushio-workflow-task-execution-policy

Conversation

@TsuyoshiUshio

@TsuyoshiUshio Tsuyoshi Ushio (TsuyoshiUshio) commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • extend the single Dynamic Workflows design record, FRD 0004, with finalized per-task execution-policy decisions
  • add bounded retry, timeout, deterministic backoff, continue-on-error, async workflow handlers, idempotency, authorization, structured status, and observability
  • route policy-aware plans through Durable-native retry while preserving compatibility for existing plans and histories
  • add a focused workflow-retry-policy sample and end-to-end coverage

Durable client lifecycle fix

Azure Durable Functions Python 2.x closes a durable_client_input client when the decorated function returns. The workflow-enabled SSE endpoint previously returned a lazy stream that retained that already-closing client, so a later tool call failed in get_status_all() with RuntimeError: The client is closed.

The SSE route now receives the host-provided durableClient binding configuration, creates one rich client for the response-stream lifetime, supplies it to all workflow management tools for that turn, and closes it on completion or failure. Non-streaming chat, MCP, declared-trigger, list, and status routes continue to use invocation-scoped injected clients.

Regression coverage includes first and repeated turns, a client closed after the prior turn, concurrent distinct streams, failure cleanup, generated binding metadata, and start/status/list/cancel/terminate routing through the current client.

Validation

  • Phase 2 architecture review findings are resolved and the task execution policy design is finalized
  • independent review of the Durable streaming ownership fix found no actionable issues
  • Ruff and strict MyPy pass
  • CI-equivalent suite: 1,238 passed, 57 deselected
  • local Functions host started with the preserved DTS azureManaged configuration
  • natural-language SSE chat invoked start_workflow; DTS executed two transient retries and reached Completed
  • a later same-session SSE turn invoked list_workflows successfully, proving a fresh live client is used across turns

This remains a draft while the complete implementation is reviewed. Follow-up to #163; planning issue: https://github.com/Azure/azure-functions-bucees-planning/issues/1278

Tsuyoshi Ushio added 21 commits August 19, 2026 21:58
Add the planning issue 1278 evolution contract to FRD 0004, covering bounded retries, timeouts, deterministic backoff, continued failures, idempotency, authorization, status, tests, and documentation. Record independent architecture-review resolutions while leaving the extension in review for human sign-off.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7fa54d6-516c-43ab-aa61-b1dda475e9e5
Record the human-approved precedence where decorator timeout and retry declarations override corresponding DAG fields, while omitted declarations allow DAG policy.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Record human approval for Decisions 67-79 and authorize implementation through an ExecPlan.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Resolve final implementation-contract ambiguities and add the living execution plan required by PLANS.md.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Add the PLANS.md-compliant living implementation plan under the path required by AGENTS.md.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Add bounded retry and timeout models, authoritative workflow-tool metadata, immutable start-time resolution, async handler discovery, classified public errors, and policy-aware Durable inputs while preserving policy-free payloads.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Expose immutable task context and stable idempotency keys, run sync and async handlers under Activity-owned deadlines, and return sanitized policy-aware outcomes without changing policy-free Activity behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Route policy-aware plans through the structured scheduler, use independent Durable backoff timers and stable attempt metadata, enforce both persisted and deployed authorization, and materialize bounded continued failures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Publish schema v3 executable-unit state, render retries and continued failures in the built-in UI, and emit bounded Activity-delivery spans and counters without orchestrator replay metrics.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Add a deterministic incident-triage policy plan covering retry, timeout, continued fan-out failure, conditioned recovery, revocation between attempts, duplicate delivery, and one-time condition evaluation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Tsuyoshi Ushio added 2 commits August 24, 2026 13:46
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Comment thread src/azure_functions_agents/workflows/engine.py Outdated
Move strict Activity-boundary validation, sanitized outcome validation, and retry disposition into an Azure-independent module while preserving the runtime-managed scheduler and all observable retry behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Comment thread samples/workflow-retry-policy/README.md
Comment thread src/azure_functions_agents/workflows/engine.py Outdated
Tsuyoshi Ushio added 3 commits August 26, 2026 14:18
Move policy-aware Activity execution into a dedicated module and make the retry sample use workflow-scoped, idempotent Blob-backed incident state.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Keep the customer sample focused on normal agent instructions and decorator retry configuration, while moving the canonical Skill/DAG precedence fixture into the official end-to-end suite.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
@TsuyoshiUshio

Tsuyoshi Ushio (TsuyoshiUshio) commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Updated through 9b208f3 to use Durable Python 2.x native Activity retry for new workflow histories while preserving the legacy Activity/timer driver for marker-free persisted histories. The migration includes strict sanitized exhaustion decoding, native status schema v4, attempt=None where the preview SDK has no attempt visibility, customer-sample/E2E separation, and preview extension/dependency setup.

The branch is synchronized with main and merge conflicts are resolved. Real Functions-host E2E covers retry-to-success and exhaustion with the original sanitized error_code/kind restored after the final TaskFailedError. Read-only skill operations are configured for automatic execution while skill scripts continue to require approval.

All current checks pass: public build and unit tests on Python 3.13/3.14, E2E on Python 3.13/3.14, documentation, SDL analysis, and CLA. The local canonical gate passed 1,235 tests with Ruff and strict mypy clean.

Tsuyoshi Ushio added 7 commits August 26, 2026 19:52
Resolve dependency updates by retaining Durable preview exact pins alongside the current Agent Framework versions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Install customer dependencies for every Function App host and allow the model-backed retry scenario to finish its required skill steps across bounded follow-up turns.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Configure repository-backed load_skill and read_skill_resource as no-approval operations while retaining approval for skill scripts. This lets the non-streaming harness complete its tool loop without emitting an unresumable pending approval.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 853bf15f-8c6c-4aa6-9fcc-7c97a749d5b5
Scope each workflow-enabled SSE request to a fresh client built from the host-supplied durableClient binding configuration, and close it when stream consumption ends.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 5b5b4036-0ba0-4453-b9aa-42833f45eb8c
@TsuyoshiUshio Tsuyoshi Ushio (TsuyoshiUshio) changed the title design: define Dynamic Workflow task execution policy feat: add Dynamic Workflow task execution policies Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant