Skip to content

Python: [BREAKING] align workflow span name with OpenTelemetry conventions - #9900

Open
Vincent Biret (baywet) wants to merge 6 commits into
mainfrom
fix/workflow-span-name-python
Open

Vincent Biret (baywet) wants to merge 6 commits into
mainfrom
fix/workflow-span-name-python

Conversation

@baywet

@baywet Vincent Biret (baywet) commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Motivation & Context

Align Python workflow invocation telemetry with the pinned OpenTelemetry GenAI invoke-workflow conventions. Graph and functional workflows previously emitted workflow.run, used the custom workflow.name attribute instead of gen_ai.workflow.name, and omitted the required gen_ai.operation.name attribute.

Description & Review Guide

  • What are the major changes? Emit invoke_workflow {gen_ai.workflow.name} when a meaningful workflow name is available, falling back to invoke_workflow otherwise. Rename workflow.name to gen_ai.workflow.name and supply gen_ai.operation.name="invoke_workflow" at span creation. Track generated graph-workflow names separately so they remain available for runtime/checkpoint identity but are omitted from the semantic attribute and invocation span suffix. Explicit names, including names resembling generated names, and functional workflows' function-name defaults remain intact. Prepare names and attributes together in the shared span factories so the workflow-name suffix does not bypass conversation-ID enrichment. Extend graph/functional, streaming/non-streaming, checkpoint, scoped conversation, and AG-UI regression coverage, and document the telemetry format in the Python consumer skill.
  • What is the impact of these changes? This is an intentional breaking telemetry change, classified with the breaking change label and title prefix. Update processors, dashboards, alerts, and filters as described below. Workflow execution, span parenting/lifecycle, sensitive-data controls, and public function signatures are unchanged. Existing ambient conversation IDs and explicit conversation-ID precedence are preserved. Missing conversation IDs are not synthesized.
  • What do you want reviewers to focus on? Literal exported span names and attributes; omission of generated graph names without changing runtime/checkpoint identity; named and missing/empty-name behavior through both attached and unattached span factories; preservation of gen_ai.conversation.id with named and unnamed workflows; and unchanged executor/message spans, error reporting, and abandoned-stream cleanup.
Existing telemetry consumer Migration
Exact span-name match on workflow.run Prefer gen_ai.operation.name == "invoke_workflow" to identify invocations independently of workflow names. Display names are now invoke_workflow {workflow_name}, or bare invoke_workflow when no non-empty name attribute is supplied.
Attribute lookup on workflow.name Use gen_ai.workflow.name. OtelAttr.WORKFLOW_NAME now resolves to this standard key. Unnamed graph workflows omit this attribute instead of emitting a generated UUID-based runtime name.
OtelAttr.WORKFLOW_RUN_SPAN Its value changes from workflow.run to the base operation name invoke_workflow; exported named spans also include the workflow-name suffix. Do not compare every exported span name directly to this constant.
Conversation/error attributes Continue using gen_ai.conversation.id and error.type; their names are unchanged.

This PR is limited to Python workflow invocation spans and their direct attributes. Framework-only spans such as workflow.build, executor-processing spans, message-send spans, and workflow events remain unchanged; the pinned invoke-workflow section does not define replacements for them. Default graph-workflow names are still generated and retained in runtime/checkpoint identity and serialized workflow definitions, but are no longer included in gen_ai.workflow.name or the invocation span name.

The following commands passed in a Python 3.14 Docker container against the current source, from python/, using the isolated interpreter at /validation/venv/bin/python:

/validation/venv/bin/python -m pytest packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/core/test_observability.py::test_workflow_span_uses_scoped_conversation_id packages/core/tests/workflow/test_workflow.py::test_workflow_abandoned_stream_finalizes_without_context_errors packages/core/tests/workflow/test_functional_workflow.py::TestStreaming::test_nested_processing_span_parents_under_workflow_run -p no:cacheprovider
# Passed: 35 tests; one expected functional-workflow ExperimentalWarning.

# From python/packages/ag-ui/:
/validation/venv/bin/python -m pytest tests/ag_ui/test_workflow_run.py::test_workflow_and_agent_spans_use_supplied_agui_thread_id -p no:cacheprovider
# Passed: 2 tests, named and unnamed workflows.

# Remaining commands from python/:
/validation/venv/bin/ruff check --no-cache packages/core/agent_framework/_workflows/_workflow_builder.py packages/core/agent_framework/_workflows/_workflow.py packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/workflow/test_workflow.py packages/ag-ui/tests/ag_ui/test_workflow_run.py
# Passed.

/validation/venv/bin/ruff format --check --no-cache packages/core/agent_framework/_workflows/_workflow_builder.py packages/core/agent_framework/_workflows/_workflow.py packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/workflow/test_workflow.py packages/ag-ui/tests/ag_ui/test_workflow_run.py
# Passed: all five changed Python files already formatted.

/validation/venv/bin/pyright --pythonpath /validation/venv/bin/python --project packages/core/pyproject.toml packages/core/agent_framework/_workflows/_workflow_builder.py packages/core/agent_framework/_workflows/_workflow.py
# Passed: strict source check, 0 errors, 0 warnings.

/validation/venv/bin/pyright --pythonpath /validation/venv/bin/python --project pyrightconfig.tests.json packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/workflow/test_workflow.py packages/ag-ui/tests/ag_ui/test_workflow_run.py
# Passed: 0 errors, 0 warnings.

/validation/venv/bin/mypy --config-file pyproject.toml --cache-dir /tmp/mypy-cache --explicit-package-bases --namespace-packages --python-executable /validation/venv/bin/python packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/workflow/test_workflow.py packages/ag-ui/tests/ag_ui/test_workflow_run.py
# Passed: no issues in three changed test files.

# Pyrefly, ty, and Zuban use VIRTUAL_ENV=/validation/venv and that venv's bin directory on PATH.
/validation/venv/bin/pyrefly check -c pyrefly.toml packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/workflow/test_workflow.py packages/ag-ui/tests/ag_ui/test_workflow_run.py
# Passed: 0 errors.

/validation/venv/bin/ty check packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/workflow/test_workflow.py packages/ag-ui/tests/ag_ui/test_workflow_run.py
# Passed.

/validation/venv/bin/zuban mypy --config-file pyproject.toml packages/core/tests/workflow/test_workflow_observability.py packages/core/tests/workflow/test_workflow.py packages/ag-ui/tests/ag_ui/test_workflow_run.py
# Passed: no issues in three changed test files.

The named-span and operation-attribute regressions failed before their respective fixes. Six unnamed-graph regressions also failed before generated names were excluded. Full package and integration suites were not run; all five configured test type checkers were run on the three test files changed for this review fix.

Related Issue

N/A - trivial change: correct unambiguous workflow invocation span-name and required-attribute mismatches with the linked specification. This PR changes only Python; #9899 contains the related .NET correction. #8559 concerns middleware telemetry and is not closed by this PR.

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: GitHub Copilot assisted with implementation, regression tests, consumer documentation, validation, and this PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fa592ce6-05c6-4421-801b-59c77df4b2df
Copilot AI balanced review requested due to automatic review settings October 30, 2026 19:03
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Oct 30, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are identified for the workflow invocation span-name update.

No open issues 🎉

What changed in this PR

Aligns Python workflow invocation span names with OpenTelemetry conventions, complementing the .NET correction in #9899. Workflow execution and public API signatures remain unchanged; telemetry filters using workflow.run must switch to invoke_workflow.

Changes:

  • Renames the shared workflow invocation span to invoke_workflow.
  • Adds four regression cases covering graph and functional workflows in streaming and non-streaming modes.
  • Updates existing core and AG-UI telemetry assertions and a test docstring.
File Description
python/​packages/​core/​tests/​workflow/​test_workflow.py Updates the abandoned-stream span assertion.
python/​packages/​core/​tests/​workflow/​test_workflow_observability.py Adds span-name regression coverage and updates telemetry expectations.
python/​packages/​core/​tests/​workflow/​test_functional_workflow.py Updates the span name in a test docstring only.
python/​packages/​core/​agent_framework/​observability.py Changes the shared workflow invocation span-name constant.
python/​packages/​ag-ui/​tests/​ag_ui/​test_workflow_run.py Updates workflow span filters in conversation-ID tests.

Review effort: Balanced

@github-code-quality

github-code-quality Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: Python

Python / code-coverage/python

main 6fdeb2d fix/workflow-span-na... 38792a7 +/-
92% 92% 0%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated November 02, 2026 18:16 UTC

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): db7ed22ba370
Model: gpt-5.6-sol

Overview

The PR centralizes the workflow invocation span rename in the shared telemetry constant and adds graph/functional and streaming/non-streaming regression coverage, while existing tests preserve span lifecycle, parenting, error, cleanup, and conversation-ID behavior. The resulting bare span name does not implement the workflow-name suffix required for named workflows by the referenced convention. The change also breaks existing telemetry consumers that match the previously emitted name without the explicit breaking-change classification required for the stable core package.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/core/agent_framework/observability.py

@baywet Vincent Biret (baywet) added the breaking change Usage: [PRs], Target: all PRs that introduce changes that are not backward compatible label Oct 30, 2026
@github-actions github-actions Bot changed the title Python: align workflow span name with OpenTelemetry conventions Python: [BREAKING] align workflow span name with OpenTelemetry conventions Oct 30, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fa592ce6-05c6-4421-801b-59c77df4b2df

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 2

Result: No findings
Scope: 1 net-new commit(s): c4c2dd58b83b
Model: gpt-5.6-sol

Overview

This incremental update adds an explicit union annotation for the graph and functional workflow instances used by the telemetry regression test. Both builders' declared return types, public exports, compatible run overloads, and the four-case streaming/non-streaming test matrix support the annotation without changing runtime behavior. No new Critical, High, or Medium issue is introduced by the reviewed commit.

Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

@eavanvalkenburg

Copy link
Copy Markdown
Member

Vincent Biret (@baywet) invoke_workflow looks right. Before we merge the breaking rename, can we use #8559 to account for the rest of the workflow spans too—especially MAF-only spans like workflow.build—and note what existing telemetry consumers need to update? Then this can land as one coherent telemetry change.

Rename workflow.name to gen_ai.workflow.name and cover graph and functional workflows in streaming and non-streaming runs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa4dae7a-fdbd-475c-8ec6-fa8333e77cc4
Set gen_ai.operation.name to invoke_workflow when workflow invocation spans are created. Cover graph and functional workflows, attached span creation, and unchanged executor/message spans.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa4dae7a-fdbd-475c-8ec6-fa8333e77cc4
Prepare workflow span names and attributes together so named invocation spans preserve operation and scoped conversation attributes. Cover both span factories, graph and functional workflows, AG-UI propagation, and unnamed fallback behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa4dae7a-fdbd-475c-8ec6-fa8333e77cc4
@agent-framework-automation agent-framework-automation Bot added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label Nov 2, 2026
@baywet

Copy link
Copy Markdown
Member Author

Eduard van Valkenburg (@eavanvalkenburg) I've now aligned the remaining invocation-span names/attributes with the pinned invoke_workflow specification:

  • workflow.name is now gen_ai.workflow.name (3b311d2).
  • gen_ai.operation.name="invoke_workflow" is supplied at span creation (2376062).
  • Named spans now use invoke_workflow {gen_ai.workflow.name}, with the bare-name fallback and conversation-ID propagation preserved (4fae498).

The description now includes the telemetry-consumer migration: use the operation attribute to identify workflow invocations, update the workflow-name attribute lookup, and account for the workflow-name suffix instead of exact matching the old span name. The 23 targeted tests, Ruff checks, and focused source/test Pyright checks passed in Docker.

I've also made the scope explicit: workflow.build and other MAF-only spans/events remain unchanged, since this specification section does not define replacements for them. This does not claim broader workflow-telemetry alignment or close #8559, which currently tracks middleware telemetry.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unnamed graph workflows expose generated UUID-based names in span names and gen_ai.workflow.name, violating low-cardinality conventions and requiring omission of generated names plus regression coverage.

1 open issue

Review effort: Balanced

Track generated-name provenance while retaining runtime names and checkpoint identity. Cover unnamed and explicitly named graph workflows in streaming and non-streaming modes and preserve scoped conversation propagation through core and AG-UI paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa4dae7a-fdbd-475c-8ec6-fa8333e77cc4

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the workflow telemetry alignment with OpenTelemetry GenAI conventions.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

This branch was previously deployed

1 inactive deployment
github-app-auth — 38792a7c Deployed Nov 2, 2026 by baywet via add_label #26878
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change Usage: [PRs], Target: all PRs that introduce changes that are not backward compatible documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants