Add team_name to DagRunProtocol to match the DagRun Pydantic model. - #68403
Add team_name to DagRunProtocol to match the DagRun Pydantic model.#68403dkranchii wants to merge 2 commits into
team_name to DagRunProtocol to match the DagRun Pydantic model.#68403Conversation
6ea808d to
dc47cdb
Compare
dc47cdb to
fcda014
Compare
|
@dkranchii you will need to run pre-commits to generate the missing files and then commit them |
fcda014 to
5fdd83e
Compare
|
@eladkal please review. |
| consumed_asset_events: list[AssetEventDagRunReference] | ||
| partition_key: str | None | ||
| note: str | None | ||
| team_name: str | None |
There was a problem hiding this comment.
This feels like the wrong place to expose this. team_name is not a property of the dag run, but of the DAG.
There was a problem hiding this comment.
@ashb You're right that team_name is semantically a property of the Dag (resolved through its bundle), not of the run - every DagRun of the same Dag has the same team_name, and there's no team_name column on the DagRun ORM either.
Where it does already live is the server-side Execution API DagRun payload - it was added there (see AddTeamNameField in execution_api/versions/v2026_06_16.py) so it can be delivered as part of TIRunContext at task start, saving the worker an extra get_dag() round trip. This PR is just keeping DagRunProtocol in sync with that server schema so SDK consumers reading dag_run.team_name don't hit a type-checker failure.
I've pushed a comment on the field that spells this out - the denormalization is a deliberate choice at the server-schema layer, not something we're claiming is the "true" home of the field. If you'd rather see this promoted to DagResponse (and thus DagResult on the SDK side) with team_name removed from DagRun, happy to open a follow-up that does that server-side refactor - but I think it's worth keeping the SDK protocol in sync with the server as long as the server exposes it here.
There was a problem hiding this comment.
Yeah, basically this was the cleanest way to plumb the team_name to the context on the worker side.
This feels like the wrong place to expose this. team_name is not a property of the dag run, but of the DAG.
This isn't even true, I wish it was, but the community would not allow it to be a property of DAG or Task (or indeed DagRun) when designing the multi team feature. It is only a property of the Dag Bundle, everything else (Dag, Dag Run, task, etc) all just infer from Bundle.
dag_run sub model on the TIRunContext already has conf, notes, triggering_user_name, etc and it was the closest thing that maps to team_name. And adding a whole new model just for team_name is way over kill. And there is already code that iterates over dag_run and exports the AIRFLOW_CTX_* env vars.
tl;dr: It was the closest match without adding hundreds of lines of code and new mechanisms just for team, this PR is just updating typing.
There was a problem hiding this comment.
@ashb Given the design context @o-nikolas laid out above - team lives on the bundle and DagRun sub-model on TIRunContext is the pragmatic carrier that already ships from the server - this PR is purely a typing sync with the already-shipped Execution API schema. If moving team_name off DagRun server-side is desirable, happy to file a follow-up issue. If you're OK with landing this typing sync as-is, could you resolve the thread?
|
@dkranchii can you address open conversations |
4091fd1 to
0515f95
Compare
1fa7658 to
8078a70
Compare
|
@dkranchii it looks like just static check failures remain, can you address those? |
8078a70 to
40444aa
Compare
Co-authored-by: Niko Oliveira <onikolas@amazon.com>
40444aa to
8fde52a
Compare
|
@o-nikolas can you review this pr. |
|
@ashb all checks have passed and the PR has received approval. If everything looks good, i'd appreciate it if this PR could be merged. Thank you ! |
Ash's concern seems valid there, so I am revoking my approval
ashb
left a comment
There was a problem hiding this comment.
I'm missing something - i don't even understand why we need to pass this from the server at all.
In multi team, a worker belongs to exactly one team doesn't it? So the worker process/config should also know its team name.
What am i missing/misunderstanding
|
Fair question - the intuition breaks down in three places, so it took me a bit to unpack too. 1. The design that landed for multi-team makes the relationship 2. Not every worker/executor is per-team, and even where it is, the task
So "the worker process/config knows its team" is only true for Celery+Edge 3. The server is the authoritative source, and it already ships this.
Scope of this PR. Given all of the above, this PR is only a typing sync: the field is If you'd rather see |
|
@ashb can you review this PR. thanks. |
|
Quite a good write up from @dkranchii but as the person who added that code to the exec api in the first place @ash the tl;dr is that we needed a reliable way to know the team of a task for triggers, xcom, etc. This is directly from the "server" side (can't be manipulated or mis configured by users.) and used in multiple places. This PR is just updating typing, no need to put any major blockers on this one. |

DagRun.team_namewas added in #65617 ("Add team name to task CTX") and isexposed to user code via
context["dag_run"].team_nameand theAIRFLOW_CONTEXT_TEAM_NAMEenv var, butDagRunProtocol(the type usedfor
Context["dag_run"]) was last synced before that change in #65347 anddid not pick the field up. Type checkers therefore flag
context["dag_run"].team_nameas[attr-defined]even though the accessis correct at runtime.
Follow-up to #65347 / #67216, same pattern.
Important
🛠️ Maintainer triage note for @dkranchii · by
@potiuk· 2026-07-08 15:38 UTCHelpful heads-up from the maintainers — please address before this PR can be reviewed:
CI image checks / Static checks). See the contributor guide.Full criteria: Pull Request quality criteria.
The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.
Automated triage — may be imperfect; a maintainer takes the next look.