Skip to content

Fix Variable.set rewriting team_name of existing variables (#71810) - #71823

Open
ccoliu wants to merge 2 commits into
apache:mainfrom
ccoliu:fix-variable-set-team-name-override
Open

Fix Variable.set rewriting team_name of existing variables (#71810)#71823
ccoliu wants to merge 2 commits into
apache:mainfrom
ccoliu:fix-variable-set-team-name-override

Conversation

@ccoliu

@ccoliu ccoliu commented Aug 19, 2026

Copy link
Copy Markdown

When core.multi_team = True, calling Variable.set(key="k", value="v2", team_name="team_a") on an existing variable rewrites team_name to "team_a" in the ON CONFLICT (key) DO UPDATE clause.
This overwrites the original ownership of existing global variables (or variables belonging to other teams), causing subsequent global lookups (e.g. Variable.get("k") without team_name, as done during DAG parsing) to fail with KeyError.
This PR removes team_name from update_fields in Variable.set(), preserving the original team_name ownership of existing variables during value updates.
closes: #71810


Was generative AI tooling used to co-author this PR?
  • Yes

Generated-by: Claude following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@ccoliu
ccoliu requested review from XD-DENG and ashb as code owners August 19, 2026 08:23
@boring-cyborg

boring-cyborg Bot commented Aug 19, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@ccoliu ccoliu changed the title Fix Variable.set rewriting team_name of existing variables (#71810) Fix Variable.set rewriting team_name of existing variables (#71810) label:area:core Aug 19, 2026
@ccoliu ccoliu changed the title Fix Variable.set rewriting team_name of existing variables (#71810) label:area:core Fix Variable.set rewriting team_name of existing variables (#71810) Aug 19, 2026

@vincbeck vincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, but doing this would then prevent to change the ownership of a variable through the public API? We might need to differentiate the public API vs execution API path?

@ccoliu
ccoliu force-pushed the fix-variable-set-team-name-override branch from ca3f712 to 29d37c6 Compare August 19, 2026 15:25
@ccoliu

ccoliu commented Aug 19, 2026

Copy link
Copy Markdown
Author

Interesting, but doing this would then prevent to change the ownership of a variable through the public API? We might need to differentiate the public API vs execution API path?

Good catch! I checked the current implementation:

  1. Public API PATCH /variables/{key} and Bulk Update: Uses update_orm_from_pydantic() which directly sets ORM attributes — it does NOT go through Variable.set(), so changing team_name ownership via the REST API remains fully supported and is unaffected by this change.
  2. Execution API PUT /execution/variables/{key}: This is the task-execution path — it calls Variable.set() with the team_name automatically derived from the task's JWT token. Removing team_name from update_fields ensures that a task value-update won't implicitly overwrite the ownership of an existing global variable.
  3. For new variables: team_name remains in upsert_values, so initial ownership is correctly assigned when a variable is first created.

So the boundary is already there — the only path that silently overwrites team_name was the Variable.set() upsert, which is primarily used by the Execution API path.

@vincbeck

Copy link
Copy Markdown
Contributor

Interesting, but doing this would then prevent to change the ownership of a variable through the public API? We might need to differentiate the public API vs execution API path?

Good catch! I checked the current implementation:

  1. Public API PATCH /variables/{key} and Bulk Update: Uses update_orm_from_pydantic() which directly sets ORM attributes — it does NOT go through Variable.set(), so changing team_name ownership via the REST API remains fully supported and is unaffected by this change.
  2. Execution API PUT /execution/variables/{key}: This is the task-execution path — it calls Variable.set() with the team_name automatically derived from the task's JWT token. Removing team_name from update_fields ensures that a task value-update won't implicitly overwrite the ownership of an existing global variable.
  3. For new variables: team_name remains in upsert_values, so initial ownership is correctly assigned when a variable is first created.

So the boundary is already there — the only path that silently overwrites team_name was the Variable.set() upsert, which is primarily used by the Execution API path.

Thanks!

@vincbeck vincbeck added the backport-to-v3-3-test Backport to v3-3-test label Aug 19, 2026
@eladkal eladkal added this to the Airflow 3.3.2 milestone Aug 19, 2026
@eladkal eladkal added the type:bug-fix Changelog: Bug Fixes label Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v3-3-test Backport to v3-3-test type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Variable.set rewrites team_name of an existing variable in multi-team mode - intended?

3 participants