Warn in UI when the same dag_id is registered from multiple files#68791
Warn in UI when the same dag_id is registered from multiple files#68791spider-man-tm wants to merge 3 commits into
Conversation
spider-man-tm
requested review from
XD-DENG,
ashb,
ephraimbuddy and
jedcunningham
as code owners
spider-man-tm
requested review from
bbovenzi,
bugraoz93,
choo121600,
dheerajturaga,
guan404ming,
jason810496,
pierrejeambrun,
potiuk,
rawwar,
ryanahamilton,
shubhamraj-git and
vatsrahul1001
as code owners
There was a problem hiding this comment.
Auto-regenerated by scripts/in_container/run_generate_openapi_spec.py to reflect the new DUPLICATE_DAG_ID value added to DagWarningType.
There was a problem hiding this comment.
Auto-regenerated by pnpm codegen to reflect the new duplicate dag id value added to DagWarningType.
There was a problem hiding this comment.
Auto-regenerated by pnpm codegen to reflect the new duplicate dag id value added to DagWarningType.
There was a problem hiding this comment.
Add DUPLICATE_DAG_ID to DagWarningType to allow recording duplicate dag_id warnings in the dag_warning table.
There was a problem hiding this comment.
Update expected Pydantic validation error message to include the newly added 'duplicate dag id' enum value.
There was a problem hiding this comment.
Auto-regenerated by datamodel-codegen to reflect the new DUPLICATE_DAG_ID value added to DagWarningType.
1 participant
This is a follow-up to #68678, which proposed raising an ImportError when the same dag_id is found in two different files.
Currently, when two files define the same dag_id, Airflow silently overwrites the existing Dag entry with whichever file is parsed last — no error, no warning, no trace in the UI. This means a mis-deployed file can invisibly destroy a production Dag, leaving operators with no indication of what happened or where the conflict originated.
@uranusjr pointed out that an ImportError would fire transiently on every legitimate file rename, since the old DB entry persists until stale_dag_threshold passes. This PR avoids that entirely by recording a DagWarning instead of blocking the parse.
The warning shows both file names and the dag_id so users can self-diagnose:
For file moves the warning is transient and self-clearing; for real duplicates it persists until resolved. The metric dag_processing.duplicate_dag_id is also emitted on each conflict.
Was generative AI tooling used to co-author this PR?