From 56fb661760ec8763a10654bb32e1ce50747e96c7 Mon Sep 17 00:00:00 2001 From: Jarek Potiuk Date: Sun, 18 Oct 2026 17:06:57 +0200 Subject: [PATCH] Record lessons from the October 2026 providers waves in the release docs The 2026-10-14 and 2026-10-17 waves hit steps the instructions did not cover: a provider outside the wave shipping with a common.compat floor below the first release that has the symbol it imports, a candidate that had to be rebuilt before its vote email went out, a build clone with no upstream remote, an svn commit that cannot prompt from an agent session, tester confirmations that only lived in issue comments, and a PyPI token prompt that needs a terminal an agent does not have. Generated-by: Claude Opus 5 --- .../prepare-providers-documentation/SKILL.md | 8 ++ dev/README_RELEASE_PROVIDERS.md | 100 ++++++++++++++++-- 2 files changed, 101 insertions(+), 7 deletions(-) diff --git a/.agents/skills/prepare-providers-documentation/SKILL.md b/.agents/skills/prepare-providers-documentation/SKILL.md index f706073229c96..ecdb78a8db754 100644 --- a/.agents/skills/prepare-providers-documentation/SKILL.md +++ b/.agents/skills/prepare-providers-documentation/SKILL.md @@ -696,6 +696,14 @@ vote rather than re-cut in this release. > doc preparation and PR creation, so it is easy to forget when the skill > hands back to the regular release workflow. +The command only resolves pins an author marked. While classifying, also +note every PR that starts importing a symbol from another provider (most often +`common.compat`) and check that the importing provider's floor is at least +the first release that shipped it, even when the importing provider is not in +this wave. If it is not, raise the floor in this PR and tell the release +manager. The recipe is in "Update versions of dependent providers to the next +version" in `dev/README_RELEASE_PROVIDERS.md`. + **Provider dependency-bump CI guard.** Every `>=` bump this produces (and any inter-provider `>=` bump made during the wave, e.g. a `breaking` provider that dependents must now require) trips the `check_provider_dependency_bumps` diff --git a/dev/README_RELEASE_PROVIDERS.md b/dev/README_RELEASE_PROVIDERS.md index ba745516126da..af262adda32cd 100644 --- a/dev/README_RELEASE_PROVIDERS.md +++ b/dev/README_RELEASE_PROVIDERS.md @@ -42,6 +42,7 @@ - [Commit the source packages to Apache SVN repo](#commit-the-source-packages-to-apache-svn-repo) - [Publish the Regular distributions to PyPI (release candidates)](#publish-the-regular-distributions-to-pypi-release-candidates) - [Push the RC tags](#push-the-rc-tags) + - [Re-cut a candidate before the vote email is sent](#re-cut-a-candidate-before-the-vote-email-is-sent) - [Prepare documentation in Staging](#prepare-documentation-in-staging) - [Prepare issue in GitHub to keep status of testing](#prepare-issue-in-github-to-keep-status-of-testing) - [Prepare voting email for Providers release candidate](#prepare-voting-email-for-providers-release-candidate) @@ -516,6 +517,22 @@ breeze release-management update-providers-next-version > interactive breeze command instead of the skill. Run it regardless of whether you think any > provider uses the `# use next version` comment — the command is a no-op when none do. +The command only acts on lines the PR author marked. It does not notice when a PR starts importing +something from another provider (most often `common.compat`) without raising the minimum version +pin. That holds even for a provider that is **not** in this wave: its next release ships with the +old floor, and users who already have that older dependent provider installed get an `ImportError`. +For every new import from another provider in the wave's PRs, find the first release that shipped +the symbol and make sure the pin is at least that version: + +```shell script +# The commit that introduced the symbol, then the first provider release tag that contains it +git log --reverse --format=%h -S "" -- providers/common/compat/src | head -1 +git tag --contains "providers-common-compat/*" --sort=v:refname | grep -v rc | head -1 +``` + +In the 2026-10-14 wave, `openai` imported `get_async_extra_dejson` but still pinned +`common.compat` below 1.21.0, the first release that has it. + ## Create a PR with the changes Make sure to set labels: `allow provider dependency bump` and `skip common compat check` to the PR, @@ -724,6 +741,11 @@ key you want to use. ```shell script export RELEASE_DATE=$(date "+%Y-%m-%d%n") export AIRFLOW_REPO_ROOT=$(pwd -P) +# The merge commit of the release-prep PR. Tag and build exactly this commit, never "whatever HEAD is". +export WAVE_SHA= +# The remote that `git remote -v` shows pushing to apache/airflow: `origin` in a fresh clone +export APACHE_REMOTE= +git checkout ${WAVE_SHA} rm -rf ${AIRFLOW_REPO_ROOT}/dist/* ``` @@ -732,13 +754,16 @@ rm -rf ${AIRFLOW_REPO_ROOT}/dist/* These instructions assume the standard remote naming convention (`upstream` → `apache/airflow`, `origin` → your fork — see [`contributing-docs/10_working_with_git.rst`](../contributing-docs/10_working_with_git.rst#git-remote-naming-conventions)). +A fresh `git clone` used for the build (see below) has only `origin`, pointing at `apache/airflow`. +Set `APACHE_REMOTE` below to whichever remote `git remote -v` shows pointing at `apache/airflow` (a +URL such as `git@github.com:apache/airflow.git` works too). Never push the tags to your fork. Set tags for the providers in the repo. ```shell script echo "Tagging with providers/${RELEASE_DATE}" -git tag -s providers/${RELEASE_DATE} -m "Tag providers for ${RELEASE_DATE}" --force -git push upstream providers/${RELEASE_DATE} +git tag -s providers/${RELEASE_DATE} ${WAVE_SHA} -m "Tag providers for ${RELEASE_DATE}" --force +git push ${APACHE_REMOTE} providers/${RELEASE_DATE} breeze release-management prepare-provider-distributions --include-removed-providers --distribution-format both --version-suffix "" breeze release-management prepare-tarball --tarball-type apache_airflow_providers --version "${RELEASE_DATE}" ``` @@ -768,8 +793,8 @@ if you only build few packages, run: ```shell script echo "Tagging with providers/${RELEASE_DATE}" -git tag -s providers/${RELEASE_DATE} -m "Tag providers for ${RELEASE_DATE}" --force -git push upstream providers/${RELEASE_DATE} +git tag -s providers/${RELEASE_DATE} ${WAVE_SHA} -m "Tag providers for ${RELEASE_DATE}" --force +git push ${APACHE_REMOTE} providers/${RELEASE_DATE} breeze release-management prepare-provider-distributions --include-removed-providers --distribution-format both --version-suffix "" PACKAGE PACKAGE .... breeze release-management prepare-tarball --tarball-type apache_airflow_providers --version "${RELEASE_DATE}" @@ -823,6 +848,10 @@ svn commit -m "Add artifacts for Airflow Providers ${RELEASE_DATE}" cd "$AIRFLOW_REPO_ROOT" ``` +`svn commit` asks for your ASF password unless SVN has it cached, so run it in your own terminal. +From an agent session or a script without a terminal it fails with `E215004` (authentication +failed) instead of prompting. + Verify that the files are available in the ${RELEASE_DATE} folder under [providers](https://dist.apache.org/repos/dist/dev/airflow/providers/) @@ -914,6 +943,22 @@ twine check ${AIRFLOW_REPO_ROOT}/dist/* read -rs TWINE_PASSWORD && export TWINE_PASSWORD # paste the token, press Enter ``` + `read -s` needs a terminal, so it cannot work when an agent or a script runs the upload for + you. In that case, use Magpie's + [`secret-prompt`](https://github.com/apache/magpie/tree/main/tools/secret-prompt) instead of the + two exports. It asks for the token in the terminal or, when there is none, in a dialog (macOS + `osascript`, Linux `zenity`/`kdialog`, or `pinentry`). It shows the exact command that will + receive the token and passes the token only to that command, so the token never lands in your + shell, a file or the agent's transcript. Install it once, then prefix `twine upload` in the + command below: + + ```shell script + uv tool install "git+https://github.com/apache/magpie#subdirectory=tools/secret-prompt" + # ... then, in the upload command below, replace "twine upload" with: + secret-prompt --env TWINE_PASSWORD --set TWINE_USERNAME=__token__ --prefix pypi- \ + --title "PyPI upload" --prompt "Paste the single-use PyPI token" -- twine upload + ``` + 4. Run the upload (below). 5. **Delete the token** at [Account settings → API tokens](https://pypi.org/manage/account/#api-tokens) and run @@ -972,8 +1017,41 @@ Verify the pushed tags point where you think they do. Plain `git ls-remote` prin SHA for annotated tags, which never matches the commit and looks alarming; dereference it with `^{}`: ```shell script -git ls-remote upstream "refs/tags/providers-/^{}" -``` +git ls-remote ${APACHE_REMOTE} "refs/tags/providers-/^{}" +``` + +## Re-cut a candidate before the vote email is sent + +When a problem turns up after the candidates are built and staged but **before** the vote email has +gone out, you can rebuild the same wave instead of starting a new one. Nobody has started verifying +it yet. Merge the fix, then: + +* Keep `RELEASE_DATE`. The SVN folder, the wave tag and the vote email keep their names. +* Check that the fix is the only change to the wave's providers since the original build: + `git log --oneline .. -- providers/ ...`. Anything else listed + would ship untested and missing from the changelog and the testing issue: either fold it in + (changelog entries, testing-issue rows) or apply the fix on a branch cut from the old `WAVE_SHA` + and tag that commit instead. +* Set `WAVE_SHA` to the new commit and move the wave tag there: + `git tag -s -f providers/${RELEASE_DATE} ${WAVE_SHA} -m "Tag providers for ${RELEASE_DATE}"`, then + `git push --force ${APACHE_REMOTE} refs/tags/providers/${RELEASE_DATE}`. +* Check out the new commit (`git checkout ${WAVE_SHA}`, and confirm with `git rev-parse HEAD`), then + re-run the [build and sign](#build-and-sign-the-source-and-convenience-packages) steps from the + cleanup of `dist/` onwards, minus the wave tag you just moved. Skip the `export RELEASE_DATE=...` + line there and keep the original date, or a re-cut on a later day renames the tarball. +* In `dev/airflow/providers/${RELEASE_DATE}`, `svn rm` the files of any provider or version that is no + longer in the wave, copy the new build over the rest, and commit (`svn add --force .` picks up any + new files). The filenames carry no `rcN`, so unchanged providers' files are simply replaced. +* PyPI never lets you reuse a filename, so publish the next candidate number (`rc2` → `rc3`), and pass + it explicitly as `--version-suffix rc3`: breeze only moves past an rcN whose tag exists locally, + and the previous candidate's tags may not have been created yet. If the wave's providers are at + different candidate numbers, build each group with its own next suffix. The previous `rcN` stays on PyPI, + and its tags stay in git if they were pushed. Nobody voted on it, and the vote email names only the + new one. +* Push the new RC tags at `${WAVE_SHA}` and republish the staging docs. + +Once the vote email is out, do not do this: cancel the vote and run a new wave, or remove the +provider from it. ## Prepare documentation in Staging @@ -1104,6 +1182,11 @@ sed -i '' -E "s/- \[ \] (.*\(#(${checked})\))/- [x] \1/" files/provider_issue.md sed -i -E "s/- \[ \] (.*\(#(${checked})\))/- [x] \1/" files/provider_issue.md ``` +Testers do not always tick the box: some confirm a PR in a comment instead ("tested #12345, works"). +Before you extract the ticks, read the previous issue's comments and tick any PR confirmed there in +the previous issue itself, so the record is complete and the extraction picks it up. In the +2026-10-14 wave, five such PRs were confirmed only in comments. + Review the resulting `[x]` lines in `files/provider_issue.md` (PRs only present in the new wave stay unchecked). If the issue was already created, apply the same local edit and then edit the online issue as described in Step 3. @@ -1933,6 +2016,9 @@ twine check ${AIRFLOW_REPO_ROOT}/dist/*.whl ${AIRFLOW_REPO_ROOT}/dist/*.tar.gz read -rs TWINE_PASSWORD && export TWINE_PASSWORD # paste the token, press Enter ``` + When an agent or a script without a terminal runs the upload, use `secret-prompt` instead, as + described in the [release candidate upload](#publish-the-regular-distributions-to-pypi-release-candidates). + 4. Run the upload (below). 5. **Delete the token** at [Account settings → API tokens](https://pypi.org/manage/account/#api-tokens) and run @@ -1990,7 +2076,7 @@ breeze release-management tag-providers --release-date ${RELEASE_DATE} Then confirm each pushed tag resolves to the wave commit (`^{}` dereferences the annotated tag): ```shell script -git ls-remote upstream "refs/tags/providers-/^{}" +git ls-remote "${APACHE_REMOTE:?set it to the remote that pushes to apache/airflow}" "refs/tags/providers-/^{}" ``` ## Update airflow-site index for new providers