Skip to content

MNT Split into build-wheels and publish workflows - #18

Merged
lesteve merged 12 commits into
scikit-learn:mainfrom
jeremiedbb:rework-workflows-alt
Nov 5, 2026
Merged

lesteve merged 12 commits into
scikit-learn:mainfrom
jeremiedbb:rework-workflows-alt

Conversation

@jeremiedbb

Copy link
Copy Markdown
Member

Alternative version of #16

Closes #13
Closes #14
Closes #15
Toward #6

This PR proposes to rework the workflows to separate the concerns and required permissions. It makes the release workflow closer to what we've been doing so far. It also allows to safely test the wheels from scikit-learn without keeping 2 wheel config in sync.

The release steps would look as follow:

  • open the release PR on sklearn. Iterate on the PR, using [cd build] to check the wheels, until it's ready.
    (using [cd build] just calls the build-wheels workflow from this repo, but it can't publish or anything, so it's safe)
    (this way the author of the release PR can iterate and test the wheels, whereas right now we need a maintainer of the release repo to test the wheels against that PR)
  • merge the PR and run the publish workflow for the maintenance branch.
    (this workflows calls build-wheels and then publishes to testpypi and pypi in a row)
    • check that everything's fine on test pypi
    • tag the release, then approve the upload to PyPI. It will check that the build about to be published corresponds to the commit that was tagged.

Remark: similar structure as the other PR. The difference is that the publish workflow directly calls build-wheels and uses the artifacts within the job, so no need to loop through previous build workflows.

The nightly builds become:

  • scheduled run of the nightly workflow. It uses the build-wheels workflow, then uploads to anaconda.
    (essentially the same as before)
  • creates an issue on the main repo on failure, once we've enabled that.
    (commented for now since it requires some setup in sklearn)

Side notes:

  • stop uploading to anaconda on merging PRs (that was an oversight of the previous setup)
  • run build-wheels on PRs, but not again on pushes to main. That was wasted work.

Follow-up:

  • if we merge this we'll need to reconfigure trusted publishers for publish.yml instead of wheels.yml.

@jeremiedbb jeremiedbb added this to Labs Sep 17, 2026
@jeremiedbb jeremiedbb moved this to PR waiting for reviews in Labs Sep 17, 2026
@lesteve

lesteve commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Remark: similar structure as the other PR. The difference is that the publish workflow directly calls build-wheels and uses the artifacts within the job, so no need to loop through previous build workflows.

I have a small preference for this PR, for simplicity reasons at least for now. I guess we can always change our mind later if we realize the additional wheel rebuilding is too painful in practice?

We were already doing the additional rebuilding in our previous setup, right? Once in the PR with [cd build] and once when merged in main with the [cd build] commit message?

@thomasjpfan

Copy link
Copy Markdown
Member

Can anyone from the scikit-learn repo trigger a wheel build in this repo using [cd build]? If so, it is more of a security risk and we'll need to be careful about the permissions in .github/workflows/build-wheels.yml.

Given the limited permissions on .github/workflows/build-wheels.yml, I do not see an obvious attack.

@lesteve

lesteve commented Oct 26, 2026

Copy link
Copy Markdown
Member

Can anyone from the scikit-learn repo trigger a wheel build in this repo using [cd build]?

Yes, that was the idea I think. You could decide to only trigger the workflow in PRs from maintainers or something like this if we want to be more restrictive.

It's a bit annoying to have to maintain two slightly different wheels.yml in scikit-learn and scikit-learn-release. At the same time, scikit-learn-release is about tightening up security.

Overall I don't know 🤷. My initial personal conservative instinct was to follow numpy and scipy approach and see later if we wanted to be more adventurous ...

@lesteve

lesteve commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

So I chatted with an agent to investigate the security implications of having a worfklow_call workflow in scikit-learn-release that gets called from scikit-learn.

tl;dr is that it seems OKish but I would kind of like a second opinion on this ... cc @ogrisel @thomasjpfan, see chat session below.

I am also in two minds for the upcoming 1.10 release (penciled in end November), my gut is telling me "let's follow the numpy + scipy approach and revisit this PR later" but that would require us to tweak a bit our habits and update the doc. I can also see how not changing our release habits too much is a good thing, but someone needs to review this PR closely and test it (I am not sure how much testing was done but agents are very good at testing this kind of things). So overall I am like I don't really know 🤷.

Agent chat about security implications


🧠 What's the security implications of workflow_call in #18. Does workflow_call called from scikit-learn trigger have access to the secrets inside the scikit-learn-release repo in principle?


🤖 No: calling workflow_call from scikit-learn does not grant access to secrets stored in scikit-learn-release. The reusable workflow executes in the caller’s security context; its location supplies the workflow code, not that repository’s privileges.

GitHub’s rules are:

  • Repository secrets: the caller must explicitly pass secrets, or use secrets: inherit. In this case, inheritance means secrets available to scikit-learn, not scikit-learn-release.
  • GITHUB_TOKEN: automatically available, but scoped to the calling repository (scikit-learn). The called workflow cannot elevate its permissions.
  • Environments: naming an environment in a reusable workflow does not unlock the called repository’s environment secrets; environments belong to the caller’s run.
  • Organization secrets: could be available if organization policy already grants the caller access and they are passed along.

For PR #18 (#18), the separation looks sensible:

  • build-wheels.yml exposes workflow_call, declares contents: read, and contains no publishing jobs or secret references.
  • The Anaconda token is used only in nightly.yml, which runs on a schedule in scikit-learn-release.
  • PyPI publishing is in publish.yml, triggered by workflow_dispatch, with id-token: write confined to publishing jobs.
  • Calling build-wheels.yml does not trigger either of those wrapper workflows.

The main remaining concern is executing untrusted scikit-learn code during builds. Keep those builds without secrets, avoid secrets: inherit, use a read-only caller token, and ensure privileged publishing only consumes artifacts from trusted runs. Pinning the cross-repository workflow call to a commit SHA also prevents later changes to the reusable workflow from silently changing what the caller executes.

References: GitHub’s secrets-passing rules (https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows#passing-inputs-and-secrets-to-a-reusable-workflow) and caller context and token permissions (https://docs.github.com/en/actions/reference/workflows-and-actions/reusing-workflow-configurations#github-context).


🧠 So if I use ${{ secrets.BLA }} in scikit-learn-release build-wheels.yaml it is actually the secrets from the scikit-learn repo when called from scikit-learn?


🤖 Yes—provided the scikit-learn caller passes it:

jobs:
build:
uses: scikit-learn/scikit-learn-release/.github/workflows/build-wheels.yml@
secrets:
BLA: ${{ secrets.BLA }}

Then ${{ secrets.BLA }} inside build-wheels.yml refers to the passed value from scikit-learn. Explicitly passing it requires declaring BLA under on.workflow_call.secrets in the reusable workflow.

Alternatively, secrets: inherit passes the caller’s available secrets without individual declarations.

Without either, ${{ secrets.BLA }} resolves to an empty string, even if BLA exists in scikit-learn-release.


🧠 Clarify this:

  • I have a secrets BLA set-up in scikit-learn-release repo
  • scikit-learn-release build-wheels.yml uses ${{ secrets.BLA }} and has a trigger workflow_call
  • scikit-learn calls the scikit-learn-release build-wheels.yml

My main question is: is secrets.BLA empty when called from scikit-learn ?

🤖 Yes. In your scenario, ${{ secrets.BLA }} is empty when called from scikit-learn. The secret stored in scikit-learn-release is not available to that run.

It would only have a value if the scikit-learn caller explicitly passed a secret named BLA, or inherited one available to the caller with secrets: inherit.

@thomasjpfan

thomasjpfan commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

I think the workflow in this PR is safe today. We'll need to be extra careful to make sure it continues to be safe. Top of mind:

  1. Do not put this workflow into a deployment environment (if possible)
  2. Do not extend the permissions of this workflow beyond:
permissions:
  contents: read
  1. Do not consume the wheel generated by this workflow in an environment with elevated access. (This PR runs uses: actions/upload-artifact)

@lesteve
lesteve merged commit 4d80eda into scikit-learn:main Nov 5, 2026
37 checks passed
@lesteve

lesteve commented Nov 5, 2026

Copy link
Copy Markdown
Member

Let's merge this one and iterate!

@github-project-automation github-project-automation Bot moved this from PR waiting for reviews to Done in Labs Nov 5, 2026
@lesteve lesteve changed the title MNT Rework workflows: split concerns (alternative) MNT Split into build-wheels and publish workflows Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants