From 470061f53e9bb70045a15b9f4f54a90358909712 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 15:49:30 +0200 Subject: [PATCH 01/29] Draft of "The Design Space" --- peps/pep-0817.rst | 250 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 250 insertions(+) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index bb15a9bc0b0..69dd165a798 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -681,6 +681,256 @@ are used in conjunction with USE flags. For example, build against. +The design space +================ + +Platform compatibility tags cannot express the characteristics described +above, so something must determine whether a given wheel is suitable for +a given system. The question that shapes the rest of the design is where +that knowledge lives, and who is able to add to it. + +One point is worth making at the outset, because it is easily lost: +every approach described here requires a PEP. They differ in what +requires one *next time*. An approach that standardizes the individual +compatibility axes requires a further round of standardization, followed +by an implementation in every installer, each time hardware or a library +introduces an axis that is not already covered. An approach that +standardizes an interface requires that round once. + +.. list-table:: + :header-rows: 1 + :widths: 20 30 26 24 + + * - Approach + - Adding a new compatibility axis requires + - Detection logic maintained by + - Third-party code runs at install time + * - Fixed set of axes + - A PEP, then an implementation in every installer + - Installer maintainers + - No + * - Logic shipped per package + - Nothing; each project decides + - Every variant-shipping project + - Yes + * - Separate package names + - Nothing; a new project name + - Not applicable + - No + * - User-declared environment + - A PEP defining the new marker + - Not applicable; values are supplied by the user + - No + * - Variant providers + - A provider package release + - Provider maintainers + - Yes, unless static information is supplied + +The table states what each approach costs and where the work lands. The +sections below explain why those costs were weighed as they were. + + +Alternatives considered +----------------------- + +A fixed set of compatibility axes +''''''''''''''''''''''''''''''''' + +A PEP could define an extended set of compatibility markers - CUDA +runtime version, GPU architecture, CPU instruction set level, and so +on - together with the semantics for matching them. Wheels would declare +their requirements against these markers, and each installer would +implement the detection needed to evaluate them. The specification would +not need to say how a tool measures any particular marker; that would +remain an implementation matter for each tool. + +This approach has real advantages, and it is the alternative most +frequently raised in discussion. It introduces no new kind of artifact +and no new execution model. The specification stays purely declarative, +and the code performing detection is first-party installer code, so no +new trust surface is created. + +The cost is that the specification becomes the gatekeeper for every +compatibility axis. Each new axis requires agreement across the +packaging community followed by an implementation in each installer, and +a newly specified axis is only as useful as its slowest widely deployed +implementation. The axes in demand today already extend well past GPUs: +CPU instruction set levels, BLAS and LAPACK implementations, OpenMP +runtimes, C++ ABIs, and matching against the ABI of a dependency such as +PyTorch_. Hardware vendors would depend on the standards process in +order to ship optimized wheels, on a timescale measured against their +own release cadence. + +A narrower form of this proposal is to standardize deliberately few +axes - a small set of community-supported hardware target levels, +refreshed periodically - rather than attempting to cover the space. This +is coherent, and it answers a genuine concern: a design permitting many +axes also permits build matrices that are difficult to test and to +audit. Two considerations weigh against it. The set requiring +maintenance is not limited to hardware, as the list above shows, so the +curation burden is larger than it first appears. And it places the +judgment about which hardware merits support with the packaging +community rather than with the vendors and projects that have the +expertise and the motivation to maintain it. The concern about +unmanageable build matrices is better met by observing that nothing +obliges a project to publish many variants; how many to ship remains +that project's decision. + + +Compatibility logic shipped by each project +''''''''''''''''''''''''''''''''''''''''''' + +Each project shipping variants could provide its own detection code, +which the installer would run to choose among that project's wheels, +with no mechanism shared between projects. This is the current situation +extrapolated rather than a new design: the approach described in `Wheel +variant selection via source distribution`_ already works this way. + +The same detection would then be written many times over. Every project +would present its own behavior, its own failure modes and its own user +interface for the same underlying questions, and there would be no +shared story for auditing any of it. Coordination across projects - +ensuring that NumPy_ and SciPy_ agree on a BLAS implementation, or that +everything in an environment is built against the same CUDA major +version - would have no mechanism at all. + + +Separate package names +'''''''''''''''''''''' + +`Package names as variants`_ describes what projects do today. A more +deliberate version of the same idea is to make the mapping part of +installation: the user requests ``torch``, and the installer resolves +that to a project name such as ``torch_cu128`` appropriate for the +current platform. + +This has the attraction of working within the format as it stands. It +requires no filename changes and no metadata changes, and publishers +retain control over what they publish. The drawbacks described in +`Package names as variants`_ still apply: name squatting becomes easier, +packages installing overlapping files cannot be declared mutually +exclusive, and releases cannot be published synchronously across several +project names. + +Two further problems are specific to the mapping. A project depending on +any variant of another project still has no way to express that, and +must either choose one name or enumerate them all. More fundamentally, +the mapping needs a source of truth: something must still determine +which name is appropriate for the current system. That is the question +this section is about, so this approach relocates it rather than +answering it. + + +A user-declared environment +''''''''''''''''''''''''''' + +Compatibility could be matched against values the user declares rather +than values a tool detects. Wheels would carry markers as above, and the +installer would compare them against a description of the environment +supplied by hand or produced by a separate tool. No detection code would +run anywhere during installation. + +This has the smallest trust surface of any approach considered, since no +code runs in order to make a compatibility decision, and it describes +how a substantial group of users already works. HPC sites and +locked-down corporate environments commonly declare the available +toolchain, accelerator and library stack as a matter of policy, using +mechanisms such as those described in `Spack / Archspec`_, and prefer an +explicit, auditable description to anything discovered at install time. + +It is nonetheless unsuitable as the only mechanism, because it requires +every user to know and state their CUDA runtime version, driver version +and instruction set level. That is precisely the knowledge this proposal +exists to stop requiring, and ``{tool} install `` selecting a +working wheel without it is the goal stated in the abstract. + +The design does not treat this as an alternative to be discarded. A +static description of the environment is a first-class part of it, as +described in `Using static compatibility information`_, and it is needed +regardless of any position on trust: installing for a target that is not +the machine running the installer - building container images for other +hardware, populating a wheelhouse, or reproducing an environment in +CI - requires exactly this mechanism. + + +The approach taken: variant providers +------------------------------------- + +The design described in the next section standardizes an interface +rather than a list of axes. Compatibility within a namespace is +determined by a provider, and a new compatibility axis is introduced by +publishing or updating a provider rather than by amending a +specification. + +This places the detection logic with those who have both the expertise +and the motivation to maintain it. A hardware vendor can ship support +for a new accelerator, and correct compatibility rules when they change, +on its own release cadence. Installers remain generic, and do not +acquire a standing obligation to understand every accelerator and ABI on +every platform. Because providers are shared between projects, +coordinating a consistent choice across an environment remains possible. +Where nothing needs to be detected, as with a choice between BLAS +implementations, a provider supplies its properties statically and no +code runs at install time at all; this is described in `Static +properties in wheels`_. + +The costs are real and worth stating plainly. The ecosystem acquires a +new kind of artifact, with its own questions of versioning, trust and +governance. System inspection moves into dependency resolution, where it +was previously confined to the tags a tool computes for itself. Users +may need to obtain a provider in order to install a package they +previously installed without one. And because the properties driving +selection are held in metadata rather than in the filename, determining +which wheel a given system will select requires reading that metadata +rather than a directory listing; :pep:`825` discusses the alternatives +considered for this. + + +How much code should run, and when +---------------------------------- + +Choosing providers settles where compatibility logic lives. It does not +settle how much of that logic should run without the user having said +so, and that question remains genuinely open. It is raised here because +it is the most contested aspect of the design; the mechanisms themselves +belong to the subsequent `Standards Track PEPs for wheel variants`_. + +Both extremes fail. An installer automatically running whatever provider +a wheel's metadata names turns that metadata into a means of executing +code on any machine that merely resolves against the package, before the +user has agreed to install anything. An installer prompting for every +provider produces prompts often enough that they stop being read, and +users grant them wholesale, which arrives at the same place with +additional ceremony. The risk of security fatigue is not a rhetorical +objection to opt-in designs; it is the reason a purely opt-in design +does not by itself resolve the problem. + +It is worth being precise about what changes. System inspection during +resolution is not new: determining whether a ``manylinux`` or +``musllinux`` wheel is compatible already requires probing the running +system, down to reading the interpreter's ELF headers to identify the C +library and its version, and installers perform that inspection today +from vendored code. A provider an installer vendors is much the same +kind of artifact. What is new is that a provider may instead come from a +third party, named by the package being installed. + +Several mechanisms narrow the gap, each with its own cost. Installers +may vendor or reimplement the most widely used providers, which keeps +the common path free of third-party code but concentrates the work on +installer maintainers. A curated list of vetted providers avoids that +concentration, but constitutes a curation authority that must be +governed and that is worth capturing. A static description of the +environment removes the question entirely for users who want that, at +the cost of requiring them to produce one. + +This PEP does not attempt to settle that trade-off. What would change +the reasoning above is a demonstration that the compatibility axes in +demand can be enumerated and kept current through the standards process +at the rate hardware actually changes, or evidence that a static +description of the environment is sufficient for the general case rather +than for those users who already prefer it. + + The wheel variants design ========================= From eef60e4f847a895d14b3182567d274087197380a Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 16:00:00 +0200 Subject: [PATCH 02/29] Updates to "A fixed set of compatibility axes" alternative --- peps/pep-0817.rst | 62 +++++++++++++++++++++++++++++------------------ 1 file changed, 38 insertions(+), 24 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 69dd165a798..6dc8bc3d5cc 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -690,12 +690,14 @@ a given system. The question that shapes the rest of the design is where that knowledge lives, and who is able to add to it. One point is worth making at the outset, because it is easily lost: -every approach described here requires a PEP. They differ in what -requires one *next time*. An approach that standardizes the individual -compatibility axes requires a further round of standardization, followed -by an implementation in every installer, each time hardware or a library -introduces an axis that is not already covered. An approach that -standardizes an interface requires that round once. +every approach described here requires standardization. They differ in +how much, and in what requires more of it *next time*. Standardizing the +compatibility axes themselves means a document for each, since the rules +governing NVIDIA CUDA compatibility have little in common with those +governing AMD ROCm and less still with x86-64 instruction set levels, +and a further document each time hardware or a library introduces an +axis not already covered - followed in every case by an implementation +in each installer. Standardizing an interface means one document, once. .. list-table:: :header-rows: 1 @@ -706,7 +708,7 @@ standardizes an interface requires that round once. - Detection logic maintained by - Third-party code runs at install time * - Fixed set of axes - - A PEP, then an implementation in every installer + - A PEP for that axis, then an implementation in every installer - Installer maintainers - No * - Logic shipped per package @@ -736,13 +738,14 @@ Alternatives considered A fixed set of compatibility axes ''''''''''''''''''''''''''''''''' -A PEP could define an extended set of compatibility markers - CUDA -runtime version, GPU architecture, CPU instruction set level, and so -on - together with the semantics for matching them. Wheels would declare -their requirements against these markers, and each installer would -implement the detection needed to evaluate them. The specification would -not need to say how a tool measures any particular marker; that would -remain an implementation matter for each tool. +The standards process could define compatibility markers for these +characteristics - CUDA runtime version, GPU architecture, CPU +instruction set level, and so on - together with the semantics for +matching them. Wheels would declare their requirements against these +markers, and each installer would implement the detection needed to +evaluate them. The specification would not need to say how a tool +measures any particular marker; that would remain an implementation +matter for each tool. This approach has real advantages, and it is the alternative most frequently raised in discussion. It introduces no new kind of artifact @@ -750,16 +753,27 @@ and no new execution model. The specification stays purely declarative, and the code performing detection is first-party installer code, so no new trust surface is created. -The cost is that the specification becomes the gatekeeper for every -compatibility axis. Each new axis requires agreement across the -packaging community followed by an implementation in each installer, and -a newly specified axis is only as useful as its slowest widely deployed -implementation. The axes in demand today already extend well past GPUs: -CPU instruction set levels, BLAS and LAPACK implementations, OpenMP -runtimes, C++ ABIs, and matching against the ABI of a dependency such as -PyTorch_. Hardware vendors would depend on the standards process in -order to ship optimized wheels, on a timescale measured against their -own release cadence. +It would not, however, be a single document. The rules governing NVIDIA +CUDA compatibility - driver and runtime versions, real and virtual GPU +architectures, and the forward-compatibility guarantees between them - +have little in common with those governing AMD ROCm, and less still with +x86-64 instruction set levels or Arm feature flags. Each needs different +expertise to write and different reviewers to assess it competently, and +a single document covering them all would be difficult to review as a +whole. A realistic first round is therefore several PEPs: plausibly one +per accelerator vendor, and at least one more for CPU instruction sets, +which may not remain single as architectures diverge. + +The recurring cost has the same shape, and makes the specification the +gatekeeper for every compatibility axis. Each new axis requires +agreement across the packaging community followed by an implementation +in each installer, and a newly specified axis is only as useful as its +slowest widely deployed implementation. The axes in demand today already +extend well past GPUs: CPU instruction set levels, BLAS and LAPACK +implementations, OpenMP runtimes, C++ ABIs, and matching against the ABI +of a dependency such as PyTorch_. Hardware vendors would depend on the +standards process in order to ship optimized wheels, on a timescale +measured against their own release cadence. A narrower form of this proposal is to standardize deliberately few axes - a small set of community-supported hardware target levels, From d393665e6098064e7dc65034c432d271186f1824 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 16:05:53 +0200 Subject: [PATCH 03/29] Updates to the "Compatibility logic shipped by each project" alternative --- peps/pep-0817.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 6dc8bc3d5cc..461a84b5eb2 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -798,7 +798,11 @@ Each project shipping variants could provide its own detection code, which the installer would run to choose among that project's wheels, with no mechanism shared between projects. This is the current situation extrapolated rather than a new design: the approach described in `Wheel -variant selection via source distribution`_ already works this way. +variant selection via source distribution`_ already works this way, and +`wheel-stub `__ packages it as +a reusable build backend. Both are confined to source distributions, so +they require arbitrary code execution at install time and leave the +wheel that is actually installed outside anything a lock file records. The same detection would then be written many times over. Every project would present its own behavior, its own failure modes and its own user From 00466e6a05f8492aed702c560249971e63b36e01 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 16:14:59 +0200 Subject: [PATCH 04/29] Update to the "Separate package names" alternative --- peps/pep-0817.rst | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 461a84b5eb2..27168b71b77 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -822,21 +822,25 @@ installation: the user requests ``torch``, and the installer resolves that to a project name such as ``torch_cu128`` appropriate for the current platform. -This has the attraction of working within the format as it stands. It -requires no filename changes and no metadata changes, and publishers +This has the attraction of working within the wheel format as it stands. +It requires no filename changes and no metadata changes, and publishers retain control over what they publish. The drawbacks described in `Package names as variants`_ still apply: name squatting becomes easier, packages installing overlapping files cannot be declared mutually exclusive, and releases cannot be published synchronously across several project names. -Two further problems are specific to the mapping. A project depending on -any variant of another project still has no way to express that, and -must either choose one name or enumerate them all. More fundamentally, -the mapping needs a source of truth: something must still determine -which name is appropriate for the current system. That is the question -this section is about, so this approach relocates it rather than -answering it. +Two further problems are specific to the mapping. The code performing it +has to live in the installer, and it is bound to a naming convention: +something must establish that the resolved name is ``torch_cu128`` +rather than ``torch-cu12`` or ``torch_cuda_128``, and that whatever +convention is chosen holds for every project using the mechanism. How +that would be agreed between projects and kept consistent across +installers is unclear, and the suffix carries meaning that nothing in +the metadata records. More fundamentally, the mapping needs a source of +truth: something must still determine which name is appropriate for the +current system. That is the question this section is about, so this +approach relocates it rather than answering it. A user-declared environment From bb47781f1c120337698cd781bc2b63a28c99b940 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 16:43:42 +0200 Subject: [PATCH 05/29] Ensure "Variant providers" is an alternative treated on par with the others --- peps/pep-0817.rst | 124 +++++++++++++++++++++++++++++++++------------- 1 file changed, 89 insertions(+), 35 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 27168b71b77..a0e8bef056e 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -732,8 +732,8 @@ The table states what each approach costs and where the work lands. The sections below explain why those costs were weighed as they were. -Alternatives considered ------------------------ +Approaches considered +--------------------- A fixed set of compatibility axes ''''''''''''''''''''''''''''''''' @@ -847,10 +847,12 @@ A user-declared environment ''''''''''''''''''''''''''' Compatibility could be matched against values the user declares rather -than values a tool detects. Wheels would carry markers as above, and the -installer would compare them against a description of the environment -supplied by hand or produced by a separate tool. No detection code would -run anywhere during installation. +than values a tool detects. This shares its vocabulary with the first +alternative above - markers for CUDA runtime version, GPU architecture, +instruction set level and so on, defined by the standards process - but +takes their values from a description of the environment that the user +supplies, by hand or produced by a separate tool, rather than from +detection. No detection code would run anywhere during installation. This has the smallest trust surface of any approach considered, since no code runs in order to make a compatibility decision, and it describes @@ -862,27 +864,35 @@ explicit, auditable description to anything discovered at install time. It is nonetheless unsuitable as the only mechanism, because it requires every user to know and state their CUDA runtime version, driver version -and instruction set level. That is precisely the knowledge this proposal -exists to stop requiring, and ``{tool} install `` selecting a -working wheel without it is the goal stated in the abstract. - -The design does not treat this as an alternative to be discarded. A -static description of the environment is a first-class part of it, as -described in `Using static compatibility information`_, and it is needed -regardless of any position on trust: installing for a target that is not -the machine running the installer - building container images for other -hardware, populating a wheelhouse, or reproducing an environment in -CI - requires exactly this mechanism. - - -The approach taken: variant providers -------------------------------------- - -The design described in the next section standardizes an interface -rather than a list of axes. Compatibility within a namespace is -determined by a provider, and a new compatibility axis is introduced by -publishing or updating a provider rather than by amending a -specification. +and instruction set level. The weight of that objection should not be +overstated. Users already need this knowledge today, where it is +translated into a choice of index URL and a bespoke install command, and +declaring it once to the installer would be an improvement on that - not +least because a declared environment composes with dependency +resolution, whereas an index URL is a property of a single command and +does not, so today a project that merely depends on PyTorch_ can undo +the user's choice. The objection is narrower: the burden remains with +the user, whereas ``{tool} install `` selecting a working wheel +without it is the goal stated in the abstract. + +The Wheel Variants design does not treat this as an alternative to be +discarded. A static description of the environment is a first-class part +of it, as described in `Using static compatibility information`_, and it +is needed regardless of any position on trust: installing for a target +that is not the machine running the installer - building container +images for other hardware, populating a wheelhouse, or reproducing an +environment in CI - requires exactly this mechanism. + + +Variant providers +''''''''''''''''' + +Compatibility could be determined by named, versioned plugins - variant +providers - each owning one or more axes of the variant space. The +standards process would define the interface such a plugin exposes and +how its results are used, rather than defining the axes themselves. A +new compatibility axis would then be introduced by publishing or +updating a provider rather than by amending a specification. This places the detection logic with those who have both the expertise and the motivation to maintain it. A hardware vendor can ship support @@ -898,14 +908,58 @@ properties in wheels`_. The costs are real and worth stating plainly. The ecosystem acquires a new kind of artifact, with its own questions of versioning, trust and -governance. System inspection moves into dependency resolution, where it -was previously confined to the tags a tool computes for itself. Users -may need to obtain a provider in order to install a package they -previously installed without one. And because the properties driving -selection are held in metadata rather than in the filename, determining -which wheel a given system will select requires reading that metadata -rather than a directory listing; :pep:`825` discusses the alternatives -considered for this. +governance. The system inspection performed during resolution expands +considerably: installers already probe the platform to evaluate +compatibility tags, but that inspection is bounded and first-party, +whereas a provider's is open-ended and may be third-party. Whether a +user benefits at all depends on the relevant provider being available to +their installer; where it is not vendored, allowlisted or opted into, +its properties are treated as incompatible and the installer falls back +to a generic wheel - a working result, but not the one the hardware +could have had. And because the properties driving selection are held in +metadata rather than in the filename, determining which wheel a given +system will select requires reading that metadata rather than a +directory listing; :pep:`825` discusses the alternatives considered for +this. + + +The approach taken +------------------ + +The Wheel Variants design, described in the next section, takes two of +the approaches above rather than one. Variant providers are the default +mechanism, and a user-declared description of the environment is +supported alongside them as a first-class path rather than as a +fallback. + +Providers are the default because they are the only approach that leaves +``{tool} install `` working for a user who does not know which +accelerator or instruction set their machine has, without making the +specification the gatekeeper for every compatibility axis. Each of the +other approaches gives up one half of that: a fixed set of axes keeps +the declarative model but puts every future axis through the standards +process, per-project logic multiplies the same work by the number of +projects, and separate names relocate the question rather than answering +it. + +The user-declared path is not present to compensate for a weakness in +providers. It answers requirements that detection cannot, whatever one's +view of running code at install time. Installing for a machine other +than the one running the installer - building a container image for +other hardware, populating a wheelhouse, reproducing an environment in +CI - offers nothing to detect, because the target system is absent. +Environments whose policy forbids executing code during installation +need a route that involves none. A user whose hardware is misdetected +needs a way to say so. + +The two fit together rather than merely coexisting, because they supply +the same thing. A provider's output is a set of supported properties in +preference order; a static description declares that set directly. +Selection consumes one form or the other without needing to know which, +so the declared path substitutes at a single point instead of running as +a second mechanism in parallel. That is what makes it reasonable to +offer both without doubling what has to be specified, implemented and +reasoned about. How much code should run, and when From 95bfede12ce71aee85cae5229880902f68dd0e21 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 16:48:36 +0200 Subject: [PATCH 06/29] Remove part of user-declared environment section that is about decision --- peps/pep-0817.rst | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index a0e8bef056e..3dd4cf0005c 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -875,15 +875,6 @@ the user's choice. The objection is narrower: the burden remains with the user, whereas ``{tool} install `` selecting a working wheel without it is the goal stated in the abstract. -The Wheel Variants design does not treat this as an alternative to be -discarded. A static description of the environment is a first-class part -of it, as described in `Using static compatibility information`_, and it -is needed regardless of any position on trust: installing for a target -that is not the machine running the installer - building container -images for other hardware, populating a wheelhouse, or reproducing an -environment in CI - requires exactly this mechanism. - - Variant providers ''''''''''''''''' @@ -930,7 +921,8 @@ The Wheel Variants design, described in the next section, takes two of the approaches above rather than one. Variant providers are the default mechanism, and a user-declared description of the environment is supported alongside them as a first-class path rather than as a -fallback. +fallback; `Using static compatibility information`_ describes the form +that description takes. Providers are the default because they are the only approach that leaves ``{tool} install `` working for a user who does not know which From 369f08dc0fd24a10bb113c612625a23b88fe8244 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 17:27:27 +0200 Subject: [PATCH 07/29] Update the "The approach taken" section --- peps/pep-0817.rst | 56 ++++++++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 3dd4cf0005c..adf4d6d610e 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -924,15 +924,27 @@ supported alongside them as a first-class path rather than as a fallback; `Using static compatibility information`_ describes the form that description takes. -Providers are the default because they are the only approach that leaves -``{tool} install `` working for a user who does not know which -accelerator or instruction set their machine has, without making the -specification the gatekeeper for every compatibility axis. Each of the -other approaches gives up one half of that: a fixed set of axes keeps -the declarative model but puts every future axis through the standards -process, per-project logic multiplies the same work by the number of -projects, and separate names relocate the question rather than answering -it. +Providers are the default because they are the only approach that +satisfies three requirements at once: the right wheel is selected +without the user having to supply the answer, a new compatibility axis +does not require a new round of standardization, and the detection logic +is written once and shared rather than once per project. + +The first of those requirements matters to more people than those who do +not know what hardware they have. The answer is perishable, since a +driver upgrade or a move to another machine can invalidate it; +establishing it is work, and work repeated for every environment; and it +is easy to get wrong in a way that surfaces much later, as a crash or as +an unexplained slowdown. + +The other approaches satisfy some of the three but not all. A fixed set +of axes selects automatically and keeps the logic out of individual +projects, but routes every future axis through the standards process. +Per-project logic avoids that, but writes the same detection once per +project and forgoes any coordination between them. A user-declared +environment hands the question back to the user, and separate names do +not answer it at all, since something must still determine which name +applies. The user-declared path is not present to compensate for a weakness in providers. It answers requirements that detection cannot, whatever one's @@ -941,17 +953,21 @@ than the one running the installer - building a container image for other hardware, populating a wheelhouse, reproducing an environment in CI - offers nothing to detect, because the target system is absent. Environments whose policy forbids executing code during installation -need a route that involves none. A user whose hardware is misdetected -needs a way to say so. - -The two fit together rather than merely coexisting, because they supply -the same thing. A provider's output is a set of supported properties in -preference order; a static description declares that set directly. -Selection consumes one form or the other without needing to know which, -so the declared path substitutes at a single point instead of running as -a second mechanism in parallel. That is what makes it reasonable to -offer both without doubling what has to be specified, implemented and -reasoned about. +need a route that involves none. Deployments that must install +identically across a fleet of machines differing in CPU generation +within one family need the same route for a different reason: detection +would correctly select a different wheel on each, when the point is for +them to match. A user whose hardware is misdetected needs a way to say +so. + +The two approaches fit together rather than merely coexisting, because +they supply the same thing. A provider's output is a set of supported +properties in preference order; a static description declares that set +directly. Selection consumes one form or the other without needing to +know which, so the declared path substitutes at a single point instead +of running as a second mechanism in parallel. That is what makes it +reasonable to offer both without doubling what has to be specified, +implemented and reasoned about. How much code should run, and when From 933864b859c5ca77a646fb04e77530e218531198 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 17:44:32 +0200 Subject: [PATCH 08/29] Updates to the "How much code should run, and when" section --- peps/pep-0817.rst | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index adf4d6d610e..17b864767d8 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -990,29 +990,34 @@ objection to opt-in designs; it is the reason a purely opt-in design does not by itself resolve the problem. It is worth being precise about what changes. System inspection during -resolution is not new: determining whether a ``manylinux`` or -``musllinux`` wheel is compatible already requires probing the running -system, down to reading the interpreter's ELF headers to identify the C -library and its version, and installers perform that inspection today -from vendored code. A provider an installer vendors is much the same -kind of artifact. What is new is that a provider may instead come from a -third party, named by the package being installed. +resolution is not new, and is already more involved than reading a +version string. Establishing ``manylinux`` compatibility requires the +glibc version, which ``packaging`` can obtain by using ``ctypes`` to +load the C library of the running process and call the function that +reports its version. Establishing ``musllinux`` compatibility requires +parsing the interpreter's ELF headers to locate the dynamic loader, then +running that loader as a subprocess and reading the version out of its +output. Installers do this today either by vendoring that code or by +reimplementing it - pip takes the first route, uv the second - and those +are exactly the two strategies proposed for providers. A provider that +an installer vendors or reimplements is much the same kind of artifact. +What is new is that a provider may instead come from a third party, +named by the package being installed. Several mechanisms narrow the gap, each with its own cost. Installers may vendor or reimplement the most widely used providers, which keeps the common path free of third-party code but concentrates the work on installer maintainers. A curated list of vetted providers avoids that concentration, but constitutes a curation authority that must be -governed and that is worth capturing. A static description of the -environment removes the question entirely for users who want that, at -the cost of requiring them to produce one. - -This PEP does not attempt to settle that trade-off. What would change -the reasoning above is a demonstration that the compatibility axes in -demand can be enumerated and kept current through the standards process -at the rate hardware actually changes, or evidence that a static -description of the environment is sufficient for the general case rather -than for those users who already prefer it. +governed. A static description of the environment removes the question +entirely for users who want that, at the cost of requiring them to +produce one. + +This PEP does not attempt to settle that trade-off. The fourth PEP in +the sequence, covering UX, maintainability, security and governance, +takes it up: the opt-in mechanisms, a community-maintained repository of +trusted variant providers that installers can treat as opt-out, and the +model that governs what goes into it. The wheel variants design From 6010dfb599bad6b545111b617248413b99a04786 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 18:47:02 +0200 Subject: [PATCH 09/29] Updates to the text under "The design space" --- peps/pep-0817.rst | 52 ++++++++++++++++++++++++++++++----------------- 1 file changed, 33 insertions(+), 19 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 17b864767d8..866f59f4efd 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -686,50 +686,63 @@ The design space Platform compatibility tags cannot express the characteristics described above, so something must determine whether a given wheel is suitable for -a given system. The question that shapes the rest of the design is where -that knowledge lives, and who is able to add to it. - -One point is worth making at the outset, because it is easily lost: -every approach described here requires standardization. They differ in -how much, and in what requires more of it *next time*. Standardizing the -compatibility axes themselves means a document for each, since the rules -governing NVIDIA CUDA compatibility have little in common with those -governing AMD ROCm and less still with x86-64 instruction set levels, -and a further document each time hardware or a library introduces an -axis not already covered - followed in every case by an implementation -in each installer. Standardizing an interface means one document, once. +a given system. The question that shapes the design is where that +knowledge lives, and who is able to add to it. + +The approaches below differ in how much standardization they require, +and in whether that requirement returns each time a new compatibility +axis appears. For two of them - logic shipped per project, and separate +package names - adding an axis requires no standardization at all, which +is why both already exist as workarounds today; their cost falls +elsewhere. + +Where standardization is required, its shape differs sharply. +Standardizing the compatibility axes themselves means a PEP per axis, +since the rules governing NVIDIA CUDA compatibility have little in +common with those governing AMD ROCm and less still with x86-64 +instruction set levels, and a further PEP each time hardware or a +library introduces an axis not already covered - followed in every case +by an implementation in each installer. Standardizing an interface means +one PEP, once. .. list-table:: :header-rows: 1 - :widths: 20 30 26 24 + :widths: 18 12 26 22 22 * - Approach + - Selects without user input - Adding a new compatibility axis requires - Detection logic maintained by - Third-party code runs at install time * - Fixed set of axes + - Yes - A PEP for that axis, then an implementation in every installer - Installer maintainers - No - * - Logic shipped per package + * - Logic shipped per project + - Yes - Nothing; each project decides - Every variant-shipping project - Yes * - Separate package names + - Unresolved - Nothing; a new project name - - Not applicable + - Unspecified; something must still choose the right name + - No + * - User-declared environments - No - * - User-declared environment - - A PEP defining the new marker + - A PEP for that axis, but no detection to implement - Not applicable; values are supplied by the user - No * - Variant providers + - Yes - A provider package release - Provider maintainers - - Yes, unless static information is supplied + - Yes, unless the provider is vendored or supplied statically The table states what each approach costs and where the work lands. The -sections below explain why those costs were weighed as they were. +sections below take each in turn; `The approach taken`_ explains the +choice that follows from them. Approaches considered @@ -875,6 +888,7 @@ the user's choice. The objection is narrower: the burden remains with the user, whereas ``{tool} install `` selecting a working wheel without it is the goal stated in the abstract. + Variant providers ''''''''''''''''' From c52c5e737ca611eafba6faeecff6c9d64c5826dc Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 19:34:03 +0200 Subject: [PATCH 10/29] Improve "The approach chosen" and add resolution-vs-file-selection --- peps/pep-0817.rst | 50 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 4 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 866f59f4efd..e16d4fc2b3e 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -98,6 +98,7 @@ a single wheel of considerable size, or using separate package names (``mypackage-gpu``, ``mypackage-cpu``, etc.). Each of these approaches has significant drawbacks and potential security implications. + The limitations of platform compatibility tags ---------------------------------------------- @@ -931,13 +932,17 @@ this. The approach taken ------------------ -The Wheel Variants design, described in the next section, takes two of -the approaches above rather than one. Variant providers are the default -mechanism, and a user-declared description of the environment is +The design set out further down in `The wheel variants design`_ takes +two of the approaches above rather than one. Variant providers are the +default mechanism, and a user-declared description of the environment is supported alongside them as a first-class path rather than as a fallback; `Using static compatibility information`_ describes the form that description takes. + +Why providers are the default +''''''''''''''''''''''''''''' + Providers are the default because they are the only approach that satisfies three requirements at once: the right wheel is selected without the user having to supply the answer, a new compatibility axis @@ -960,6 +965,10 @@ environment hands the question back to the user, and separate names do not answer it at all, since something must still determine which name applies. + +Why a declared environment sits alongside +''''''''''''''''''''''''''''''''''''''''' + The user-declared path is not present to compensate for a weakness in providers. It answers requirements that detection cannot, whatever one's view of running code at install time. Installing for a machine other @@ -984,6 +993,33 @@ reasonable to offer both without doubling what has to be specified, implemented and reasoned about. +Why variants stay out of the resolver's search +''''''''''''''''''''''''''''''''''''''''''''''' + +One boundary of the design is worth stating explicitly. Variant +selection chooses among the wheels of a package version that the +resolver has already selected; with one optional exception noted below, +it does not make variants part of the resolver's search. This keeps a +substantial change out of dependency +resolution, where the algorithms in use - backtracking, SAT, PubGrub - +would all have to accommodate it, and where the nearest precedent, +extras, is a cautionary one. The cost is that a conflict between the +variants of two different packages cannot be resolved by searching: if +one package is installed against one BLAS implementation and another +publishes only wheels built against a different one, an installer can +backtrack to an older version of the first package, but it cannot +revisit the wheel it chose for the version it had already settled on. It +has to report the conflict rather than work around it. + +Two things limit how often that arises. Packages sharing a provider +receive the same properties in the same preference order, so independent +per-package choices tend to converge without coordination, and a user +who needs a particular combination can declare it. The one exception is +the ``abi_dependency`` provider described in `Package ABI matching`_, +which does require a resolver to take variants into account, and which +is optional for precisely that reason. + + How much code should run, and when ---------------------------------- @@ -1660,7 +1696,13 @@ version. ``abi_dependency`` is meant to be the only piece of metadata introduced by the specification that affects dependency resolution rather than only -wheel selection within a version. +wheel selection within a version. Choosing among variants in order to +satisfy constraints elsewhere in the dependency graph is ruled out for +variants in general (see `Why variants stay out of the resolver's +search`_), because doing so may be difficult to implement. Support for +``abi_dependency`` is therefore optional for installers: one that does +not implement it selects a variant in the ordinary way and reports any +conflict it cannot resolve. Key points From 7dd99cd10a57e8b8e722b7ced473c9a717f56319 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 20:30:26 +0200 Subject: [PATCH 11/29] Fix one issue in "How much code should run" --- peps/pep-0817.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index e16d4fc2b3e..179345d4ec4 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -1025,8 +1025,11 @@ How much code should run, and when Choosing providers settles where compatibility logic lives. It does not settle how much of that logic should run without the user having said -so, and that question remains genuinely open. It is raised here because -it is the most contested aspect of the design; the mechanisms themselves +so. Both extremes are ruled out, for the reasons below: a provider is +not run merely because a wheel names it, and the user is not asked about +each one in turn. What remains open is how a provider comes to be +trusted, so that it runs without the user opting in to it. That is the +most contested aspect of the design, and the mechanisms themselves belong to the subsequent `Standards Track PEPs for wheel variants`_. Both extremes fail. An installer automatically running whatever provider From 49778c35dc71d251e9af9db3ed762d276073f20c Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 20:42:15 +0200 Subject: [PATCH 12/29] Lift the overview of Standards Track PEPs up to the top of this PEP --- peps/pep-0817.rst | 89 +++++++++++++++++++++++------------------------ 1 file changed, 43 insertions(+), 46 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 179345d4ec4..f2d7fed5e64 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -52,13 +52,46 @@ The goal is for the obvious installation commands (``{tool} install user experience. -Informational PEP ------------------ - -This PEP presents the minimal scope required to meet modern -heterogeneous system needs. It leaves aspects beyond the minimal scope -to evolve via tools or future PEPs. A non-exhaustive list of these -aspects include: +Scope and the PEP sequence +========================== + +This PEP is Informational: it specifies nothing normatively. It serves +as the umbrella for a sequence of four Standards Track PEPs, supplying +the motivation, the prior art, and the reasoning behind the design +choices those PEPs make, so that each of them can be read against a +shared context rather than restating it. It aims to give a concise +high-level overview of wheel variants; it does not cover the complete +functionality proposed as part of the standard, nor does it provide +normative guidance on implementing it. + +The four PEPs are: + +- :pep:`PEP 825: Package Format <825>`: changes to the wheel format, the + initial definition of variant metadata, the index-level metadata file, + variant wheel ordering rules, environment markers and ``pylock.toml`` + integration. + +- Providers: the governance of variant namespaces; extending variant + metadata with provider information; introducing providers that supply + compatible properties statically via variant metadata or dynamically + via opt-in Python packages, along with the security considerations for + their use; defining the relevant plugin API; introducing a special + provider concerned with ABI compatibility. + +- Building: integration with build backends and the ``pyproject.toml`` + file; extending variant metadata and the plugin API to support + verifying the validity of variant properties and embedding static + properties via plugins at build time. + +- UX, maintainability, security and governance: the introduction + strategy, and a community maintained repository of trusted variant + providers that can be integrated with installers for opt-out behavior. + +Of these, :pep:`825` has been accepted as Provisional; the remaining +three are in preparation, and their scope and details may change. + +Some aspects are deliberately left outside the sequence, to evolve via +tools or further PEPs. A non-exhaustive list: - The format of a static file to select variants deterministically or include variants in a ``pylock.toml`` file, @@ -1030,7 +1063,8 @@ not run merely because a wheel names it, and the user is not asked about each one in turn. What remains open is how a provider comes to be trusted, so that it runs without the user opting in to it. That is the most contested aspect of the design, and the mechanisms themselves -belong to the subsequent `Standards Track PEPs for wheel variants`_. +belong to the Standards Track PEPs listed in `Scope and the PEP +sequence`_. Both extremes fail. An installer automatically running whatever provider a wheel's metadata names turns that metadata into a means of executing @@ -1114,43 +1148,6 @@ Variant provider compatible with the system. -Standards Track PEPs for wheel variants ---------------------------------------- - -This PEP aims to provide a concise high-level overview of wheel -variants. It does not cover the complete functionality proposed as part -of the standard, nor does it provide normative guidance on -implementing it. These are provided in subsequent PEPs. - -Currently, the following PEPs are published: - -- :pep:`PEP 825: Package Format <825>`: covering changes to the wheel - format, initial definition of variant metadata, index-level metadata - file, variant wheel ordering rules, environment markers and - ``pylock.toml`` integration. - -The following subsequent PEPs are currently planned: - -- Providers: covering the governance of variant namespaces; extending - variant metadata with provider information; introducing providers that - supply compatible properties statically via variant metadata or - dynamically via opt-in Python packages, along with the security - considerations for their use; defining the relevant plugin API; - introducing a special provider concerned with ABI compatibility. - -- Building: covering integration with build backends and - ``pyproject.toml`` file; extending variant metadata and plugin API to - support verifying the validity of variant properties and embedding - static properties via plugins at build time. - -- TBD: covering UX, maintainability, security, governance and - introduction strategy; introducing a community maintained repository - of trusted variant providers that can be integrated with installers - for opt-out behavior. - -The scope and details of these PEPs may be subject to change. - - High-level overview ------------------- @@ -1747,7 +1744,7 @@ Security Implications ===================== The full security considerations will be provided in the individual -`Standards Track PEPs for wheel variants`_. +Standards Track PEPs listed in `Scope and the PEP sequence`_. The most important concern is that wheel variants introduce a plugin system for querying the platform capabilities. Tools may install these From 4d1add152bedc519218ddc8e04c73be8a98fea33 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 20:51:18 +0200 Subject: [PATCH 13/29] Fix up the Scope section in how it describes the static file format --- peps/pep-0817.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index f2d7fed5e64..866d6cac316 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -93,8 +93,8 @@ three are in preparation, and their scope and details may change. Some aspects are deliberately left outside the sequence, to evolve via tools or further PEPs. A non-exhaustive list: -- The format of a static file to select variants deterministically or - include variants in a ``pylock.toml`` file, +- The exact format of a static file describing an environment's variant + properties, - The list of variant providers that are vendored or re-implemented by installers, - The specific opt-in mechanisms and UX for allowing an installer to run @@ -102,6 +102,10 @@ tools or further PEPs. A non-exhaustive list: - How to instruct build backends to emit variants through the :pep:`517` mechanism. +The first of these concerns the encoding only. The sequence requires +installers to accept a static description of an environment's variant +properties; what it leaves open is the form that description takes. + Motivation ========== From 1c5d90bfc026ac44e76b84c6e5a56a452e50c4cf Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 20:53:31 +0200 Subject: [PATCH 14/29] Fix section title of Standards Track PEPs --- peps/pep-0817.rst | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 866d6cac316..39a01039c16 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -52,8 +52,8 @@ The goal is for the obvious installation commands (``{tool} install user experience. -Scope and the PEP sequence -========================== +Scope and Standards Track PEPs +============================== This PEP is Informational: it specifies nothing normatively. It serves as the umbrella for a sequence of four Standards Track PEPs, supplying @@ -1067,8 +1067,7 @@ not run merely because a wheel names it, and the user is not asked about each one in turn. What remains open is how a provider comes to be trusted, so that it runs without the user opting in to it. That is the most contested aspect of the design, and the mechanisms themselves -belong to the Standards Track PEPs listed in `Scope and the PEP -sequence`_. +belong to the PEPs listed in `Scope and Standards Track PEPs`_. Both extremes fail. An installer automatically running whatever provider a wheel's metadata names turns that metadata into a means of executing @@ -1747,8 +1746,8 @@ The key points in the design we arrived at are: Security Implications ===================== -The full security considerations will be provided in the individual -Standards Track PEPs listed in `Scope and the PEP sequence`_. +The full security considerations will be provided in the individual PEPs +listed in `Scope and Standards Track PEPs`_. The most important concern is that wheel variants introduce a plugin system for querying the platform capabilities. Tools may install these From c56cd19c452ef4e394732c42cc08b43ad5ed1c01 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 20:59:32 +0200 Subject: [PATCH 15/29] Update Change History --- peps/pep-0817.rst | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 39a01039c16..8c33044d914 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -1827,9 +1827,24 @@ and Zanie Blue. Change History ============== -- 14-Aug-2026 - - - Change to informational PEP +- 21-Sep-2026 + + - Change this PEP to an Informational PEP, as an umbrella for + the Standards Track PEPs split off from the first version of + this PEP. + - Added a "The design space" section between the prior art and the + design itself, covering the approaches considered for determining + which variant fits a system, the reasons for choosing variant + providers together with a user-declared environment, and the open + question of how much provider code should run without the user + asking for it. + - Stated that variant selection chooses among the wheels of an + already-selected version rather than taking part in the resolver's + search, and that support for ``abi_dependency``, which is the + exception to that, is optional for installers. + - Clarified that the sequence requires installers to accept a static + description of an environment's variant properties, while leaving + the format of that description open. - 18-Mar-2026 From 751a3778f946c7471e4ca9ffa3bbc3269c8aa96b Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 21:09:40 +0200 Subject: [PATCH 16/29] Expand Backwards Compatibility a bit for previous discussion on DPO --- peps/pep-0817.rst | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 8c33044d914..cf72e27b03f 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -1769,10 +1769,24 @@ Backwards Compatibility Full backwards compatibility considerations are provided in :pep:`825`. The key point is that variant wheels add an additional variant label -component to the wheel filename. Tools commonly used to install wheels -at the time of writing reject (ignore) wheels containing that component, -making it possible to publish variant wheels on an index alongside -non-variant wheels without risk of them being installed accidentally. +component to the wheel filename, and that tools commonly used to install +wheels at the time of writing reject wheels carrying it. This makes it +possible to publish variant wheels on an index alongside non-variant +wheels without risk of them being installed accidentally. + +That rejection is not incidental. A variant wheel has one filename +component more than a non-variant one, so a tool unaware of variants +reads the Python tag in the position where it expects a build number; +build numbers start with a digit and Python tags do not, so the filename +fails verification. To keep that true in future, :pep:`825` requires that +the Python tag component of a wheel filename must not start with a +digit, which no current tag does. + +The guarantee reaches only as far as filename verification does. A +script that parses wheel filenames loosely, rather than verifying them +in full, may treat a variant wheel as an ordinary one. Conversely, +tooling that audits wheel filenames will report variant wheels as +invalid until it is updated to recognize them. Reference Implementation From 670ffdf2ebc53ea8c99635018fdc1e7a0907c924 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 21:25:28 +0200 Subject: [PATCH 17/29] Add section "Depending on a variant-enabled package" --- peps/pep-0817.rst | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index cf72e27b03f..2c2044d538c 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -1669,6 +1669,50 @@ The obtained list will be embedded as ``static-properties`` in the built wheel, so the installer does not have to query the plugin anymore. +Depending on a variant-enabled package +-------------------------------------- + +A project cannot declare a dependency on a particular variant of another +project. There is no way to write ``dependencies = ["torch+cu126"]`` or +anything equivalent; a dependency names a project and a version range, +as it does today. Making ``dependencies = ["torch"]`` reliable is the +goal of this design, rather than allowing build choices to be fixed +across project boundaries. + +Three things argue against permitting it. The first is that naming a +variant couples one project's metadata to another project's build +choices, and those shift over time. A project depending on +``torch+cu124`` would not fail once ``torch`` moved on to ``cu126``; it +would quietly hold its users on the last release that still shipped +``cu124``, where a dependency on ``torch`` would have kept selecting the +highest version. The second is that it would draw variants into the +resolver's search, with the consequences described in `Why variants stay +out of the resolver's search`_. The third is that installers assume +every wheel of a given project version declares the same dependency +metadata, with any differences expressed through environment markers +rather than through separate metadata per wheel; variant-specific +dependencies would break that assumption, and locking across platforms +rests on it. Such a specifier would also offer a way to install a +variant wheel on a system whose user has enabled no providers at all. + +What holds a set of packages together instead is the providers they +share. Where a project and its dependency offer variants governed by the +same provider, the installer makes each selection independently and +arrives at matching values, because the provider reports the same +ordered properties in both cases. + +Not every axis needs to match. A project built without a particular CPU +instruction set runs perfectly well alongside a dependency built with +one; what matters are the axes where combinations are genuinely +incompatible. For those, the recommendation is that a project should not +publish variant builds that fail to work with one of the variants of its +dependency. Exceptions should be rare, and worth stating plainly in the +project's documentation. + +Matching the ABI of a dependency is a related but distinct problem, and +one the design does address; see `Package ABI matching`_ below. + + Package ABI matching -------------------- From 51f716a44ebb3f36f321ac25c265c1939c25fb5b Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 21:40:37 +0200 Subject: [PATCH 18/29] Expand Security Implications section --- peps/pep-0817.rst | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 2c2044d538c..3d3a7af18d0 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -1791,7 +1791,8 @@ Security Implications ===================== The full security considerations will be provided in the individual PEPs -listed in `Scope and Standards Track PEPs`_. +listed in `Scope and Standards Track PEPs`_. The most significant are +summarized here. The most important concern is that wheel variants introduce a plugin system for querying the platform capabilities. Tools may install these @@ -1807,6 +1808,27 @@ will be allowed to vendor, reimplement or otherwise trust specific plugins. A governance model to maintain a repository of trustworthy plugins will be proposed as well. +A provider is expected to run in an environment isolated from the one +being modified, so that it does not gain access to the environment whose +contents it is helping to select, and so that the packages it requires +do not enter that environment. This is expressed as a recommendation +rather than a requirement because the environment that is appropriate +differs between tools: a build frontend, for instance, already has an +isolated environment of its own and may reasonably reuse it. + +A second concern is name squatting, which `Package names as variants`_ +describes as a hazard of the current workarounds: projects register many +similarly named packages, and users learn to expect suffixes that an +attacker can then supply. Variant wheels reduce that pressure rather +than repeat it, since every variant of a release lives under one project +name. Nor do variant namespaces amount to a new global namespace to be +claimed. A namespace has meaning only within the variant metadata of the +project that uses it, and that project also names the provider package +implementing it, so choosing a provider is an ordinary dependency +decision, open to the same scrutiny as any other. Which parties may +define a namespace, and how reserved names such as ``abi_dependency`` +are administered, is covered by the Providers PEP. + Backwards Compatibility ======================= From db810df1296344948128207afff64560368b1594 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 21 Sep 2026 21:58:16 +0200 Subject: [PATCH 19/29] Update and polish the Abstract --- peps/pep-0817.rst | 59 ++++++++++++++++++++++++++--------------------- 1 file changed, 33 insertions(+), 26 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 3d3a7af18d0..1bf1c498f75 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -24,32 +24,39 @@ Abstract Python's existing wheel packaging format uses :doc:`packaging:specifications/platform-compatibility-tags` to specify a -given wheel's supported environments. These tags are unable to express -modern hardware configurations and their features, such as the -availability of GPU acceleration. The tags fail to provide custom -package variants, such as builds against different dependency ABIs. -These inabilities are particularly challenging for scientific computing, -artificial intelligence (AI), machine learning (ML), and -high-performance computing (HPC) communities. - -This PEP proposes "Wheel Variants", an extension to the -:doc:`packaging:specifications/binary-distribution-format`. This -extension introduces a mechanism for package maintainers to declare -multiple variant builds for the same package version, while allowing -installers to automatically select the most appropriate variant based on -system hardware and software characteristics. More specifically, it -proposes: - -- An evolution of the wheel format called **Wheel Variant** that allows - wheels to be distinguished by hardware or software attributes. - -- A **variant provider plugin** interface that allows installers to - dynamically detect platform attributes and select the most suitable - wheel. - -The goal is for the obvious installation commands (``{tool} install -``) to select the most appropriate wheel, and provide the best -user experience. +given wheel's supported environments. These tags cannot express modern +hardware configurations and their features, such as the availability of +GPU acceleration or the instruction sets a CPU supports. Neither can +they express build-time choices that matter at run time, such as which +BLAS implementation a wheel was built against, or which version of a +dependency it matches at the ABI level. These inabilities are +particularly challenging for scientific computing, artificial +intelligence (AI), machine learning (ML), and high-performance computing +(HPC) communities. + +"Wheel Variants" is an extension to the +:doc:`packaging:specifications/binary-distribution-format` that lets a +project publish several wheels for the same version, distinguished by +properties the platform tags cannot carry, and lets installers choose +between them automatically. It comprises: + +- **Variant wheels**, which declare the variant properties they were + built for alongside the existing platform tags. + +- **Variant providers**, which determine the properties a system + supports. A provider may establish that by inspecting the system at + install time, or supply it statically where there is nothing to + detect, as when choosing between BLAS implementations. + +- A **static description of an environment**, which a user may supply in + place of any inspection. It is needed when installing for a machine + other than the one running the installer, and available to anyone who + would rather no provider code ran at all. + +The goal is for the obvious installation command (``{tool} install +``) to select the most appropriate wheel, whether that choice +is detected or declared, with no index URLs, package-name suffixes or +per-project install instructions. Scope and Standards Track PEPs From 8931c078ca59fc68f73623b085589682705a5f12 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Tue, 22 Sep 2026 08:46:27 +0200 Subject: [PATCH 20/29] Fixes to ``abi_dependency`` section Also a minor wording improvement to one other section. --- peps/pep-0817.rst | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 1bf1c498f75..90d2c69434b 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -109,9 +109,9 @@ tools or further PEPs. A non-exhaustive list: - How to instruct build backends to emit variants through the :pep:`517` mechanism. -The first of these concerns the encoding only. The sequence requires -installers to accept a static description of an environment's variant -properties; what it leaves open is the form that description takes. +The first of these concerns the encoding only. Installers are expected +to accept a static description of an environment's variant properties; +what is left open is the form that description takes. Motivation @@ -1743,10 +1743,12 @@ publish the following variants: - ``vllm-0.27.1-...-torch212.whl`` for PyTorch 2.12.x - ``vllm-0.27.1-...-torch211.whl`` for PyTorch 2.11.x -The installer would then default to installing the ``torch213`` variant, -with options to use ``torch212`` or ``torch211`` if necessary to satisfy -other dependencies, rather than being forced to install an older vLLM -version. +The installer would then select whichever of these matches the PyTorch +version it resolves to. If only ``vllm`` is requested, that is +``torch213``, since nothing constrains PyTorch and the newest version is +chosen. Where something else in the resolution holds PyTorch at an older +version, ``torch212`` or ``torch211`` is selected instead, rather than +vLLM having to be downgraded to a release built against that version. ``abi_dependency`` is meant to be the only piece of metadata introduced by the specification that affects dependency resolution rather than only @@ -1755,8 +1757,8 @@ satisfy constraints elsewhere in the dependency graph is ruled out for variants in general (see `Why variants stay out of the resolver's search`_), because doing so may be difficult to implement. Support for ``abi_dependency`` is therefore optional for installers: one that does -not implement it selects a variant in the ordinary way and reports any -conflict it cannot resolve. +not implement it treats the variants using it as incompatible, and is +expected to tell the user that those wheels were skipped. Key points From 41daf5765daf9bf37ddef774d2b92857453eecdf Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 07:24:17 +0000 Subject: [PATCH 21/29] PEP 817: say "platform compatibility tags" where all three are meant "Platform tag" is one of the three tags the wheel filename carries, alongside the Python tag and the ABI tag; "platform compatibility tags" is the collective name, and the name of the specification. Using the bare plural understated the claim being made - none of the three can express these properties, not just the third - and pointed readers at the wrong tag for the ABI example. --- peps/pep-0817.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 90d2c69434b..7f3cfe13392 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -37,11 +37,11 @@ intelligence (AI), machine learning (ML), and high-performance computing "Wheel Variants" is an extension to the :doc:`packaging:specifications/binary-distribution-format` that lets a project publish several wheels for the same version, distinguished by -properties the platform tags cannot carry, and lets installers choose -between them automatically. It comprises: +properties the platform compatibility tags cannot carry, and lets +installers choose between them automatically. It comprises: - **Variant wheels**, which declare the variant properties they were - built for alongside the existing platform tags. + built for alongside the existing platform compatibility tags. - **Variant providers**, which determine the properties a system supports. A provider may establish that by inspecting the system at @@ -147,7 +147,7 @@ The limitations of platform compatibility tags ---------------------------------------------- The current wheel format encodes compatibility through three platform -tags: +compatibility tags: 1. **Python tag**: encoding the minimum Python version and optionally restricting Python distributions (e.g., ``py3`` for any Python 3, From 727bc1ee8e47cc9c5bbbed73cac25c7ba1a48454 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 09:46:00 +0200 Subject: [PATCH 22/29] Mention Intel XPU, improve table entry for opt-in, smaller textual fixes --- peps/pep-0817.rst | 55 ++++++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 25 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 7f3cfe13392..375eabfc3b4 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -63,13 +63,13 @@ Scope and Standards Track PEPs ============================== This PEP is Informational: it specifies nothing normatively. It serves -as the umbrella for a sequence of four Standards Track PEPs, supplying -the motivation, the prior art, and the reasoning behind the design -choices those PEPs make, so that each of them can be read against a -shared context rather than restating it. It aims to give a concise -high-level overview of wheel variants; it does not cover the complete -functionality proposed as part of the standard, nor does it provide -normative guidance on implementing it. +as the umbrella for a sequence of four Standards Track PEPs, and it +supplies the motivation, the prior art, and the reasoning behind the +design choices those PEPs make, so that each of them can be read +against a shared context rather than restating it. It aims to give a +concise high-level overview of wheel variants; it does not cover the +complete functionality proposed as part of the standard, nor does it +provide normative guidance on implementing it. The four PEPs are: @@ -78,12 +78,16 @@ The four PEPs are: variant wheel ordering rules, environment markers and ``pylock.toml`` integration. -- Providers: the governance of variant namespaces; extending variant - metadata with provider information; introducing providers that supply - compatible properties statically via variant metadata or dynamically - via opt-in Python packages, along with the security considerations for - their use; defining the relevant plugin API; introducing a special - provider concerned with ABI compatibility. +- Providers: how the variant properties a system supports are + determined; binding every variant namespace to the one provider that + governs it (e.g., ``nvidia`` for CUDA capabilities, ``x86_64`` for + CPU instruction-set levels, ``blas_lapack`` for the choice of BLAS + implementation); extending variant metadata with provider + information; introducing providers that supply compatible properties + statically via variant metadata or dynamically via opt-in Python + packages, along with the security considerations for their use; + defining the relevant plugin API; introducing a special provider + concerned with ABI compatibility. - Building: integration with build backends and the ``pyproject.toml`` file; extending variant metadata and the plugin API to support @@ -738,7 +742,7 @@ The approaches below differ in how much standardization they require, and in whether that requirement returns each time a new compatibility axis appears. For two of them - logic shipped per project, and separate package names - adding an axis requires no standardization at all, which -is why both already exist as workarounds today; their cost falls +is why both already exist as workarounds today; their costs fall elsewhere. Where standardization is required, its shape differs sharply. @@ -783,7 +787,7 @@ one PEP, once. - Yes - A provider package release - Provider maintainers - - Yes, unless the provider is vendored or supplied statically + - No, unless the user opts in The table states what each approach costs and where the work lands. The sections below take each in turn; `The approach taken`_ explains the @@ -814,13 +818,14 @@ new trust surface is created. It would not, however, be a single document. The rules governing NVIDIA CUDA compatibility - driver and runtime versions, real and virtual GPU architectures, and the forward-compatibility guarantees between them - -have little in common with those governing AMD ROCm, and less still with -x86-64 instruction set levels or Arm feature flags. Each needs different -expertise to write and different reviewers to assess it competently, and -a single document covering them all would be difficult to review as a -whole. A realistic first round is therefore several PEPs: plausibly one -per accelerator vendor, and at least one more for CPU instruction sets, -which may not remain single as architectures diverge. +have little in common with those governing AMD ROCm or Intel XPU, and +less still with x86-64 instruction set levels or Arm feature flags. Each +needs different expertise to write and different reviewers to assess it +competently, and a single document covering them all would be difficult +to review as a whole. A realistic first round is therefore several +PEPs: plausibly one per accelerator vendor, and at least one more for +CPU instruction sets, which may not remain single as architectures +diverge. The recurring cost has the same shape, and makes the specification the gatekeeper for every compatibility axis. Each new axis requires @@ -1834,9 +1839,9 @@ name. Nor do variant namespaces amount to a new global namespace to be claimed. A namespace has meaning only within the variant metadata of the project that uses it, and that project also names the provider package implementing it, so choosing a provider is an ordinary dependency -decision, open to the same scrutiny as any other. Which parties may -define a namespace, and how reserved names such as ``abi_dependency`` -are administered, is covered by the Providers PEP. +decision, open to the same scrutiny as any other. How a namespace is +bound to the provider that governs it, and which names such as +``abi_dependency`` are reserved, is specified by the Providers PEP. Backwards Compatibility From 80b023b3fed0ef561b76b48b3d456c7714516e11 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 10:00:17 +0200 Subject: [PATCH 23/29] Expand on the recurring cost of fixed per-axis PEPs as hardware evolves --- peps/pep-0817.rst | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 375eabfc3b4..59da676f6cd 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -834,9 +834,20 @@ in each installer, and a newly specified axis is only as useful as its slowest widely deployed implementation. The axes in demand today already extend well past GPUs: CPU instruction set levels, BLAS and LAPACK implementations, OpenMP runtimes, C++ ABIs, and matching against the ABI -of a dependency such as PyTorch_. Hardware vendors would depend on the -standards process in order to ship optimized wheels, on a timescale -measured against their own release cadence. +of a dependency such as PyTorch_. + +The cost also recurs within an axis, not only across axes. An axis that +has been specified still has to be revised as its subject moves: new +CUDA and ROCm releases, new GPU architectures and new instruction set +extensions each change which values exist and how they compare. That is +a far faster tempo than the existing platform compatibility tags deal +with, where operating systems, C libraries and broad CPU architectures +turn over slowly - and even at that pace, writing a PEP per tag did not +hold up: :pep:`600` replaced that practice for ``manylinux`` in order to +let maintainers adopt new tags sooner and to make better use of limited +volunteer time. Hardware vendors would depend on the standards process +in order to ship optimized wheels, on a timescale measured against their +own release cadence. A narrower form of this proposal is to standardize deliberately few axes - a small set of community-supported hardware target levels, From e568441675872016bfacfe2f25079a34d97b4583 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 10:09:52 +0200 Subject: [PATCH 24/29] Improve "separate package names" section, use JAX package names --- peps/pep-0817.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 59da676f6cd..05537800740 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -905,14 +905,18 @@ exclusive, and releases cannot be published synchronously across several project names. Two further problems are specific to the mapping. The code performing it -has to live in the installer, and it is bound to a naming convention: -something must establish that the resolved name is ``torch_cu128`` -rather than ``torch-cu12`` or ``torch_cuda_128``, and that whatever -convention is chosen holds for every project using the mechanism. How -that would be agreed between projects and kept consistent across -installers is unclear, and the suffix carries meaning that nothing in -the metadata records. More fundamentally, the mapping needs a source of -truth: something must still determine which name is appropriate for the +has to live in the installer, and it is bound to a naming convention +that does not exist. Projects already spell the same distinction +differently: JAX_ publishes ``jax-cuda12-plugin`` and +``jax-cuda13-plugin``, identifying CUDA by major version alone, while +PyTorch_ labels its builds ``cu129`` and ``cu130`` in index URLs and +local version segments, by major and minor. Something must establish +which form an installer resolves to, and that whatever convention is +chosen holds for every project using the mechanism. How that would be +agreed between projects and kept consistent across installers is +unclear, and the name carries meaning that nothing in the metadata +records. More fundamentally, the mapping needs a source of truth: +something must still determine which name is appropriate for the current system. That is the question this section is about, so this approach relocates it rather than answering it. From 9f192d0c6bc15f831125677df421ba53cd09ee1e Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 10:31:36 +0200 Subject: [PATCH 25/29] Expand Backwards Compatibility, add an Adoption and rollout section --- peps/pep-0817.rst | 51 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 48 insertions(+), 3 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 05537800740..066cd5c547f 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -1865,9 +1865,7 @@ Backwards Compatibility Full backwards compatibility considerations are provided in :pep:`825`. The key point is that variant wheels add an additional variant label component to the wheel filename, and that tools commonly used to install -wheels at the time of writing reject wheels carrying it. This makes it -possible to publish variant wheels on an index alongside non-variant -wheels without risk of them being installed accidentally. +wheels at the time of writing reject wheels carrying it. That rejection is not incidental. A variant wheel has one filename component more than a non-variant one, so a tool unaware of variants @@ -1883,6 +1881,53 @@ in full, may treat a variant wheel as an ordinary one. Conversely, tooling that audits wheel filenames will report variant wheels as invalid until it is updated to recognize them. +What this buys is coexistence. Variant and non-variant wheels are meant +to be published to the same index, under the same project name and for +the same release; neither a separate index nor a separate project name +is needed, and avoiding both is much of the point of this proposal. An +installer that does not understand variants sees the non-variant wheel +and behaves exactly as it does today, while one that does understand +them sees the variant wheels as well and chooses between them. + + +Adoption and rollout +==================== + +The introduction strategy belongs to the fourth PEP in the series, and +this section sketches only its shape. + +A project adopting variants adds to what it publishes on PyPI rather +than replacing it. Variant wheels go into a release that still carries a +non-variant wheel, so that variant-aware installers choose among the +variants while every other consumer continues to receive the wheel it +would have received anyway. Nothing that works today stops working, and +no user has to do anything. + +Dropping the non-variant wheel is a later and separate decision, and a +much slower one. It only becomes reasonable once variant-aware +installers have reached essentially all of a project's users, which is a +matter of years rather than releases, given how long older installers +stay in service in pinned CI images and long-term-support +distributions. Until then, a release published without a non-variant +wheel is simply uninstallable for part of its audience, and what an +installer does instead is not necessarily graceful: it may attempt a +build from source, which can fail with a hard to understand error. + +The workarounds can go sooner. A project that today distributes +per-accelerator builds through separate index URLs, or under separate +project names, can retire those well before it drops its non-variant +wheel, because reaching the users of a workaround is a much easier +problem: using one is an explicit step taken on the project's own +instructions, and the project can change the instructions. The plain +``{tool} install `` path offers no comparable lever, which is +why the non-variant wheel is the part that has to linger. + +The ``null`` variant does not substitute for a non-variant wheel here. +It is a fallback for installers that understand variants but find no +better match; an installer that does not understand variants cannot +select it either, because it carries a variant label like any other +variant wheel. + Reference Implementation ======================== From 1fa0f74d9ccb7a79966bd518ea1111da517e7832 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 10:55:08 +0200 Subject: [PATCH 26/29] Treat environment markers in Backwards Compatibility section --- peps/pep-0817.rst | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 066cd5c547f..98d54b96265 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -1889,6 +1889,27 @@ installer that does not understand variants sees the non-variant wheel and behaves exactly as it does today, while one that does understand them sees the variant wheels as well and chooses between them. +One thing can undermine that, and it is arguably the more consequential +compatibility question of the two. :pep:`825` introduces environment +markers that describe variant properties. A project does not have to use +them, but they are what allows a project whose variants differ in their +dependencies - a CUDA build and a CPU build, say - to keep one +dependency list across the entire release instead of diverging per +wheel, and projects such as PyTorch_ are expected to use them for that +reason. Those markers then appear in the metadata of every wheel in the +release, the non-variant wheel included. + +Unlike an unrecognized filename component, an unrecognized marker is not +uniformly survivable, because it changes the dependency specifier +grammar rather than adding a value to an existing field. What installers +do when they see an unrecognized marker varies. pip from 24.1 onwards, +uv and PDM treat the release as having invalid metadata and resolve to +an earlier one, which is inconvenient but safe; earlier versions of pip +fail outright; Poetry discards markers it cannot parse and proceeds, +which makes the dependencies they guard unconditional, so the +dependencies of every variant are installed together - usually more than +the user needs, and in rare cases things that conflict. + Adoption and rollout ==================== From 27243650483575513a43e3e4d667ee2a9fb4dc89 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 11:06:57 +0200 Subject: [PATCH 27/29] Sharpen text under The Design Space; point out vendors own versioning --- peps/pep-0817.rst | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 98d54b96265..dd0a9f3579c 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -747,12 +747,9 @@ elsewhere. Where standardization is required, its shape differs sharply. Standardizing the compatibility axes themselves means a PEP per axis, -since the rules governing NVIDIA CUDA compatibility have little in -common with those governing AMD ROCm and less still with x86-64 -instruction set levels, and a further PEP each time hardware or a -library introduces an axis not already covered - followed in every case -by an implementation in each installer. Standardizing an interface means -one PEP, once. +and a further one each time hardware or a library introduces an axis not +already covered, followed in every case by an implementation in each +installer. Standardizing an interface means one PEP, once. .. list-table:: :header-rows: 1 @@ -849,6 +846,14 @@ volunteer time. Hardware vendors would depend on the standards process in order to ship optimized wheels, on a timescale measured against their own release cadence. +The vendors are also not bound by the standards process. A +specification that writes down how a platform's versions relate to one +another records rules that the packaging community does not own and +cannot hold still: the vendor can redefine them in any hardware or +software release it makes, leaving the specification wrong until it is +amended. The renumbering of macOS versions is a reminder that this is +not hypothetical. + A narrower form of this proposal is to standardize deliberately few axes - a small set of community-supported hardware target levels, refreshed periodically - rather than attempting to cover the space. This From 6a924a5e2781cdbb581d916ee33e9855f7f073a2 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 11:25:52 +0200 Subject: [PATCH 28/29] Reword "a static description of an environment" description Focus on the use cases where this functionality is essential, and mention the "runs the least code" as a secondary benefit. The providers have a good security story, so there's no need to have the static file option if that were the only reason; instead it's a hard necessity for some use cases like building containers. --- peps/pep-0817.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index dd0a9f3579c..4bdf8c8f8af 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -49,9 +49,9 @@ installers choose between them automatically. It comprises: detect, as when choosing between BLAS implementations. - A **static description of an environment**, which a user may supply in - place of any inspection. It is needed when installing for a machine - other than the one running the installer, and available to anyone who - would rather no provider code ran at all. + place of any inspection. Building a container image, a native + installer, or a wheel that vendors a particular build of a dependency + calls for a stated target rather than a discovered one. The goal is for the obvious installation command (``{tool} install ``) to select the most appropriate wheel, whether that choice @@ -937,13 +937,17 @@ takes their values from a description of the environment that the user supplies, by hand or produced by a separate tool, rather than from detection. No detection code would run anywhere during installation. -This has the smallest trust surface of any approach considered, since no -code runs in order to make a compatibility decision, and it describes -how a substantial group of users already works. HPC sites and +Many installs have nothing to detect. An install that produces an +artifact for use elsewhere - a container image, a native installer, or a +wheel that vendors a particular build of a dependency - is a statement +about the target, not about the machine doing the work, and the machine +doing the work may have no GPU in it at all. The approach also describes +how a substantial group of users already works: HPC sites and locked-down corporate environments commonly declare the available toolchain, accelerator and library stack as a matter of policy, using -mechanisms such as those described in `Spack / Archspec`_, and prefer an -explicit, auditable description to anything discovered at install time. +mechanisms such as those described in `Spack / Archspec`_. And it has +the smallest trust surface of any approach considered, since no code +runs in order to make a compatibility decision. It is nonetheless unsuitable as the only mechanism, because it requires every user to know and state their CUDA runtime version, driver version From e10c3c01ac1bae1806dc93c84810ea8bb1442d6e Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 28 Sep 2026 18:07:22 +0300 Subject: [PATCH 29/29] Apply suggested textual edits, fix mixed braces in a few places --- peps/pep-0817.rst | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/peps/pep-0817.rst b/peps/pep-0817.rst index 4bdf8c8f8af..e73c5f283f2 100644 --- a/peps/pep-0817.rst +++ b/peps/pep-0817.rst @@ -54,7 +54,7 @@ installers choose between them automatically. It comprises: calls for a stated target rather than a discovered one. The goal is for the obvious installation command (``{tool} install -``) to select the most appropriate wheel, whether that choice +{package}``) to select the most appropriate wheel, whether that choice is detected or declared, with no index URLs, package-name suffixes or per-project install instructions. @@ -98,7 +98,7 @@ The four PEPs are: strategy, and a community maintained repository of trusted variant providers that can be integrated with installers for opt-out behavior. -Of these, :pep:`825` has been accepted as Provisional; the remaining +Of these, :pep:`825` has been provisionally accepted; the remaining three are in preparation, and their scope and details may change. Some aspects are deliberately left outside the sequence, to evolve via @@ -959,7 +959,7 @@ least because a declared environment composes with dependency resolution, whereas an index URL is a property of a single command and does not, so today a project that merely depends on PyTorch_ can undo the user's choice. The objection is narrower: the burden remains with -the user, whereas ``{tool} install `` selecting a working wheel +the user, whereas ``{tool} install {package}`` selecting a working wheel without it is the goal stated in the abstract. @@ -975,7 +975,7 @@ updating a provider rather than by amending a specification. This places the detection logic with those who have both the expertise and the motivation to maintain it. A hardware vendor can ship support -for a new accelerator, and correct compatibility rules when they change, +for a new accelerator, and correct the compatibility rules when they change, on its own release cadence. Installers remain generic, and do not acquire a standing obligation to understand every accelerator and ABI on every platform. Because providers are shared between projects, @@ -1864,8 +1864,7 @@ claimed. A namespace has meaning only within the variant metadata of the project that uses it, and that project also names the provider package implementing it, so choosing a provider is an ordinary dependency decision, open to the same scrutiny as any other. How a namespace is -bound to the provider that governs it, and which names such as -``abi_dependency`` are reserved, is specified by the Providers PEP. +bound to the provider that governs it is specified by the Providers PEP. Backwards Compatibility @@ -1949,7 +1948,7 @@ project names, can retire those well before it drops its non-variant wheel, because reaching the users of a workaround is a much easier problem: using one is an explicit step taken on the project's own instructions, and the project can change the instructions. The plain -``{tool} install `` path offers no comparable lever, which is +``{tool} install {package}`` path offers no comparable lever, which is why the non-variant wheel is the part that has to linger. The ``null`` variant does not substitute for a non-variant wheel here.