Skip to content

perf(venv): skip the distutils shim on Python 3.10+ - #1358

Open
konsti-openai wants to merge 1 commit into
aspect-build:mainfrom
konsti-openai:konsti/remove-virtualenv-distutils-shim
Open

perf(venv): skip the distutils shim on Python 3.10+#1358
konsti-openai wants to merge 1 commit into
aspect-build:mainfrom
konsti-openai:konsti/remove-virtualenv-distutils-shim

Conversation

@konsti-openai

Copy link
Copy Markdown

This is a port of pypa/virtualenv#3184 and astral-sh/uv#20222, which removed the distutils hack recently. Removing this brings venv creation in line with virtualenv and uv again and improves Python startup perf.

This is the conservative version of this change. The workaround is only required in an obscure setup where you have the bogus ancient configuration file in the current directory when performing operations. If we can safely assume that bazel users don't do that, we can remove the two files entirely.


Changes are visible to end-users: yes/no

  • Searched for relevant documentation and updated as needed: yes
  • Breaking change (forces users to change their own code or config):no
  • Suggested release notes appear below: yes

Test plan

  • Covered by existing test cases

Stop materializing and importing the virtualenv distutils shim for Python 3.10+, where pip and setuptools already ignore the affected install configuration. Keep the shim for Python 3.9 and move the rules_py-specific PATH setup into the existing venv .pth so console scripts remain discoverable.

Match the upstream changes:
pypa/virtualenv#3184
astral-sh/uv#20222
@konsti-openai
konsti-openai force-pushed the konsti/remove-virtualenv-distutils-shim branch from e57b68d to 6d2673b Compare July 20, 2026 22:21
@jbedard
jbedard requested review from jbedard and xangcastle August 6, 2026 23:59
@jbedard

jbedard commented Aug 7, 2026

Copy link
Copy Markdown
Member

@tamird review

@tamird

tamird commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

@tamird review

what am I reviewing?

@jbedard

jbedard commented Aug 7, 2026

Copy link
Copy Markdown
Member

@tamird review this PR

@tamird

tamird commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

But why? Konsti works at OAI and is a core contributor to uv.

@jbedard

jbedard commented Aug 7, 2026

Copy link
Copy Markdown
Member

@tamird because you provide valuable feedback, and I'm uncertain if someone being an OAI employee means they know how a bazel change will effect your full build/CI?

@tamird

tamird commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

One compatibility case this doesn’t cover: setuptools/distutils checks its venv ignore list before normalizing option names, so underscore-form [install] settings such as install_purelib and install_scripts bypass the built-in guard. The existing _virtualenv.py hook is what strips them. Removing that hook on Python ≥3.10 therefore changes direct legacy setup.py install behavior even though ordinary pip paths stay green. Could we preserve that protection, cover the underscore-form settings, or explicitly document dropping that compatibility?

Separately, this changes the observable virtualenv layout: _virtualenv.py and _virtualenv.pth disappear, and the main .pth changes. We have downstream Python 3.12 image-layout goldens outside the ordinary CI path, so those will need a coordinated update.

@jbedard

jbedard commented Aug 10, 2026

Copy link
Copy Markdown
Member

Please checkout the failing test.

The workaround is only required in an obscure setup where you have the bogus ancient configuration file in the current directory when performing operations. If we can safely assume that bazel users don't do that, we can remove the two files entirely.

Since we are still in v2 alpha maybe we can remove those entirely, how does it effect the tests? Do we know how common this is though? @xangcastle

@acozzette

Copy link
Copy Markdown
Collaborator

I believe the e2e tests can be fixed by running bazel run //:snapshots in e2e/cases. The examples/py_venv failures I'm not sure about, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants