Repository navigation
tests: make the Windows PowerShell tests launch PowerShell less, and faster - #238
Merged
Merged
Conversation
Aaron Li (aaron-jx-li)
force-pushed
the
ci/faster-windows-powershell-tests
branch
2 times, most recently
from
November 16, 2026 03:33
2c2c298 to
c1b70e3
Compare
…faster Since #206 each Windows test job takes about 22 minutes instead of about two: its PowerShell tests launch PowerShell about 100 times per job, and each Windows PowerShell 5.1 launch took 12-18 s on the runner (the job log's progress line that holds them went from 17 s to 21 min). - The tests gave PowerShell no PSModuleAnalysisCachePath. GitHub's Windows image builds the module analysis cache ahead of time at that path, so without it every Windows PowerShell launch analyzed each installed module again before its first cmdlet. Passed through where set, a launch of the Copilot hook takes 0.4 s instead of 18 s (bisected over the runner's environment in #238). pwsh was fast either way. - The slash commands, the piped payload and the no-Python answer ran once per hook event as well as per shell and policy, though none of them depends on the event. The hook keeps its shell x event x policy test (which now also checks that either event leaves bin/te.json); the rest runs once per shell and policy. - The PowerShell tests carry a `powershell` marker, and the Windows job on Python 3.10 leaves them out (PYTEST_ADDOPTS): they test the PowerShell text, which the 3.13 job covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aaron Li (aaron-jx-li)
force-pushed
the
ci/faster-windows-powershell-tests
branch
from
November 16, 2026 03:39
c1b70e3 to
11194a0
Compare
Aaron Li (aaron-jx-li)
marked this pull request as ready for review
November 16, 2026 03:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Since #206, each
tests (windows-latest, …)job takes about 22 minutes instead of about two, and every PR and push tomainwaits on it.mainafter #194 (1e16a37)mainafter #206 (393b9c8)The job log's progress line that holds #206's PowerShell tests went from 17 s to 20 min 57 s. Those tests launch PowerShell about 100 times per job, and each Windows PowerShell 5.1 launch took 12-18 s on the runner. pwsh took about 0.5 s.
Cause
The tests start PowerShell with a minimal environment (
WINDOWS_ENV), which leaves outPSModuleAnalysisCachePath. GitHub's Windows image builds PowerShell's module analysis cache ahead of time at that path (C:\PSModuleAnalysisCachePath\ModuleAnalysisCache). Without it, every Windows PowerShell 5.1 launch analyzes each installed module again (the image has large ones, such as Az) before its first cmdlet (Get-Content,Join-Path,ConvertFrom-Json). Running PowerShell itself was never slow:exit 0takes 0.1 s in any environment.I found it with a temporary probe on the runner, since dropped from this branch. It timed the Copilot hook under Windows PowerShell 5.1, then took the variables out of the runner's environment in halves while the hook stayed fast:
PSModulePathand the profile folders (USERPROFILE,LOCALAPPDATA,APPDATA, …)PSModuleAnalysisCachePathonly (found by the bisect over 141 variables)This only affects the tests. The Copilot CLI runs the hook with the person's whole environment.
Changes
PSModuleAnalysisCachePaththrough where it is set (POWERSHELL_ENV,_powershell_env). Nothing in the plugin reads it.test_powershell_slash_commands_without_git_bash). The hook keeps its shell × event × policy test (test_powershell_hook_without_git_bash), which now also checks that either event leavesbin/te.json. That is about 66 launches per job instead of about 102, and the same checks.powershellmarker. The Windows job on Python 3.10 setsPYTEST_ADDOPTS: -m "not powershell", since these tests check the PowerShell text and the 3.13 job covers it.Results (this PR's CI)
mainnowOn 3.13, 1120 passed, 27 skipped and none failed, and all 16 PowerShell items ran and passed. On 3.10, 1104 passed and those 16 were deselected.
🤖 Generated with Claude Code