Speed up CI, especially the Windows matrix - #74
Merged
Conversation
Windows CI runs the suite in ~265-300s vs ~65s on Ubuntu (~4x), largely concentrated in file-concurrency tests hitting the Windows fsops retry-on-PermissionError path, plus a long tail of thousands of tests each somewhat slower (atomic writes, msvcrt locks, real subprocess spawns in test_e2e_*). Under load this recently tipped a fixed 30s subprocess timeout in test_store_concurrency.py into a flaky failure. - pytest-xdist + `-n auto`: distributes tests across the runner's cores. Locally, full suite: ~25s serial -> ~7-9s parallel, no cross-test contamination (2720 passed either way). - Skip coverage instrumentation on Windows: only ubuntu/3.12's coverage is uploaded, so the other five matrix legs -- ubuntu included -- were paying for it with nothing to show. `-o addopts=` clears pyproject's default `--cov` flags for the Windows invocation. - Cache pip downloads via actions/setup-python's `cache: pip`. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pushed run: windows-latest/3.13 regressed under -n auto -- slower overall (347s vs the ~298s pre-xdist baseline) and still hit the pre-existing "writer timed out" flake, this time in a different test in the same file. test_store_concurrency.py spawns 8 real Python subprocesses itself and asserts they finish within a fixed deadline. Under -n auto those 8 processes now also compete with three other xdist worker processes for the runner's CPU -- more contention than serial execution had, not less. Run that file outside the parallel pool so it gets the runner to itself; everything else still parallelizes. Also widen _WRITER_TIMEOUT 30 -> 90 for margin: this test's assumption of near-dedicated CPU to start 8 interpreters and contend for one file lock doesn't hold reliably on shared Windows CI runners even in isolation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Windows CI runs the suite in ~265-300s vs ~65s on Ubuntu (~4x). Under load this recently tipped a fixed 30s subprocess timeout in
test_store_concurrency.pyinto a flaky failure on PR #70 (run).Changes
pytest-xdist+-n auto— distributes tests across the runner's cores. Locally: full suite ~25s serial → ~7-9s parallel (with or without coverage), 2720 passed both ways, no cross-test contamination found.ubuntu-latest, 3.12leg's coverage is uploaded (see theUpload coverage reports to Codecovstep'sif:), so the other five matrix legs, including three of the four Ubuntu ones, were paying the instrumentation cost for nothing.-o addopts=clears pyproject's default--covflags for the Windows invocation only; Ubuntu keeps coverage as before, now also parallelized.cache: piponactions/setup-python, keyed onpyproject.toml— cuts most of the ~55s dependency install.Verified locally
ruff clean, workflow YAML validated.
Doesn't change test semantics or the matrix's version coverage — a follow-up could additionally right-size the heaviest concurrency tests' iteration counts or trim the Windows matrix to one Python version, if this isn't enough on its own.
🤖 Generated with Claude Code