Skip to content

Slice A: coverage fully out of the GUI process; children stop importing the GUI — v0.9.1 - #167

Merged
jamesabel merged 4 commits into
masterfrom
slice-a-coverage-out-of-process
Aug 23, 2026
Merged

Slice A: coverage fully out of the GUI process; children stop importing the GUI — v0.9.1#167
jamesabel merged 4 commits into
masterfrom
slice-a-coverage-out-of-process

Conversation

@jamesabel

Copy link
Copy Markdown
Owner

Summary

Slice A of the crash review: remove the two biggest native-crash surfaces from the GUI process.

  1. No coverage.Coverage object is ever constructed in the GUI process. Aggregation already ran in a spawn child after an access violation in python.dll, but CoverageTracker's per-test pass (Coverage(cov_file).load() on its worker thread, once per completed test per recalculation) and compute_per_test_coverage on the run-prep thread still did. CoverageAggregator now takes optional per_test_names and returns a CoverageResult (totals + per-test executed-line counts + union); the tracker and the ordering path derive their fractions from that.
  2. Spawn children no longer import pytest-fly's GUI package. Every child re-imports pytest_fly.__main__main.py, which imported .gui at module level — so each test process, ProcessMonitor, SystemMonitor, CoverageAggregator and GetTests loaded PySide6 and all eight tabs first (measured 0.58 s and tens of MB commit per child, two children per test), and pytest-fly's own Qt binding sat in the test process before the PUT's tests ran. main.fly_main is now a lazy wrapper and pytest_runner/__init__.py resolves its exports via PEP 562. The PUT is unaffected — if its tests import PySide6/PyQt, they still do exactly as before; only pytest-fly's own side-effect import is gone.

Measured

before after
PySide6 in a child's sys.modules yes no
pytest_fly.gui.* in a child yes no
import pytest_fly.__main__ 0.58 s 0.28 s

pytest is still imported by children: interfaces.py builds PyTestFlyExitCode from pytest.ExitCode by design.

Tests

  • test_child_returns_per_test_executed_lines — child per-test counts match compute_per_test_coverage, both fraction conventions.
  • test_spawn_child_entry_does_not_import_gui — subprocess probe pins the import footprint.
  • Full suite: 487 passed. ruff clean; ty unchanged (pre-existing diagnostics only).

Behaviour change

Per-test coverage numbers for the Coverage tab now arrive with the rate-limited aggregation pass rather than being computed on the tracker thread — same values, same cadence as the combined number.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KEZ9TpT9VwBujBq1S3v2Jd

jamesabel and others added 4 commits August 23, 2026 11:58
…the GUI process

Aggregation (combine + report) already ran in a spawn child after an access violation
in python.dll inside the GUI process, but two in-process coverage paths survived:
CoverageTracker's per-test pass constructed Coverage(cov_file).load() on its worker
thread for every completed test on every recalculation, and compute_per_test_coverage
did the same on the run-prep thread for coverage-efficiency ordering. Same library, same
process, same thread situation as the documented crash.

CoverageAggregator now takes optional per_test_names and returns a CoverageResult
(totals + per-test executed-line counts + the union size). The tracker and the ordering
path both derive their fractions from that — per_test_fractions(total_statements) for
the Coverage tab, per_test_fractions() (union) for ordering — so the GUI process never
constructs a coverage.Coverage object. compute_per_test_coverage remains as a pure
in-process helper for the child and tests; calculate_coverage's tuple return is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZ9TpT9VwBujBq1S3v2Jd
Under the forced spawn start method every child (test process, ProcessMonitor,
SystemMonitor, CoverageAggregator, GetTests) re-imports pytest_fly.__main__, which imported
main.py, which did `from .gui import fly_main` at module level. So a process monitor that
only samples psutil first loaded PySide6 and all eight tab widgets: measured 0.58 s and
tens of MB of commit per child, two children per test, and pytest-fly's own Qt binding
resident in the test process before the program under test's tests ran — sys.modules
pollution, and a real crash source for a PUT on a different Qt binding. The PUT's own
imports are untouched: if its tests import PySide6/PyQt they still do, exactly as before.

main.fly_main is now a thin lazy wrapper that imports the GUI package at call time (the
attribute stays for monkeypatching), and pytest_runner/__init__.py resolves PytestRunner /
PytestRunState / GetTests via PEP 562 __getattr__ so unpickling a monitor class no longer
drags the orchestration layer in. pytest itself is still imported (interfaces.py builds
PyTestFlyExitCode from pytest.ExitCode) — that is a deliberate dependency.

After: PySide6 and pytest_fly.gui absent from every child; __main__ import 0.58 s → 0.28 s.
test_spawn_child_entry_does_not_import_gui pins this.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZ9TpT9VwBujBq1S3v2Jd
…der-test stand-in

tests/ is the PUT when pytest-fly runs on itself, so test_pyside_app.py gives every run a
real Qt program under test: a CounterWindow driven with qtbot (clicks, reset, a custom
signal via waitSignal). Its fourth test runs only inside a pytest-fly child (detected by the
exported PYTEST_FLY_FAULTHANDLER) and asserts that pytest_fly.gui is not pre-loaded into
the PUT's process — verified to fail against master's module-level GUI import when the
parent is `python -m pytest_fly`, and to pass on this branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZ9TpT9VwBujBq1S3v2Jd
…ironment

The isolation check keyed on PYTEST_FLY_FAULTHANDLER, but the suite's own
enable_faulthandler tests export that variable into the top-level pytest process, so
in CI the check ran outside a child (where the GUI tests legitimately import
pytest_fly.gui) and failed. PytestProcess names its process after the test module, so
current_process().name is the only signal that is true exactly inside the child.
Re-verified: skips under top-level pytest with the variable set, passes as a child on
this branch, fails as a child against master's module-level GUI import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZ9TpT9VwBujBq1S3v2Jd
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.45614% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.08%. Comparing base (89677c0) to head (9642c93).

Files with missing lines Patch % Lines
...rc/pytest_fly/pytest_runner/coverage_aggregator.py 78.57% 6 Missing ⚠️
src/pytest_fly/gui/run_tab/control_window.py 50.00% 2 Missing ⚠️
src/pytest_fly/main.py 33.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #167      +/-   ##
==========================================
- Coverage   86.14%   86.08%   -0.06%     
==========================================
  Files          74       74              
  Lines        6150     6168      +18     
==========================================
+ Hits         5298     5310      +12     
- Misses        852      858       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jamesabel
jamesabel merged commit feaf70c into master Aug 23, 2026
3 checks passed
@jamesabel
jamesabel deleted the slice-a-coverage-out-of-process branch August 23, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants