Skip to content

Crash diagnosability + out-of-process coverage aggregation — v0.9.0 - #166

Merged
jamesabel merged 4 commits into
masterfrom
crash-hardening-and-coverage
Aug 23, 2026
Merged

Crash diagnosability + out-of-process coverage aggregation — v0.9.0#166
jamesabel merged 4 commits into
masterfrom
crash-hardening-and-coverage

Conversation

@jamesabel

Copy link
Copy Markdown
Owner

Implements crash_hardening_and_coverage_plan.md (motivated by the silent 0xC0000005 death of the GUI process on 2026-08-22, and once before under 0.7.3).

What's in

  • faulthandler (faults.py): armed in the parent and every spawn child via PYTEST_FLY_FAULTHANDLER; dumps to .pytest-fly/logs/faulthandler-<pid>.log. report_previous_crashes() logs + archives non-empty dumps at the next launch (Log tab, EVENT_EXTRA). pytest's own faulthandler plugin is disabled in the test child (plan option b). Pref faulthandler_enabled (default on).
  • WER LocalDumps (platform/wer.py + Configuration tab "Crash Diagnostics" group): read-only status, dump folder/type/count prefs, Configure (UAC) / Remove / Copy command, machine-wide caution, startup sweep of new *.dmp files.
  • Single cov.report() pass in calculate_coverage() (§4.2).
  • Out-of-process aggregation (coverage_aggregator.py): CoverageTracker and the Coverage tab's HTML report run calculate_coverage() in a spawn child with a timeout (coverage_timeout_seconds, default 300). A crashed/hung child is a warning; last good values stand; the run continues.
  • Rate limiting: coverage_refresh_seconds (default 30, 0 = every completion); final pass always runs once nothing is running/queued.

Deliberately not done

  • §4.3 incremental combine — premise is wrong for coverage 7.14.2: Coverage.combine() replaces the combined data rather than merging into it (verified experimentally), so merging only new files would silently drop coverage.
  • §0 line-length 160 — the user-level standard for this repo is 192.
  • §0 "AI-authored test marker" — no such convention exists in this repo.
  • §2.5 Run-tab crash banner (marked optional) — Log tab WARNING only.

Tests: 486 passed locally, incl. a real segfaulting child (faulthandler end-to-end) and a dying/hung aggregator child.

🤖 Generated with Claude Code

https://claude.ai/code/session_018j7yAH7CooFHFLEANWM5vM

jamesabel and others added 4 commits August 23, 2026 09:56
A native crash (access violation in python.dll) killed the GUI process twice with zero diagnostic residue. faults.py arms faulthandler in the parent and every spawn child (via PYTEST_FLY_FAULTHANDLER), writes dumps to logs/faulthandler-<pid>.log, and report_previous_crashes() logs and archives non-empty dumps at startup. pytest's own faulthandler plugin is disabled in the test child so one file per PID stays authoritative.

platform/wer.py reads the Windows Error Reporting LocalDumps state, builds the elevated configure/remove commands, and sweeps new *.dmp files at startup. New prefs: faulthandler_enabled, wer_*, coverage_* (UI wiring follows).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018j7yAH7CooFHFLEANWM5vM
Coverage.report() returns the total percentage for every output_format, so the total-only pass was a complete second parse of every PUT source file just to read a number the text pass already returns. Halves the cost of the step that was on the stack when the process died.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018j7yAH7CooFHFLEANWM5vM
CoverageAggregator (spawn child) runs calculate_coverage() out of the GUI process; aggregate_coverage() joins it with a timeout (coverage_timeout_seconds pref) and treats a crashed, killed, or hung child as a logged warning that keeps the last good values. CoverageTracker holds new completed-test sets for coverage_refresh_seconds (default 30) while the run is active and always does a final pass once nothing is running or queued. The Coverage tab's HTML report goes through the same child, polled from a timer so the GUI no longer blocks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018j7yAH7CooFHFLEANWM5vM
…ut fields — v0.9.0

Crash Diagnostics group (faulthandler checkbox; on Windows the WER LocalDumps status, folder/type/count, Configure (UAC) / Remove / Copy command buttons and a machine-wide caution). Coverage Refresh and Coverage Timeout fields next to Refresh Rate. Restore-defaults wiring, CLAUDE.md architecture notes, version 0.9.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018j7yAH7CooFHFLEANWM5vM
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 67.70335% with 135 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.14%. Comparing base (6cfa052) to head (52d4458).

Files with missing lines Patch % Lines
.../pytest_fly/gui/configuration_tab/configuration.py 34.86% 71 Missing ⚠️
src/pytest_fly/platform/wer.py 64.70% 36 Missing ⚠️
src/pytest_fly/gui/coverage_tab/coverage_tab.py 78.26% 10 Missing ⚠️
src/pytest_fly/faults.py 90.00% 6 Missing ⚠️
src/pytest_fly/pytest_runner/pytest_process.py 16.66% 5 Missing ⚠️
...rc/pytest_fly/pytest_runner/coverage_aggregator.py 90.47% 4 Missing ⚠️
src/pytest_fly/pytest_runner/process_monitor.py 50.00% 1 Missing ⚠️
src/pytest_fly/pytest_runner/system_monitor.py 50.00% 1 Missing ⚠️
src/pytest_fly/pytest_runner/test_list.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #166      +/-   ##
==========================================
- Coverage   87.41%   86.14%   -1.27%     
==========================================
  Files          71       74       +3     
  Lines        5759     6150     +391     
==========================================
+ Hits         5034     5298     +264     
- Misses        725      852     +127     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jamesabel
jamesabel merged commit 89677c0 into master Aug 23, 2026
3 checks passed
@jamesabel
jamesabel deleted the crash-hardening-and-coverage branch August 23, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants