Skip to content

fix(crashlog): build the crash context outside the lock - #210

Merged
donislawdev merged 1 commit into
masterfrom
fix/crashlog-no-deadlock
Sep 28, 2026
Merged

donislawdev merged 1 commit into
masterfrom
fix/crashlog-no-deadlock

Conversation

@donislawdev

@donislawdev donislawdev commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Summary

The crash logger could deadlock the whole program. crashlog._record built the report context while holding its only lock, and the context provider is the GUI's own code (gui/crash.context). Both ways it deadlocked were reproduced before the fix:

  • Same thread. When a Control page field held an invalid value (a letter in Loss), the provider's read of the form raised ValueError and passed it to crashlog.note(). That call went back into record() and tried to take the lock the same thread already held. The thread hung, and every later report from every thread queued behind it, the main thread included.
  • Across threads (real Tk 9.0). A worker thread's provider read a tk.StringVar, which waits for the Tk main loop, while the main loop was waiting on the logger's lock.

The WinDivert handle still closes in both cases: the stop path closes it first and the 10 s stall check fires. The damage was a frozen window (STOP did nothing), and a watchdog whose own crash-log calls queue behind the wedged lock.

Changes

  • crashlog.py
    • The duplicate check stays under the lock. The context is built outside it, and the entry is inserted under the lock with a second look. If another thread recorded the same new fault in the meantime, the occurrence merges into its record (_count_again) and nothing is written to disk twice.
    • A thread-local re-entrancy guard in _collect_context: a fault the provider records itself gets the base context only (context_provider_skipped: "re-entered"). Without the guard, provider -> note -> provider would recurse even with the lock out of the way.
    • The module docstring and set_context_provider now state the provider's contract: plain data, no GUI toolkit calls, no lock a recording thread may hold. A stale sentence claiming the CLI sets a provider is removed; only the GUI does.
  • gui/crash.py
    • context reads plain data only:
      • Form: a raw copy taken by leave_breadcrumb on the main thread every tick, kept in a WeakKeyDictionary in this module. App is exactly on both class ratchets, so the copy is not stored on App.
      • Counters: app.last_snapshot, the same numbers one tick old, read without the engine lock.
      • Invalid field: the report gets settings_error plus the raw form instead of recording a fault.
    • A real failure inside context is still recorded (crashlog.quiet). That is safe now, because no lock is held and the provider is not asked again.
    • The per-tick copy costs about 23 us on real Tk 9.0.4 (42 fields, measured), against a tick of ~0.7 s.

Tests

New in tests/test_crashlog.py. Each test swaps in a private lock, so a regression fails that one test instead of hanging the suite.

  • test_a_provider_that_records_a_fault_of_its_own_does_not_wedge_the_logger
  • test_two_threads_can_build_their_context_at_the_same_time: a barrier inside the provider, passable only when both threads are in it at once.
  • test_the_same_new_fault_from_two_threads_at_once_is_one_record
  • test_a_gui_crash_report_never_reads_tk_off_the_main_thread: the real App on the fake Tk, an invalid field, and a worker writing a report.

Mutation proof: four new entries in tests/test_mutation_registry.py, all four caught:

  • re-entrancy guard off
  • provider under the lock again
  • no second look
  • GUI report reading Tk again

All 12 crashlog/crash: entries were run and caught. The move_to_end anchor moved into the helper, so its indentation was updated.

Run locally:

  • the guard selection for the changed files, plus ruff and mypy on the changed modules;
  • the reproduction probes before and after the fix. Before: hang after 3 s, and deadlock until the 8 s faulthandler dump. After: both finish in about 0.1 s, and the form is read on the main thread only.

Not run locally: the full suite (it runs here on Linux and Windows).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Fixed an issue that could freeze the app when a form contained an invalid value or a background error occurred, leaving STOP unresponsive.
    • Crash reports continue to include available settings and now identify invalid form values as invalid, rather than reading form values during error reporting.

The context provider is the GUI's own code, and it ran while the crash
logger held its only lock. An invalid form field made the provider record
its own ValueError into that lock on the same thread, and a worker's Tk
read waited for a main loop that was waiting on the lock. Either way every
later report from every thread queued behind it: the window froze and STOP
did nothing.

- crashlog: de-duplicate under the lock, build the context outside it,
  insert under the lock again with a second look (a concurrent record of
  the same new fault merges into one). A thread-local guard records a
  fault the provider raises itself without asking the provider again.
- gui/crash: the report reads plain data only - a raw form copy taken on
  the main thread each tick, and the last counter snapshot. An invalid
  field is reported as settings_error instead of being recorded.
- Four tests and four mutation entries, all caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Advanced

Run ID: fc23ce17-a700-4c72-bc44-8c4e945585c6

📥 Commits

Reviewing files that changed from the base of the PR and between 1382383 and 2e36b66.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • beantester/crashlog.py
  • beantester/gui/crash.py
  • tests/test_crashlog.py
  • tests/test_mutation_registry.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (9)
  • GitHub Check: pip-audit (advisories against the pinned set)
  • GitHub Check: Analyze (python)
  • GitHub Check: mutation registry
  • GitHub Check: tests (ubuntu-latest, py3.14)
  • GitHub Check: tests (windows-latest, py3.14)
  • GitHub Check: mypy
  • GitHub Check: semgrep (ERROR, HIGH and CRITICAL block)
  • GitHub Check: Analyze (actions)
  • GitHub Check: review new dependencies
🧰 Additional context used
📓 Path-based instructions (15)
Applies to text shown to the user (labels, buttons, tooltips, placeholders, dialogs, errors, status messages, empty states, translations).

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
Verify tests check real behavior and would fail if the implementation were broken.

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
Domain: network condition simulator (latency, loss, throttling, disconnects) built on WinDivert via PyDivert, with a Tkinter GUI and a CLI over one engine.

⚙️ CodeRabbit configuration file

Files:

  • beantester/gui/crash.py
  • beantester/crashlog.py
These are end-user desktop applications.

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
Performance is a known weak spot of these projects.

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
Applies only to code that builds or styles a GUI.

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
User-facing changelog.

⚙️ CodeRabbit configuration file

Files:

  • CHANGELOG.md
SECURITY, HIGH PRIORITY.

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
These apps are QA/developer tools.

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
Check that documentation matches the actual code in this PR: commands, flags, config keys, file paths, build steps and examples must exist.

⚙️ CodeRabbit configuration file

Files:

  • CHANGELOG.md
Python code.

⚙️ CodeRabbit configuration file

Files:

  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
All code in this repository is written by an AI coding agent (Claude Code).

⚙️ CodeRabbit configuration file

Files:

  • CHANGELOG.md
  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
Source excerpt: **Flat hyphen only.**

📄 CodeRabbit inference engine (.github/claude-review-rules.md)

Files:

  • CHANGELOG.md
  • tests/test_crashlog.py
  • tests/test_mutation_registry.py
  • beantester/gui/crash.py
  • beantester/crashlog.py
No hardcoded UI styling: Only if the PR adds or changes GUI code (XAML, Slint, Fyne, Tkinter, WPF code-behind): warn if new or changed UI code sets colors, fonts, font sizes, margins, paddings, sizes or corner radii as literal values on ind...

📄 CodeRabbit inference engine (Custom checks)

Files:

  • beantester/gui/crash.py
Source excerpt: **Anything visible from outside goes in the changelog.**

📄 CodeRabbit inference engine (.github/claude-review-rules.md)

Files:

  • CHANGELOG.md
🔇 Additional comments (4)
beantester/gui/crash.py (1)

73-89: LGTM!

tests/test_mutation_registry.py (1)

1917-1959: LGTM!

CHANGELOG.md (1)

8-15: LGTM!

beantester/crashlog.py (1)

151-167: LGTM!

Also applies to: 246-276, 304-311


📝 Walkthrough

Walkthrough

Crash reports now use cached GUI settings rather than reading form widgets during failure handling. The crash logger runs context providers outside its lock, detects provider re-entry, and rechecks fingerprints to deduplicate concurrent reports.

Changes

Crash reporting

Layer / File(s) Summary
Cached GUI crash context
beantester/gui/crash.py, tests/test_crashlog.py, tests/test_mutation_registry.py, CHANGELOG.md
The GUI caches raw settings and uses them to build crash context. Invalid settings are reported with their raw values and parse error. A worker-thread test checks this path, and the changelog describes the freeze fix.
Provider execution and concurrent recording
beantester/crashlog.py, tests/test_crashlog.py, tests/test_mutation_registry.py
The logger runs context providers outside its lock, skips provider re-entry, and counts duplicate fingerprints after a locked recheck. Tests cover concurrent context collection, re-entry, and duplicate reports.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested labels: bug, performance

Merge Risk: ⚪ Minimal · up to 2e36b

No actionable merge-blocking defect is established. The change is mergeable after normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 2e36b

The change addresses a program-wide logging deadlock without adding an identified external entry point or verified security flaw. Crash reports can now include invalid form values, so the contents and protection of local crash files remain relevant.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The demonstrated exposure is the running desktop process and its local crash files: form input reaches report data, not a newly identified remote or privileged entry point.

Trust Boundaries and Controls

  • observed — The GUI-thread cache separates Tk reads from worker-thread crash collection. Settings validation gates reproduction-command generation, while invalid values remain available as report data.

Resilience and Maintainability Implications

  • observed — The thread-local guard prevents a provider-owned fault from recursively collecting provider context, and the second locked lookup prevents concurrent creators from writing two records for the same new fingerprint.
🚥 Pre-merge checks | ✅ 14
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: crash context is built outside the logger lock. It is specific enough for release notes and future git history.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Tests For Changed Behavior ✅ Passed The PR changes runtime crash logging and GUI context behavior, and it adds four focused tests in tests/test_crashlog.py covering provider re-entry, concurrent context construction, concurrent dedupl…
No Secrets Or Debug Leftovers ✅ Passed The PR changes only CHANGELOG.md, beantester modules, and tests. It adds no CLAUDE.md, AGENTS.md, .claude, or .env files. The added test print("CONTEXT_OK") is an intentional subprocess success marker…
No Hardcoded Ui Styling ✅ Passed The PR changes beantester/gui/crash.py, but only crash-context collection and breadcrumb data handling. The diff adds no Tkinter controls and sets no colors, fonts, sizes, margins, paddings, corner …
No Obvious Performance Problems ✅ Passed No clear performance problem matches the check. The new GUI form snapshot reads the existing 42-field raw form once per 700 ms tick; it performs no parsing or I/O, and the code documents this path as …
Desktop Robustness ✅ Passed The changed code only moves crash-context construction outside _lock, adds re-entry/concurrency handling, and caches raw GUI form data in memory. settings_from_raw handles invalid cached input wit…
Safe File Parsing ✅ Passed The changed code does not add a reader for XML, CSV, XLSX, YAML, archive, or settings files. settings_from_raw() parses cached GUI form values, not file content. The crash logger uses the existing s…
System Changes Are Reversible ✅ Passed PASS — The PR changes crash-context collection, in-memory deduplication, breadcrumb caching, and tests. It does not add or change code that modifies network filters, proxies, firewalls, system time, p…
Clear User-Facing Text ✅ Passed PASS: The only user-facing prose added is the clear changelog entry. It names the freeze, the invalid Control-page value, the affected Loss field, and the STOP effect. The new settings_error uses ex…
No Resource Leaks ✅ Passed No new resource leak is evident. beantester/gui/crash.py stores one raw-form dict per live App in a WeakKeyDictionary, replacing it on each tick. crashlog._seen remains bounded by `MAX_RECORDS…
Scope, Duplication And Docs ✅ Passed The PR scope matches the title and description. The five changed files contain the crash-lock fix, GUI plain-data context handling, tests, mutation checks, and the related CHANGELOG entry. The GUI reu…

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added bug Something isn't working performance labels Sep 28, 2026
@donislawdev
donislawdev merged commit 7e47f4a into master Sep 28, 2026
15 checks passed
@donislawdev
donislawdev deleted the fix/crashlog-no-deadlock branch September 28, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant