Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ pip install -r requirements-dev.txt
`src/pytest_fly/__main__.py` → `main.py` initializes the stdlib-based logger (`logger.py`) and launches the Qt app.

### GUI layer (`src/pytest_fly/gui/`)
- `gui_main.py` — `FlyAppMainWindow`: 7-tab Qt window with a periodic timer (default 3 s) that pulls updates from the runner and refreshes all tabs.
- Tabs: `run_tab/` (run/stop controls, status, system metrics, failed tests, live output), `graph_tab/` (time-based progress chart), `table_tab/` (per-test status grid), `coverage_tab/` (coverage-over-time chart), `log_tab/` (live application event log — admission-gate, resource-guard, and stall-watchdog events, each line date/time-prefixed; default view shows tagged `EVENT_EXTRA` events + warnings, Verbose shows all INFO+), `configuration_tab/` (parallelism, thresholds, gates), `about_tab/`.
- `gui_main.py` — `FlyAppMainWindow`: 8-tab Qt window with a periodic timer (default 3 s) that pulls updates from the runner and refreshes all tabs.
- Tabs: `run_tab/` (run/stop controls, status, system metrics, failed tests, live output), `graph_tab/` (time-based progress chart), `table_tab/` (per-test status grid), `coverage_tab/` (coverage-over-time chart), `history_tab/` (recent-run summaries — run times, pass/fail statistics, failed-test lists; run count set by the History Run Limit preference), `log_tab/` (live application event log — admission-gate, resource-guard, and stall-watchdog events, each line date/time-prefixed; default view shows tagged `EVENT_EXTRA` events + warnings, Verbose shows all INFO+), `configuration_tab/` (parallelism, thresholds, gates), `about_tab/`.

### Core runner (`src/pytest_fly/pytest_runner/`)
- `pytest_runner.py` — `PytestRunner` (thread): orchestrates worker threads, schedules tests, handles run modes.
Expand Down
11 changes: 9 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ python -m pytest_fly

## Features

- Real-time monitoring of test execution in a GUI with seven tabs:
- Real-time monitoring of test execution in a GUI with eight tabs:
- **Run** — run controls and live panels:
- Controls: **Run**, **Stop** (waits for the running tests to finish and, while pending,
becomes **Cancel Stop** so the stop can be called off and the queued tests keep running),
Expand All @@ -48,14 +48,17 @@ python -m pytest_fly
- **Graph** — time-based progress chart showing each test module as a horizontal bar
- **Table** — per-test status grid with elapsed time, peak CPU, memory usage, and individual coverage
- **Coverage** — line chart of combined code coverage over time with covered/total line counts
- **History** — summaries of recent runs, most recent first: start time, duration, completion
status, pass/fail statistics, and each run's failed tests as expandable rows. The number of
runs shown is configurable
- **Log** — live application event log, each line date/time-prefixed. The default view shows
only notable run events (admission-gate deferrals, resource-guard and stall-watchdog
triggers, force stops) plus all warnings; a **Verbose** checkbox shows every log line.
The Verbose and Follow-tail selections persist across sessions, and the retained line
count is configurable
- **Configuration** — Resume-vs-Check toggle, a reorderable test-ordering aspect list, process
count, refresh rate, utilization thresholds, tooltip line limit, system-metrics chart window,
Progress Graph font size, Log tab line limit, target project path (applies on the next run),
Progress Graph font size, Log tab line limit, History run limit, target project path (applies on the next run),
test-results DB directory, a Liveness / Recovery group (stall watchdog with optional
automatic force-stop), an Admission Gates group (process-count / commit-charge / CPU
dispatch throttles), a Resource Guard group (low-resource automatic soft stop with
Expand Down Expand Up @@ -125,6 +128,10 @@ takes effect on the next run (no relaunch). See [Choosing Which Tests Run](#choo

![Coverage tab](https://raw.githubusercontent.com/jamesabel/pytest-fly/master/docs/images/coverage.png)

### History

![History tab](https://raw.githubusercontent.com/jamesabel/pytest-fly/master/docs/images/history.png)

### Log

![Log tab](https://raw.githubusercontent.com/jamesabel/pytest-fly/master/docs/images/log.png)
Expand Down
Binary file modified docs/images/about.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/configuration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/coverage.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/graph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/history.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/log.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/run.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/run_animation.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/table.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[project]
name = "pytest-fly"
description = "pytest runner and observer"
version = "0.7.4"
version = "0.8.0"
readme = "README.md"
requires-python = ">=3.12"
authors = [
Expand Down
3 changes: 2 additions & 1 deletion scripts/capture_assets.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
docs/images/graph.png
docs/images/table.png
docs/images/coverage.png
docs/images/history.png
docs/images/log.png
docs/images/configuration.png
docs/images/about.png
Expand Down Expand Up @@ -48,7 +49,7 @@
POST_RUN_SETTLE_DELAY_MS = 1500
RUN_TRIGGER_DELAY_MS = 800

TAB_FILENAMES = ["run", "graph", "table", "coverage", "log", "configuration", "about"]
TAB_FILENAMES = ["run", "graph", "table", "coverage", "history", "log", "configuration", "about"]


def qpixmap_to_pil(pixmap: QPixmap) -> Image.Image:
Expand Down
41 changes: 41 additions & 0 deletions src/pytest_fly/db/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,20 @@ def _query_last_pass(execute_fn: _ExecuteFn) -> dict[str, tuple[float, float]]:
return result


def _query_recent_run_guids(execute_fn: _ExecuteFn, limit: int) -> list[str]:
"""Return the *limit* most recent run GUIDs, newest first.

Recency ordering relies on run GUIDs being UUIDv7 (time-ordered; see
:func:`pytest_fly.guid.generate_uuid`), the same rule ``run_guid=None`` queries use.
"""
statement = f"SELECT DISTINCT run_guid FROM {_TABLE_NAME} ORDER BY run_guid DESC LIMIT ?"
try:
return [row[0] for row in execute_fn(statement, [limit])]
except sqlite3.OperationalError as e:
log.debug(f"query_recent_run_guids failed (table may not exist yet): {e}")
return []


def _query_ever_run_names(execute_fn: _ExecuteFn) -> set[str]:
"""Return the set of test node_ids that have ever been run, across all runs and PUT versions.

Expand Down Expand Up @@ -374,3 +388,30 @@ def query_last_pass(self) -> dict[str, tuple[float, float]]:
def query_ever_run_names(self) -> set[str]:
"""Return the set of test node_ids that have ever been run, across all runs and PUT versions."""
return _query_ever_run_names(self._execute)

def query_recent_runs(self, limit: int) -> list[PytestProcessInfo]:
"""Return the records of the *limit* most recent runs, with the ``output`` column omitted.

Used by the History tab; group the result by ``run_guid`` for per-run views.

:param limit: Maximum number of distinct runs to include.
:return: The runs' records (``output=None``); empty when there are no runs yet.
"""
result: list[PytestProcessInfo] = []
for run_guid in _query_recent_run_guids(self._execute, limit):
result.extend(_query_records(self._execute, self._columns, run_guid, include_output=False))
return result

def query_change_token(self) -> tuple[int, int]:
"""Return a cheap ``(row_count, max_rowid)`` token that changes whenever the table's content changes.

Lets per-tick consumers (the History tab) skip re-querying and rebuilding when
nothing was written since the previous tick. ``COUNT(*)`` runs off an index scan and
``MAX(rowid)`` is an O(log n) b-tree seek, so this stays cheap even with large output
blobs in the table.
"""
rows = self._execute(f"SELECT COUNT(*), MAX(rowid) FROM {_TABLE_NAME}")
if not rows:
return (0, 0)
count, max_rowid = rows[0]
return (count or 0, max_rowid or 0)
20 changes: 20 additions & 0 deletions src/pytest_fly/gui/configuration_tab/configuration.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
get_active_put_path,
get_pref,
graph_font_size_default,
history_run_limit_default,
log_tab_line_limit_default,
max_descendant_processes_default,
process_count_gate_enabled_default,
Expand All @@ -71,6 +72,7 @@
minimum_chart_window_minutes = 0.5
minimum_graph_font_size = 6
minimum_log_tab_line_limit = 100
minimum_history_run_limit = 1


def _add_labeled_lineedit(
Expand Down Expand Up @@ -365,6 +367,19 @@ def __init__(self):

layout.addWidget(QLabel("")) # space

history_run_limit_label = f"History Run Limit (min {minimum_history_run_limit}, {history_run_limit_default} default)"
self.history_run_limit_lineedit = _add_labeled_lineedit(
layout,
history_run_limit_label,
str(pref.history_run_limit),
QIntValidator(),
self.update_history_run_limit,
char_width=6,
tooltip="Number of recent test runs summarized in the History tab. Applies on the next refresh tick.",
)

layout.addWidget(QLabel("")) # space

# Target project path (PUT). Stored as a preference (independent of where pytest-fly keeps
# its own data), so it is freely editable here and takes effect on the next test run.
self._active_put_path = str(get_active_put_path())
Expand Down Expand Up @@ -864,6 +879,10 @@ def update_log_tab_line_limit(self, value: str):
"""Persist the Log tab line limit (clamped to *minimum_log_tab_line_limit*)."""
self._set_int_pref("log_tab_line_limit", value, minimum=minimum_log_tab_line_limit)

def update_history_run_limit(self, value: str):
"""Persist the History tab run limit (clamped to *minimum_history_run_limit*)."""
self._set_int_pref("history_run_limit", value, minimum=minimum_history_run_limit)

def restore_defaults(self):
"""Ask for confirmation, then reset every Configuration-tab setting to its default."""
response = QMessageBox.question(
Expand Down Expand Up @@ -914,6 +933,7 @@ def _apply_defaults(self) -> None:
("chart_window_minutes", self.chart_window_minutes_lineedit, chart_window_minutes_default),
("graph_font_size", self.graph_font_size_lineedit, graph_font_size_default),
("log_tab_line_limit", self.log_tab_line_limit_lineedit, log_tab_line_limit_default),
("history_run_limit", self.history_run_limit_lineedit, history_run_limit_default),
("cpu_active_epsilon", self.cpu_active_epsilon_lineedit, cpu_active_epsilon_default),
("max_descendant_processes", self.max_descendant_processes_lineedit, max_descendant_processes_default),
("commit_gate_threshold", self.commit_gate_threshold_lineedit, commit_gate_threshold_default),
Expand Down
8 changes: 7 additions & 1 deletion src/pytest_fly/gui/gui_main.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
from .coverage_tracker import CoverageTracker
from .graph_tab import GraphTab
from .gui_util import PhaseTimer, get_font, get_text_dimensions, qt_state_from_hex, qt_state_to_hex
from .history_tab import HistoryTab
from .log_tab import LogTab
from .run_tab import RunTab
from .table_tab import TableTab
Expand All @@ -47,7 +48,7 @@


class FlyAppMainWindow(QMainWindow):
"""Top-level application window containing the seven main tabs."""
"""Top-level application window containing the eight main tabs."""

def __init__(self, data_dir: Path):
self.data_dir = data_dir
Expand Down Expand Up @@ -101,13 +102,15 @@ def __init__(self, data_dir: Path):
self.graph_tab = GraphTab()
self.table_tab = TableTab(self.data_dir)
self.coverage_tab = CoverageTab(self.data_dir)
self.history_tab = HistoryTab()
self.log_tab = LogTab()
self.configuration = Configuration()
self.about = About(self, self.data_dir)
self.tab_widget.addTab(self.run_tab, "Run")
self.tab_widget.addTab(self.graph_tab, "Graph")
self.tab_widget.addTab(self.table_tab, "Table")
self.tab_widget.addTab(self.coverage_tab, "Coverage")
self.tab_widget.addTab(self.history_tab, "History")
self.tab_widget.addTab(self.log_tab, "Log")
self.tab_widget.addTab(self.configuration, "Configuration")
self.tab_widget.addTab(self.about, "About")
Expand Down Expand Up @@ -279,6 +282,9 @@ def _update_tick(self):
self._last_pass_cache_run_guid = run_guid
self._last_pass_cache_pass_count = pass_count
last_pass_data = self._last_pass_cache
with timer.time("db_history"):
# No-op unless the DB content (or the configured run limit) changed since the last tick.
self.history_tab.update_tick(db)

control = self.run_tab.control_window
with timer.time("build"):
Expand Down
1 change: 1 addition & 0 deletions src/pytest_fly/gui/history_tab/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .history_tab import HistoryTab as HistoryTab
105 changes: 105 additions & 0 deletions src/pytest_fly/gui/history_tab/history_tab.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
"""
History tab — summaries of recent test runs.

Shows one row per run (start time, duration, completion status, pass/fail statistics, and
the program-under-test version), with the run's failed tests as expandable child rows.
The number of runs shown is the Configuration tab's "History Run Limit" preference.
"""

from datetime import datetime

from PySide6.QtCore import Qt
from PySide6.QtWidgets import QGroupBox, QSizePolicy, QTreeWidget, QTreeWidgetItem, QVBoxLayout

from ...colors import TABLE_COLORS
from ...db import PytestProcessInfoReader
from ...interfaces import PytestRunnerState
from ...preferences import get_pref
from ...run_history import RunHistorySummary, build_run_history
from ..gui_util import format_runtime

_COLUMNS = ("Start", "Duration", "Status", "Pass", "Fail", "Other", "Total", "Version")
_START_COLUMN, _DURATION_COLUMN, _STATUS_COLUMN, _PASS_COLUMN, _FAIL_COLUMN, _OTHER_COLUMN, _TOTAL_COLUMN, _VERSION_COLUMN = range(len(_COLUMNS))

# Run GUID stored on each top-level item so expansion state survives rebuilds.
_RUN_GUID_ROLE = Qt.ItemDataRole.UserRole


class HistoryTab(QGroupBox):
"""Tab displaying per-run summaries of recent test runs, most recent first."""

def __init__(self):
super().__init__()
self.setTitle("Run History (most recent first)")
self.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Expanding)

layout = QVBoxLayout()
self.setLayout(layout)

self._tree = QTreeWidget()
self._tree.setHeaderLabels(list(_COLUMNS))
self._tree.setToolTip(
"One row per recent test run; expand a run to see its failed tests.\n"
"Other = terminated, stopped, or still queued/running tests.\n"
"The number of runs shown is set by the Configuration tab's History Run Limit."
)
layout.addWidget(self._tree)

# Change-detection state: rebuild only when the DB content or the run limit changed.
self._change_token: tuple[int, int] | None = None
self._run_limit: int | None = None

def update_tick(self, db: PytestProcessInfoReader) -> None:
"""Refresh the run summaries from the DB; a no-op when nothing changed since the last tick."""
run_limit = max(get_pref().history_run_limit, 1)
change_token = db.query_change_token()
if change_token == self._change_token and run_limit == self._run_limit:
return
self._change_token = change_token
self._run_limit = run_limit
self._rebuild(build_run_history(db.query_recent_runs(run_limit)))

def _rebuild(self, summaries: list[RunHistorySummary]) -> None:
"""Repopulate the tree, preserving each still-present run's expansion state."""
expansion_by_guid: dict[str, bool] = {}
for index in range(self._tree.topLevelItemCount()):
item = self._tree.topLevelItem(index)
# Only a row with failed-test children has a meaningful expansion state to keep.
# Recording childless rows would freeze an in-progress run in its initial collapsed
# state and defeat the auto-expand when its first failure appears.
if item is not None and item.childCount() > 0:
expansion_by_guid[item.data(0, _RUN_GUID_ROLE)] = item.isExpanded()

self._tree.clear()
fail_color = TABLE_COLORS[PytestRunnerState.FAIL]
pass_color = TABLE_COLORS[PytestRunnerState.PASS]
for summary in summaries:
texts = [""] * len(_COLUMNS)
texts[_START_COLUMN] = datetime.fromtimestamp(summary.start_ts).strftime("%Y-%m-%d %H:%M:%S")
texts[_DURATION_COLUMN] = format_runtime(summary.duration)
texts[_STATUS_COLUMN] = "Complete" if summary.is_complete else "In progress"
texts[_PASS_COLUMN] = str(summary.n_pass)
texts[_FAIL_COLUMN] = str(summary.n_fail)
texts[_OTHER_COLUMN] = str(summary.n_other)
texts[_TOTAL_COLUMN] = str(summary.n_total)
texts[_VERSION_COLUMN] = summary.put_version
run_item = QTreeWidgetItem(texts)
run_item.setData(0, _RUN_GUID_ROLE, summary.run_guid)
if summary.n_pass > 0:
run_item.setForeground(_PASS_COLUMN, pass_color)
if summary.n_fail > 0:
run_item.setForeground(_FAIL_COLUMN, fail_color)
for failed_test in summary.failed_tests:
failed_item = QTreeWidgetItem([failed_test])
failed_item.setForeground(0, fail_color)
run_item.addChild(failed_item)
self._tree.addTopLevelItem(run_item)
# Runs with failures start expanded so the failed tests are immediately visible;
# a run the user explicitly collapsed (or expanded) stays that way across rebuilds.
run_item.setExpanded(expansion_by_guid.get(summary.run_guid, summary.n_fail > 0))
# A failed-test child row is a single name, not tabular data — span it across all columns.
for child_index in range(run_item.childCount()):
run_item.child(child_index).setFirstColumnSpanned(True)

for column in range(len(_COLUMNS)):
self._tree.resizeColumnToContents(column)
3 changes: 3 additions & 0 deletions src/pytest_fly/preferences.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
chart_window_minutes_default = 5.0 # width of the system-metrics chart time window on the Run tab, in minutes
graph_font_size_default = 10 # point size of the font used in the Progress Graph tab
log_tab_line_limit_default = 10_000 # max lines retained/displayed in the Log tab — bounds memory over a long session
history_run_limit_default = 10 # number of recent runs summarized in the History tab

# Time-duration units offered for the stall timeouts. Stored as a (value, unit) pair so the
# user can express a timeout in whichever unit reads best; converted to seconds for the runner.
Expand Down Expand Up @@ -158,6 +159,8 @@ class FlyPreferences(Pref):
log_tab_follow_tail: bool = attrib(default=True) # keep the Log tab scrolled to the newest line
log_tab_line_limit: int = attrib(default=log_tab_line_limit_default) # max lines retained/displayed in the Log tab

history_run_limit: int = attrib(default=history_run_limit_default) # number of recent runs summarized in the History tab

# Wall-clock start of the most recent run; the Progress Graph time-axis origin, restored on
# restart so RESUME-carried records still shift onto the run timeline (0.0 = none).
last_run_start: float = attrib(default=0.0)
Expand Down
Loading
Loading