Skip to content

Stop owned pytest processes when a TestPilot run is interrupted #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jacob-Met merged 2 commits into from
Oct 8, 2026
Merged

Stop owned pytest processes when a TestPilot run is interrupted #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
95a3116
Stop the owned pytest group when a run is interrupted
Jacob-Met Oct 8, 2026
3798534
Keep complete process identity in cancellation tests
Jacob-Met Oct 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions docs/receipts/run-cancellation-18a24bf0c281/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# TestPilot cancellation receiving

This repair stops the launched POSIX process group when KeyboardInterrupt interrupts the sandbox's active subprocess communication. It reaps the leader with a bounded wait and propagates the interrupt. The original CLI returned while pytest continued performing fixture work after that return; the candidate stops that work and retains ordinary successful runs.

Tracked scope is [issue #35](https://github.com/Jacob-Met/testpilot/issues/35). Root owns GitHub integration and publication. The source proposal changes only `testpilot/sandbox.py::_run`, adds `tests/test_sandbox_interrupt.py` and `docs/run-cancellation.md`, and carries this additive evidence.

## Source identity

| Role | Exact identity |
| --- | --- |
| Original discovery source, actually executed | main `f3b2135cad31852b599350564fe76dd160a6a522`, tree `d0107a6f8b3b8cc00dadebc5f794060f275f3f5b` |
| Current maintained parent | main `7f44c4f134f4d56d37dd14c7c2d2aaa3c8c8749e`, tree `ea3f7b4170f1515abb7188d31dc269623b9e0738` |
| Original and current sandbox blob | `8ca4a4102c7890e89eae99548e28ba8325974f1e` |
| Frozen candidate native source commit | `649dc7d1fd02493af298876eea0697849404bba9` |
| Candidate sandbox blob | `84330f96ebd71b37dc40575fa2ac59ba53cf2a72` |
| Candidate sandbox SHA256 | `0adf5eb81f8fd15ca09170c5328e6ff4165e1a0f903221b9730de3b480be0b1b` |

The original native source capture has 12 files. The separate current-parent capture has 14: the complete nine-module TestPilot package, its configuration/README/workflow, and two existing sandbox test files. The current canonical tree has 1,063 blobs; these receiving captures are not a full upstream clone. The publication packet is a delta and must be applied while preserving that full parent tree.

Output-staging PR34 changed `loop.py` and README between the original and current captures. The entire sandbox remained byte-identical. Current `loop.py` blob `ed0f5f7f057c217b931a1756bd679ae087247e0a` and every other received current-parent file remain unchanged by this repair.

The production patch adds 15 handler lines and changes its docstring. Removing that handler and restoring the docstring reconstructs the entire original sandbox byte-for-byte, including the existing timeout branch. Candidate freeze records all 16 source/context/test/documentation inputs before execution.

## Native results

All runs below used the existing native Mac environment: macOS 26.6.2 arm64, Python 3.12.8, pytest 9.1.1. No package was installed.

| Receiving gate | Original outcome | Frozen candidate outcome |
| --- | --- | --- |
| Actual CLI, interrupted baseline pytest plus uninterrupted ScriptedModel control | Two CLI actions on f3: three of four predicates pass; pytest performs work released only after its parent exits | Same frozen driver on current7f + repair: four of four predicates pass |
| Focused real-process interruption tests | Four cases on exact current7f: three fail and one detached-session control passes | All four pass |
| Existing sandbox and timeout controls | Preserved source; no duplicate baseline run | Thirteen pass; one existing coverage test skips because coverage is absent |

The candidate combined pytest gate is **17 passed, 1 skipped**, with no test errors. The actual CLI gate independently confirms no interrupted output publication, no post-parent-exit pytest work, successful ordinary execution, and unchanged source bytes. Its interrupted caller exits with -2 in about 0.290 seconds. The ordinary control produces two passing pytest cases and two accepted ScriptedModel entries.

These are four focused pytest cases, fourteen existing pytest cases, and two CLI actions with four predicates; the predicates are not four independent CLI tests.

The focused process observations before fixture cleanup are:

| Fixture | Current baseline | Candidate |
| --- | --- | --- |
| Live leader, same-group child | Leader and child still running; leader not reaped | Both stopped; leader reaped |
| Already-exited leader, same-group child | Leader reaped; child still running | Child stopped; leader remains reaped |
| Live leader, detached child holding stdout | Leader and child still running; leader not reaped | Leader stopped/reaped; detached child still running |
| Already-exited leader, detached child holding stdout | Leader reaped; detached child still running | Same explicit detached-session boundary; caller returns promptly |

The receiver subsequently stops its own detached helpers. New fixtures bind signals to the unique worker script path and recorded process group, provide eight-second worker lifetimes, and have a ten-second fallback cleanup deadline. All completed focused tests verified that their owned fixture processes were no longer running at cleanup completion. The CLI receiver separately records owned pytest-group shutdown.

## Preserved failures and provenance details

The first CLI receiver actually completed both native flows, but aggregate metadata construction read `ledger.calls` instead of the maintained `ledger.entries` field. Its raw KeyError, successful ordinary report, cancellation continuation, and source bytes remain in the raw archive. It is not counted as a clean aggregate run.

The separately frozen v2 receiver corrects that key and exposes input/output/interpreter arguments. Its fixture and cancellation boundary are unchanged. Driver SHA256 is `e80ecd44e02f6bc32c0994bc1665c8ed21c86fa6de266f81a2a0043d0d72b16c`. The original clean negative receipt SHA256 is `2f8ee8748971ac121032f8db75746bd89e523c48fecf3815e4aa1329c13d98b8`.

The unchanged v2 driver's raw receipt contains historical hardcoded f3 head/tree labels. For the candidate run, **candidate-cli-composition.json is the authoritative current7f + repair binding**, with all actual source hashes. The raw candidate receipt is retained without rewriting those historical fields; it is not evidence that f3 was executed again.

The current-parent focused tests were frozen before the repair at `d017324348fba8035646b77426aa3b94497d6d89`, SHA256 `466bbdd470001a5e23e55a1205352dfccc8c714470fe903ff241b28416449325`. Their three failures and passing control were retained at `fe6adc337ada4421a1e978d58998eec3e8478d43` before the production source freeze.

## Artifacts and review boundaries

- `candidate-source-freeze.json`, `current-parent-source.json`, and `qualification.json` bind the source, environment, commands, outcomes, and ownership scope.
- `original-cli-receipt.json`, `current-focused-receipt.json`, `candidate-pytest-receipt.json`, and `candidate-cli-composition.json` preserve the separate stages.
- `production.patch` is the complete production-code delta.
- `receive-cancellation.py` is the unchanged real CLI receiver. The focused regression tests are published in the ordinary tests directory.
- `raw-native.tar.gz` contains every retained original/current/candidate input, raw gate output, fixture and receipt selected by `raw-manifest.json`. Every regular archive member is reopened and byte-verified. Pytest convenience symlinks are retained as target metadata in that manifest rather than emitted as absolute archive links.

The native signal receiving is scoped to KeyboardInterrupt during the sandbox's initial communication on POSIX. It does not qualify SIGTERM/SIGKILL handling, pre-Popen interruption, Windows behavior, escaped-session containment, optimized-Python execution, or the whole repository suite. Cleanup remains bounded if the OS refuses a signal or the leader cannot be reaped promptly; the original interrupt propagates. The sandbox remains process-level isolation. No live provider, developer repository, deployment, goal, service or other owner's source was used.

The existing timeout and normal-result contracts remain unchanged. An interrupted attempt does not reach the CLI's normal publication step; earlier test effects or model calls are not rolled back.
128 changes: 128 additions & 0 deletions docs/receipts/run-cancellation-18a24bf0c281/candidate-cli-composition.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
{
"source_commit": "649dc7d1fd02493af298876eea0697849404bba9",
"actual_canonical_parent": "7f44c4f134f4d56d37dd14c7c2d2aaa3c8c8749e",
"actual_canonical_tree": "ea3f7b4170f1515abb7188d31dc269623b9e0738",
"actual_source": "/Users/me/testpilot-cancellation-18a24bf0c281/candidate",
"source_freeze": [
{
"path": ".github/workflows/ci.yml",
"size": 586,
"sha256": "236fd600bea41471b373b91d8e389aa12c58cba79c83efa071f6ab805f1ebf59",
"git_blob": "86a3236e38155506b5fe6077322d508f7f676649"
},
{
"path": "README.md",
"size": 12669,
"sha256": "9e1c93aa93ca1942d002c3bd28779d780b568b606cfa7e001058c13997e34854",
"git_blob": "24a6c0fab34c05d17ef216a42e97a0fae2389fc2"
},
{
"path": "docs/run-cancellation.md",
"size": 2985,
"sha256": "ab37125e75d03005813477f6ec1703117ca844209a559f289a8be5ee720bf126",
"git_blob": "d8956abb39915e235531c3e217050ccc2089173d"
},
{
"path": "pyproject.toml",
"size": 676,
"sha256": "5503eab7b6c145d4de241e5be805a4fb8f30828053cbe39dc05a43c2896642eb",
"git_blob": "952255ac87e2c7da57457fdbf3554914f17f6d05"
},
{
"path": "testpilot/__init__.py",
"size": 390,
"sha256": "65c63b80d7da34a83a6049a79c24b8335878cd49cc79951fdefc5f1532ae841f",
"git_blob": "296210dd0d58703de4a4e4c9ad72a0944c9400ca"
},
{
"path": "testpilot/__main__.py",
"size": 6065,
"sha256": "90577ea3a939642f5cb29818f6360df96d9fa0d35d05af11bf97054425c32c84",
"git_blob": "2d98e2a85320ac2827ea48fa9d58d30a60354b6a"
},
{
"path": "testpilot/_pytest_generated.py",
"size": 2943,
"sha256": "d65352d7bade965d7b9dc73efe5d4d01844fa8c7474a1bdcdb3b708f6decf768",
"git_blob": "90b82725bc0eb1ff3d7cb517c4c9dedcfd14bdcc"
},
{
"path": "testpilot/diff.py",
"size": 9387,
"sha256": "2cf38707ea175bb1f04d55335de344ae943458658055332b4aa68f27763214c0",
"git_blob": "29dd325ab288ac1d43fd70f33b207281f5375d4b"
},
{
"path": "testpilot/html_report.py",
"size": 20656,
"sha256": "183703d05f32552f7f7c39aa958c39bae61d824e3fa9cd6a6dd686f0d4c228a4",
"git_blob": "bb30d903542d836f5afe8e116ba894f93d9d3a0c"
},
{
"path": "testpilot/loop.py",
"size": 25480,
"sha256": "dde4ee52e0a1fabded9902137ede6dd68aa17f6542c316e5d134b8510495170b",
"git_blob": "ed0f5f7f057c217b931a1756bd679ae087247e0a"
},
{
"path": "testpilot/model.py",
"size": 10175,
"sha256": "f6ae697f1f5d686ffdcb96f5451d1a9600d7d41c7bcf324d7e8c276a107e8673",
"git_blob": "6d4abca38cc1c256df00bcac7923e2dc0bd42b16"
},
{
"path": "testpilot/recheck.py",
"size": 11599,
"sha256": "1ecbe0ff7da7bb4a8117d730a76624c8929e0e4b10cb9d502ab7358cb38d8f2e",
"git_blob": "4ab26cbb83beff563b9c3683c593b3efd2495b6b"
},
{
"path": "testpilot/sandbox.py",
"size": 13277,
"sha256": "0adf5eb81f8fd15ca09170c5328e6ff4165e1a0f903221b9730de3b480be0b1b",
"git_blob": "84330f96ebd71b37dc40575fa2ac59ba53cf2a72"
},
{
"path": "tests/test_sandbox.py",
"size": 2545,
"sha256": "ee47d6a3f40c2f612197614088ea40cbdd50a7f04047780c95a71e37c1555992",
"git_blob": "de2fde7cd5dd683c338810d4707987ea5a2e9055"
},
{
"path": "tests/test_sandbox_deadline.py",
"size": 4248,
"sha256": "6a9abfdc79bcb2cc3102d94b6b61d20ea09b0f89d5c7c5842387ce77c136d7e1",
"git_blob": "d8ebce5a81205aad005e45a280aec898e3e89820"
},
{
"path": "tests/test_sandbox_interrupt.py",
"size": 6944,
"sha256": "466bbdd470001a5e23e55a1205352dfccc8c714470fe903ff241b28416449325",
"git_blob": "184e969e57fc2a483b6a4e7b1709e16f66ce37e6"
}
],
"raw_receiver_receipt": "/Users/me/testpilot-cancellation-18a24bf0c281/evidence/candidate-cli/receipt.json",
"raw_receiver_receipt_sha256": "dde39c5ceb688d4c04f165b77362e19b66247b5214cf7f289ff9b290a9fcb785",
"historical_label_notice": "The unchanged frozen receiver contains hardcoded canonical_head/source_tree fields naming its original f3 discovery source. They are historical labels, not the executed source identity for this candidate run. The actual executed current parent plus repair and all input hashes are explicitly bound here; raw receipt bytes are preserved.",
"argv": [
"/Users/me/capturesuite-independent-checkpoint-18a24bf0c281/venv/bin/python",
"/Users/me/testpilot-cancellation-18a24bf0c281/receiving/receive_cancellation.py",
"--source",
"/Users/me/testpilot-cancellation-18a24bf0c281/candidate",
"--output",
"/Users/me/testpilot-cancellation-18a24bf0c281/evidence/candidate-cli"
],
"exit_code": 0,
"elapsed_s": 3.4259277919773012,
"platform": "macOS-26.6.2-arm64-arm-64bit",
"python": "3.12.8 (main, Dec 3 2024, 18:42:41) [Clang 16.0.0 (clang-1600.0.26.4)]",
"driver_sha256": "e80ecd44e02f6bc32c0994bc1665c8ed21c86fa6de266f81a2a0043d0d72b16c",
"checks": {
"interrupted_cli_failed_without_publishing_result": true,
"interrupted_pytest_work_stops_before_parent_returns": true,
"ordinary_scripted_run_passes": true,
"all_original_source_bytes_unchanged": true
},
"source_unchanged": true,
"scope": "Two actual native CLI actions with the unchanged frozen receiver; no live provider or developer repository. Original f3 evidence remains separate. This receipt qualifies the exact current7f44 staging composition plus the cancellation repair."
}
Loading
Loading