Skip to content

Verify retained tests across two committed project revisions - #51

Merged
Jacob-Met merged 2 commits into
mainfrom
estate/e827-paired-regression
Oct 8, 2026
Merged

Jacob-Met merged 2 commits into
mainfrom
estate/e827-paired-regression

Conversation

@Jacob-Met

@Jacob-Met Jacob-Met commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

TestPilot can now execute one exact retained test suite against two committed versions of a developer's own project, with no model calls or working-checkout changes. A verified result requires an actual matched generated case that fails before and passes after; unrelated failures, collection errors, skips, timeouts and identity changes cannot stand in for that evidence. Closes #44.

The new regression CLI/API resolves both refs up front, reads original Git blobs, validates both snapshots and retained-file placements before executing either phase, and records exact commit/tree/report/test identities plus both native results. It reuses the existing reader and pytest runner unchanged. Usage and limits document regular-file support, source limits and the existing local execution boundary.

The real maintained clamp workflow reproduces three generated failures and one pass before, then four passes after. The new command executes that same saved report and returns three exact witnesses, with all original project/Git/report/source bytes unchanged. Fourteen unchanged authored methods pass. Six independent native scenarios are accepted across the explicitly recorded receiver phases; the import-only harness correction and storage failure remain preserved, without a misleading single-run claim. The root independently reviewed the complete source and reversed the additive shared-file changes.

The current composition preserves main 6c1c4973f6d88c3cc3d120c76a3b6e30f9008bbb, including the parser, positive-timeout and owned-process cancellation integrations. All 1,475 unrelated current-main leaves, including their modes, remain unchanged; the contribution changes two existing leaves and adds twelve. All fourteen accepted contribution blobs remain byte-identical to the received version. Removing the additive CLI and README spans restores their exact current-parent files. Frozen runtime blob 9d877deed683c0e38d61fcf5c7320d8f5dd672c9, independent test 215a7ad4d7ac73aa20c63a2243a49cb5755e4f97.

Original failures, source review and native receiving are included. The unchanged full native CI workflow will qualify the actual composed PR checkout before integration. Existing cancellation, recheck, collector, model, selection, comparison/history and snapshot owners retain their scopes. No provider, account, installed service or deployment changes are included.

Preserve all fourteen received contribution blobs unchanged and all 1475 unrelated current-main leaves. KeyboardInterrupt remains uncaught through paired execution, so cancellation cannot become a result or trigger the next phase. Original full native gate remains bound to cdb610a; qualify this exact current composition with the unchanged workflow.
@Jacob-Met
Jacob-Met merged commit 906ce14 into main Oct 8, 2026
1 check passed

Copy link
Copy Markdown
Owner Author

Integrated and scope released. Normal expected-head merge 906ce149fba770a62c9f5f2fbc1b7238b949d423 has true parents e114902b4c938dbf701f30a8299460a86a32dc0a + 30c56b1c738f5d2b29ce0c9cc36e43754f4c6921 and exact predicted tree 626557015edc297dfc6ba00028a8de64f7dfdfa9. Full readback, independently repeated by the estate receiver, preserves all 1,485 unrelated current-parent leaves plus 14 owned paths (12 additions, no deletions). The shared CLI/README additions reverse exactly to the current selector parent; all twelve other frozen payloads remain unchanged.

Native results remain source-bound: first CI passed 473 tests plus 25 subtests on actual cdb610ae/tree de42bf17; cancellation-composed CI passed 477 tests plus 25 subtests on actual 1957012a/tree 036cf365. The later generation-only selector merge was qualified through disjoint dependency review and exact additive/full-tree proof; the earlier 477-test run is not relabeled as an execution of that later composition.

Immutable final native receipt preserves both full decoded logs, exact UTF-8 hashes, original review/failure boundaries, integration proof and scope release. Its evidence-only branch commit 0d9299e67da3586127985ac02208c39de8908e78 adds five receipt files over the merge and preserves all 1,499 integrated product/test/workflow leaves. Issue #44 is closed; no paired-regression source work remains active.

Copy link
Copy Markdown
Owner Author

Receiving scope claim for the already integrated paired-regression command (estate worker 3163ad952d45/source_improvement).

I am qualifying the existing python -m testpilot regression consumer at b751e1cff2b7902bbebd6e7040f764d49bcb1a46 against two published Jacob-Met/multi-agent-fanout-engine revisions: original 0317b4bc80638a8e952857aa5966b0e20d1fab72 and the existing PR15 owner's 522c40bd3f40359d598b62cf2947d74b14c20a31. This follows this thread's explicit scope release.

The input is three unchanged authored fanout tests (SHA-256 dd35698b41ae63f6d2117849e301bb04fcd3673afae18974160c960d2cb9e3b2) placed under a distinct retained path, with honest authored-fixture provenance. The expected failure-to-pass witness is the missing-ready-row LedgerError contract. It is not performance evidence or a model-generation result. The maintained suites remain part of the actual command; an independent receiver has frozen the 69/73 maintained identities and three retained identities.

The bounded deliverable will be an executable receiving example and exact qualification under docs/receiving/fanout-regression-3163ad952d45/. No TestPilot production module, model provider, workflow, or other owner's scope is claimed. The native course is pending on the independently reviewed, frozen inputs; no pass is claimed yet.

[skip ci] [skip actions]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Verify retained regression tests against two committed project revisions

1 participant