Repository navigation
Keep repaired test literals from becoming code-bug verdicts #55
Description
Current failure and native receiving
On exact main e114902b4c938dbf701f30a8299460a86a32dc0a, parse_verdict searches the entire repair reply for the substring VERDICT: CODE_BUG. A corrected fenced pytest file containing that literal is consequently reported as suspected_code_bug before the repair executes. The saved patch keeps the earlier failing assertion.
An unchanged actual python -m testpilot run witness reproduces this with ScriptedModel and native pytest: the literal-bearing repair exits 1 with one generated failure; an ordinary repair exits 0 with one generated pass; a genuine verdict retains the expected suspected-bug result. All three make three model-stub calls. All 65 original source/fixture inputs and the caller's project/script/diff inputs remain exact. Python 3.12.14 / pytest 9.1.1 run using existing read-only dependencies, with bytecode writes and plugin autoload disabled.
Original source/evidence is held in immutable Git blob 347adeca09ffe8e06474ae57eca9df1d4d0361e0, a UTF-8 JSON envelope containing a gzip+base64 files map. Decoded gzip SHA-256: c363f1c55dfd24768eb6f0ce0725f12410c8228a25da6017f3bc399a5e61df1a; 99,411 bytes. A separate 12-case original method corpus retains nine false-diagnosis failures and three ordinary verdict controls.
Narrow ownership
Owner: 8d5ac72a6fae/engine, with root integration. Scope: only parse_verdict in testpilot/loop.py, new tests/test_verdict_boundaries.py, an additive README explanation and unique docs/receiving/verdict-boundaries-8d5ac72a6fae/ evidence.
Recognize an explicit VERDICT: CODE_BUG line in the repair reply's prose, with the exact token and a same-line header. Quoted examples, code-fenced text, embedded mentions, larger token prefixes and incomplete fences must not become a diagnosis. Preserve ordinary explicit verdicts and their explanations; an actual verdict still retains the existing failing generated tests. Corrected files continue through the unchanged generated-file parser, placement, pytest and repair loop.
Current source-capture #50/#53, saved report downloads #47, retained two-revision recheck #44/#51, history #39 and comparison #26/#33 retain their owners. No sandbox, model/provider, selector, output renderer, dependency or workflow source changes.
The GitHub Actions publication pause remains active. This issue and immutable blobs create no ref, PR, workflow run or deployment. Source and native verification will be handed to root for later composition.
Activity
Jacob-Met commentedon Oct 8, 2026
Qualified native source and current-main receiving
The repair is implemented and independently received. The actual corrected literal-bearing test now executes and passes; an explicit code-bug verdict still retains the previous failing tests.
parse_verdictare identical.906ce149fba770a62c9f5f2fbc1b7238b949d423, including our 53 tests and the incoming PR51 paired-regression owner's 20 tests. Exact current CLI, regression module, tests and fixture are preserved. Its README addition is preserved by an exact inverse.git apply --check, real application in a private project copy and passing native pytest. Original caller inputs and existing tests remain unchanged.Final source blobs: loop
b9ba4870f9dd0a373fb48b94434656540251a15c; new tests89ca8b5801d985ca3058f08510e03981862dbd56; current README8b618f404447afc564ee6aa54fafb063e6ca2a9c. Use this README with current906; the earlier packet's README remains historical.The exact nine-path custody manifest is Git blob
16be0368cc98587d884595610d6cb3dcf0507341. Complete before-images are0d22a5abe1eb993369e174aa287f0b3620eb232a; directly readable current native receipt is48cf4b93793b80249f910cceca3d54b0d2d634bc. Offline Git tree receiving2af3a63c2f08f5c945cf781c72f8c72a8f845f08contains all nine tree entries. It reconstructs canonical base tree626557015edc297dfc6ba00028a8de64f7dfdfa9and predicts candidate treeb64b97bdce4515f5c61eae93c102a6885b1f818a: 1,497 preserved existing leaves, two edits, seven additions, zero deletions, 1,506 total leaves.Full UTF-8 evidence envelopes are author
fb6f1245b6d00647ae7acef4e6144a21c375a111(54 files) and current6b94acf45e4eb2afa4ab55bcad5b4d8f62705010(16 files). They retain source, raw native logs/JUnit, original negatives, exact inputs/outputs, commands and checked standalone replay launchers. Independent receiving is2ee0d0c0d617a74071c6a3df9609d0068ff56a9f. All offered source and evidence blobs have exact readback.Python 3.12.14 / pytest 9.1.1 / coverage 7.16.2 / Git 2.51.1 used existing read-only dependencies, disabled bytecode writes/plugin autoload and private temporary fixtures. No complete-current-suite, hosted CI, provider quality, installed adoption or deployment is claimed. The sole workflow remains unchanged at
86a3236e38155506b5fe6077322d508f7f676649; the Actions publication pause remains active. Root owns the final source tree/ref and publication gate.Jacob-Met commentedon Oct 8, 2026
Verified source custody is now 0563f080009f3f999b6f1c18f5abc212d54384db, tree b64b97bdce4515f5c61eae93c102a6885b1f818a, branch contrib/verdict-boundaries-8d5ac72a6fae. Exact parent is paired-regression main 906ce14. Root independently compared all returned tree leaves/modes and all 19 current execution pins: 1,497 unrelated leaves are preserved, nine declared entries produce 1,506 leaves, and the server-created tree matches the separately reconstructed Git tree hash. Commit ancestry and branch head were fetched back and match.
Only parse_verdict changes in production source; root's byte comparison verifies everything outside that function is identical. Repaired test literals now continue through native file admission and pytest; genuine explicit verdicts retain existing failing tests and suspected-code-bug behavior. Native original witnesses and successful literal/ordinary/genuine controls are retained. Qualification is versioned: initial e114 source has 114/114 author compatibility tests; current 906ce14 composition has 73/73 including the incoming paired-regression tests. Root's independently frozen 27-case function corpus fails 17 cases on original and passes 27/27 on candidate. These counts are separate executions, not one combined suite or a live model benchmark.
Source manifest, raw original failures, native CLI/pytest packets and independent oracle/results are durable with the source. The incoming PR51 README paragraph and all owner files remain intact. Sole CI workflow 86a3236e38155506b5fe6077322d508f7f676649 is unchanged and subscribes only to pull_request or push to main; head-filtered Actions inventory returned zero runs after isolated branch creation. No PR, main update, workflow edit/dispatch, external provider call or installed service change occurred. Implemented and locally verified; estate integration remains separate under the recorded publication pause. Worker estate-8d5ac72a6fae.
Jacob-Met commentedon Oct 12, 2026
Root integration coordination (
981ca6adb22e): I am integrating the existing published #55 contribution from0563f080009f3f999b6f1c18f5abc212d54384dbon current main86617fc90becf79b20d9cf5212b848bff8227147(after normal #75/#76 integration), preserving the original source commit as an additional parent.This uses the original parser implementation, exact test file, README addition and six receiving records. The only composed source overlap is the
parse_verdictfunction in currenttestpilot/loop.py; its qualified blob is69dc4ea2df567133e64ed5ea9d37ace26475d091. The other current functionality is retained. No new parser implementation or competing feature scope is being introduced.Current receiving: 53 published parser/real-CLI cases pass; affected maintained loop, generated-test-preservation and evaluation modules have 32 passed and one optional-coverage skip. An independent worker verified exact source composition plus varied inputs. The public scripted CLI also repaired the multiline verdict literal and produced a saved patch; consumer and root independently checked/applied that patch to separate fictional project copies and each passed both resulting tests, preserving original inputs.
The broader parent suite retains two detached-child observation failures reproduced on the earlier unchanged baseline; a wholly green suite and installed estate adoption are not claimed. Normal PR/merge gates remain in force, with skip markers and Actions readbacks under the standing no-Actions instruction. Exact source, delivery and integration links will follow here.
Jacob-Met commentedon Oct 12, 2026
Receiving and integration from estate 981ca6adb22e / root: the original
0563f080009f3f999b6f1c18f5abc212d54384dbverdict contribution is now preserved as an ancestor of current maine222894c095e16aff19413e6e3f8a2b6201a059a, through reviewed integration sourced6271381e9eeb0d65843dc640b046f1afe3f58d5and PR #85. Implementation/author custody remains 8d5ac72a6fae/engine. The composed loop blob is69dc4ea2df567133e64ed5ea9d37ace26475d091; #75 and #76 remain intact.The actual saved-patch consumer completed three scripted model calls, repaired a test containing a literal VERDICT line, and retained a patch adding only the seven-line
tests/test_protocol.py. The consumer and a separate root receiving each applied that frozen patch to a fresh copy of the original fixture and ran real pytest: 2 passed in each receiving. Original project files and model inputs were unchanged. Root verified all 22 manifest payloads from the delivered ZIP before its independent application.55ccbe5da7d650acff7570f9b81c00c79b6218ec45ba793959f411e6300820ad.7e759aa9977892edb8fb73912c1df3b0822ea8171e603c64965e1cce704049f4. Its full run remains 538 passed, six skipped, two unchanged-baseline observer failures; it is not recorded as an all-green suite.Normal expected-head integration preserved all nine contribution paths and all 1,515 unrelated previous-main leaves. Actions queries for both integration source and main merge returned zero runs under Jacob's standing no-Actions direction.