Repository navigation
test: nineteen #1339 entries in the harness self-tests, each measured red-first (#1339) - #1664
Merged
Merged
Conversation
… red-first (#1339) Fixes my a-m #1339 entries in test_layer.py, test_compare_to_bash.py, test_harness_deps.py, test_failed_query_sentinel.py and test_guards_pinned.py. Every entry was shown first: a planted bug the old test passed, a one-line change to the code under test that reddened a "premise:", or a docstring contradicted by the code. Then fixed, and the same plant now fails by name. Mislabelled premises (12), each reddened by a one-line change to what it reads, so each loses the prefix: - test_layer :1013 :1037 :1067 :1098 :1166 -- the layer's own refusals switched off or inverted (nocollapse, nobroad, noraises, numpass, numfail). - test_compare_to_bash :319 (three: loopnone, firstword, nofstr), :412 (noexpand). - test_harness_deps :771 (allbound, allfree); :1122 re-opened: its "premise:" was a control on the test's own detector (inst_empty), now "control:". - test_failed_query_sentinel :324 (mintother), test_guards_pinned :373 (noemptyplan: the check is what catches the empty-plan refusal being removed). Weak or vacuous arms (6): - test_layer :837: the inequality scan read positional arguments only; a planted `expect.num(got=int(a != b), ...)` passed the population arm. Keywords are read. - test_compare_to_bash :468: the constant refusal was overdetermined (the fixture also interpolates `col`); admitting constants (constok) stayed green. A constant-only fixture now reddens by name. - test_harness_deps :782: at_least(2) passed an invented root (everyfix); the derived set is asserted exactly. :964: "pip show psycopg" was counted as text, passing `|| true` and a comment; now a code line that can fail. :1007: a count over the file passed the guard job's flag moved into a comment; now per job. - test_failed_query_sentinel :186: an excluded method's reason exists by construction; replaced by "no dead exemption" (a planted unused kind reddens). Inaccurate text (3): test_compare_to_bash :1174 (one suite holds all three helpers, not one each) and :1455 (#1046 landed; dropping a stem from COMPLETE reddens two arms), test_guards_pinned :1 (it pins 17 guards, not every refusal). Also: outcomes() now names its check when it fails (it raised pytest's bare dict comparison), and a stale citation in test_harness_deps names the test file the closure arm actually lives in. Guard set 467 passed (guard_tests unchanged); no ledger rows (pytest-only names). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jdatcmd
approved these changes
Oct 22, 2026
jdatcmd
left a comment
Collaborator
There was a problem hiding this comment.
Approved at 9c86065. I reran everyfix (every conftest fixture taken as a root, in cluster_fixtures) against both versions of test_harness_deps.py:
- main: the root-fixture test passes;
- this head: it fails with "conftest's cluster-reaching fixtures are derived exactly: got 'pgc_cluster, pgc_conn, pgc_own_db, server_dir' want 'pgc_cluster, pgc_conn, pgc_own_db'".
Clean at this head, the five touched files with --pg-config: 152 passed. CI 15/15.
Note: pinning the exact set means a new connecting fixture in conftest now reddens this check by name. The old docstring said that needed no edit here. Your comment states the trade, and I agree with it.
The test_ordered.py collapse premise you left for me is relabelled in my #1667.
87 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@jdatcmd Nineteen of my a–m #1339 entries, in the harness self-tests. No cluster, no ledger rows (the names are pytest-only), and
guard_testsis unchanged at 474.Method: every entry was measured first, then fixed and re-measured.
premise:, or a docstring the code contradicts.Mutants are in
~/pgc-tools/i1339h/mut.pyon my side;run.sh TAG NODE…applies one, runs, and restores.test_layer.py:1013nocollapse(the order-collapse scan never applied) reddenspremise: the collapse is refused…test_layer.py:1037nobroadreddenspremise: the broad except is refusedtest_layer.py:1067noraisesreddenspremise: an unpinned compound raises block is refusedtest_layer.py:1098numpass(num never fails) reddenspremise: a false claim fails…, unnamed:assert {'passed': 1…} == {…}outcomes()now names its check on failuretest_layer.py:1166numfail(num fails on equal) reddenspremise: an honest test passes, unnamedtest_layer.py:837expect.num(got=int(a != b), want=1, name=…)in the corpus: 1 passedno test hand-rolls an inequality: got "['test_zz_plant_kwineq.py:3']"; a planted-offence arm for the keyword formtest_compare_to_bash.py:319loopnone/firstword/nofstrredden the three loop-reader premisestest_compare_to_bash.py:412noexpandreddenspremise: with both members present…test_compare_to_bash.py:468constok(constants admitted to the expansion gate): all green. The fixture also interpolatescol, so the two-names rule refused it toogot "raised KeyError 'FLOAT_RTOL'" want '[]'test_compare_to_bash.py:1174parallel_copy.shtest_compare_to_bash.py:1455advisory_lock_classfromCOMPLETEreddens two armstest_harness_deps.py:771allbound/allfreeredden the two partition premisestest_harness_deps.py:782everyfix(every conftest fixture a root):at_least(2)passedgot 'pgc_cluster, pgc_conn, pgc_own_db, server_dir'test_harness_deps.py:964pip show psycopg … || trueand a comment-only mention: both passedtest_harness_deps.py:1007test_harness_deps.py:1122inst_emptyreddens itspremise:, which is a control on the test's own detectorcontrol:test_failed_query_sentinel.py:186frame: passedevery exclusion kind is the first parameter of a public method, so none is a dead exemption: got 'frame'test_failed_query_sentinel.py:324mintother(query_error mints another prefix) reddenspremise: minted under the real prefixtest_guards_pinned.py:373noemptyplan(the empty-plan refusal disabled) reddenspremise: all three were found…, so it is the check that catches the regressiontest_guards_pinned.py:1Also:
outcomes()failed with pytest's bare dict comparison. It now prefixes the check name, so the corpus no longer has one assertion that fails without saying what.partition()cited a test that doesn't exist in that file. It now namestest_harness_deps_classifier.py::…. Undernoclose, that test is what reddens.Left:
premise: the collapse is refused when nobody rebindsexists intest_ordered.py, which is in your half.test_mutation_ledger.py's 13 entries are their own PR.Gates at
9c860657: guards 474 passed; gate rc=0; self-test PASSED.🤖 Generated with Claude Code