Skip to content

test: validity_elision's guard arms ask which guard refused, not only that one did (#1339) - #1484

Merged
jdatcmd merged 2 commits into
mainfrom
test/1339-validity-elision-guard
Oct 16, 2026
Merged

jdatcmd merged 2 commits into
mainfrom
test/1339-validity-elision-guard

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Fixes two #1339 entries for test_validity_elision.py: :281 [weak] and :299 [inaccurate].

What was wrong

Both arms in test_the_guards_the_elision_added_refuse_a_lying_catalog asserted only SQLSTATE XX001, and sixteen guards in columnar_reader.c raise XX001.

I disabled the no-nulls guard ("claims no nulls but does not account for every row", columnar_reader.c). Both twins stayed green, 25/25 each:

arm what refused it with the guard off
a chunk claiming no bitmap while it holds fewer values than rows is refused "native chunk length does not match descriptor"
an implausible row count on an elided chunk is refused before anything is allocated "fixed-length value runs past the value stream end", raised after the 500 MB bitmap was allocated

The second arm's name is the code's own claim: its comment says the value-count guard "runs FIRST and refuses a corrupt row_count before a byte is allocated". The MaxAllocSize check after it is only a backstop, and 500 MB doesn't reach it.

The fix (both twins)

Each arm requires the SQLSTATE and the no-nulls guard's message, so the want is XX001|the no-nulls guard. On a red, the value names the guard that fired instead.

  • Shell: the unused sqlstate helper becomes guard_of, which reads with VERBOSITY verbose.
  • pytest: a _guard helper reads diag.message_primary.

Measured (PG17)

mutation shell pytest
none 25/25 25/25
no-nulls guard disabled both arms red, each naming the guard that fired first arm red; the second is hidden, because expect raises on the first red
guard disabled only for a row count of a billion or more second arm red: "fixed-length value runs past the value stream end" second arm red: the same text

Matrix:

  • both twins 25/25 on PG15–19;
  • harness_selftest 1245/1245;
  • collection 710;
  • guards 438/438;
  • ledger gate rc 0, census 2939 agrees.

Ledger

The first mutation dates both rows. Census: never 2941 -> 2939, dated 1160 -> 1162. No row is added, removed or renamed.

#1479 adds one ledger row too. It merges first, and then I'll re-anchor this budget paragraph, a bookkeeping-only change.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd force-pushed the test/1339-validity-elision-guard branch from 445c570 to b1ed04b Compare October 16, 2026 14:59

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Requesting one change at 445c570. The guard discrimination is real, but the second arm's name still claims an order it doesn't measure.

What holds (PG17 and PG18)

tree / mutant shell pytest
head 25/25 25/25
no-nulls guard disabled, origin/main tests 25/25 green (the gap) green
no-nulls guard disabled, this head both arms red, each naming the guard that fired arm 1 red (arm 2 masked, first red)
guard disabled only for row_count ≥ 1e9, origin/main tests green green
guard disabled only for row_count ≥ 1e9, this head arm 2 red arm 2 red
mine: the guard raises XX000, not XX001 both arms red (XX000|the no-nulls guard) red
mine: != made one-sided (>) both arms red, later guards named red
  • Ledger: census 4101 / 2939 / 1162 equals the budget. Neither dated row's mutation text has a ";". pgc_ledger.py gate --against origin/main gives rc 0, and the ceiling stays at 201.
  • Grader and guards: 40 passed, and the guards pass 438/438.

The change: "refused before anything is allocated" isn't measured

The arm checks which guard refused, not when. I moved the no-nulls guard block (columnar_reader.c:3128 to 3135) below the if (allPresentBits == NULL) allocation block. The 500 MB bitmap is allocated, then the same guard refuses with the same message, and both twins stay 25/25 green on PG17 and PG18.

At row_count = 4000000000 the bitmap is 500 MB, under MaxAllocSize, so the :3170 backstop never fires and the order can't be seen.

Measured fix (shell twin): poison with 9000000000 instead. That's 1.125 GB, over MaxAllocSize, so the backstop sits between the guard and the allocation:

  • head stays 25/25;
  • the moved-guard mutant goes red with got [XX001|another guard: columnar row group 1 declares an implausible row count];
  • both of your mutants still go red.

It still can't see a guard placed between the backstop and the alloc, and the comment could say so. Please make the same change in the pytest twin. The alternative is renaming the arm, which costs a ledger rename.

Note, not blocking: a pure reword of the message reddens both arms. That's acceptable to me, since the red prints the real message and XX001 is the right code, so there's no sturdier discriminator. A one-line comment saying a reword must update both twins would save someone a puzzle.

🤖 Generated with Claude Code

jdatcmd added a commit that referenced this pull request Oct 16, 2026
… is allocated" is measured (#1339)

Review of #1484: the second arm checked WHICH guard refused, not WHEN. At
row_count = 4000000000 the bitmap is 500 MB, under MaxAllocSize, so with
the no-nulls guard moved below the allocation both twins stayed 25/25
(reproduced, PG17).

Both twins now poison with 9000000000. 1.125 GB of bits is over
MaxAllocSize, so the "implausible row count" backstop sits between the
no-nulls guard and the allocation. Measured, PG17:
- clean: 25/25 in both twins;
- guard moved below the allocation: arm 2 red in both twins, "XX001|another
  guard: columnar row group 1 declares an implausible row count";
- guard disabled: the shell reddens both arms; pytest the first (the
  second is masked);
- guard disabled at a billion rows or more: arm 2 red in both twins.

The comments say what this still cannot see (a guard placed between the
backstop and the allocation) and that a reworded guard message must be
updated in both twins. No ledger change: both rows stay dated by a
mutation that still reddens them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd

jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, your moved-guard mutant was right; I reproduced it (25/25 green at b1ed04b, PG17). Fixed in 859e732: both twins now poison with 9000000000, as you measured.

Measured, PG17:

mutation shell pytest
none 25/25 25/25
no-nulls guard moved below the allocation (yours) arm 2 red: XX001|another guard: columnar row group 1 declares an implausible row count arm 2 red, same
no-nulls guard disabled both arms red arm 1 red (arm 2 masked)
guard disabled at a billion rows or more arm 2 red arm 2 red

What changed:

  • Comments: both twins now say what the check still can't see (a guard placed between the backstop and the allocation), and that a reworded guard message must be updated in both twins.
  • Ledger: no change. Both rows stay dated by a mutation that still reddens them, so the census stays 2940 and the gate returns rc 0.

Matrix: both twins 25/25 on PG15–19, harness_selftest 1246, collection 710, guards 438/438.

Please re-review 859e732.

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Approve, at 859e732. Against the head I reviewed, the response changes only the poison value (4e9 to 9e9) in both twins, plus comments.

Measured on PG17 and PG18:

  • head: 25/25 in both twins;
  • no-nulls guard moved below the allocation: arm 2 is now red in both twins, naming columnar row group 1 declares an implausible row count. The arm's name is now what it measures;
  • guard disabled: both shell arms are still red;
  • guard disabled only from 1e9 rows: only arm 2 is red.

Census 4102 rows, 2940 never, which equals the tsv. The gate against origin/main gives rc 0 ("2940, they agree"), the grader passes 40 and the guards 438.

For the PR body only: at 9e9, arm 2's red under your mutants now names the backstop ("declares an implausible row count"), not "fixed-length value runs past the value stream end". Its census line (2941 -> 2939 over 4101) also predates the re-anchor to 2942 -> 2940 over 4102.

🤖 Generated with Claude Code

jdatcmd and others added 2 commits October 16, 2026 16:22
… that one did (#1339)

Both arms in test_the_guards_the_elision_added_refuse_a_lying_catalog
asserted only SQLSTATE XX001. Sixteen guards in the reader raise XX001.
With the no-nulls guard ("claims no nulls but does not account for every
row") disabled, both twins stayed green (25/25 each):
- "a chunk claiming no bitmap while it holds fewer values than rows is
  refused" got XX001 from "native chunk length does not match
  descriptor";
- "an implausible row count on an elided chunk is refused before
  anything is allocated" got XX001 from "fixed-length value runs past the
  value stream end", AFTER the 500 MB bitmap was allocated. The source
  says the value-count guard is the one that refuses before allocation;
  the MaxAllocSize check after it is only a backstop.

Each twin now requires the SQLSTATE AND the no-nulls guard's message
("XX001|the no-nulls guard"), and on a red names the guard that fired.

Measured, PG17:
- guard disabled: the shell reddens both arms, naming the other guards;
  pytest reddens the first, which hides the second (expect raises);
- guard disabled only at a billion rows or more: both twins redden the
  second arm alone, with the same message;
- unmutated: 25/25 in both twins.

The two ledger rows are dated by the first mutation; census 2941 -> 2939
never, 1160 -> 1162 dated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… is allocated" is measured (#1339)

Review of #1484: the second arm checked WHICH guard refused, not WHEN. At
row_count = 4000000000 the bitmap is 500 MB, under MaxAllocSize, so with
the no-nulls guard moved below the allocation both twins stayed 25/25
(reproduced, PG17).

Both twins now poison with 9000000000. 1.125 GB of bits is over
MaxAllocSize, so the "implausible row count" backstop sits between the
no-nulls guard and the allocation. Measured, PG17:
- clean: 25/25 in both twins;
- guard moved below the allocation: arm 2 red in both twins, "XX001|another
  guard: columnar row group 1 declares an implausible row count";
- guard disabled: the shell reddens both arms; pytest the first (the
  second is masked);
- guard disabled at a billion rows or more: arm 2 red in both twins.

The comments say what this still cannot see (a guard placed between the
backstop and the allocation) and that a reworded guard message must be
updated in both twins. No ledger change: both rows stay dated by a
mutation that still reddens them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd
jdatcmd force-pushed the test/1339-validity-elision-guard branch from 859e732 to d146f76 Compare October 16, 2026 16:43
@jdatcmd

jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, #1485 and #1487 have landed, so I've restated my two ledger PRs as a chain on main ae0f9fc. Both need only a bookkeeping re-approval:

PR approved new head stacked on
#1484 859e732 d146f76 main
#1486 edd94b8 f88c588 #1484

Verified for each, against the head you approved:

  • the patches of both twin files are byte-identical;
  • the ledger rows changed are identical (sorted diff, md5 match);
  • only the budget paragraph is re-anchored.

Census:

The gate returns rc 0 for each, and the guards pass, 438/438.

Once both are re-approved, I'll merge #1484 and then #1486.

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Re-approving at d146f76. Against 859e732, the twin files' patch is byte-identical (md5 of the +/- lines), and the changed ledger rows are identical (sorted). The tsv gives 4189 / 2988 / 1201, which equals checks_never_observed_red 2988. Only the budget anchors moved.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants