Repository navigation
test: native_dml gets its pytest twin, and its min/max arm can see a zone-map answer that ignores deletes (#1341) - #1454
Merged
Merged
Conversation
jdatcmd
approved these changes
Oct 16, 2026
jdatcmd
left a comment
jdatcmd
left a comment
Collaborator
There was a problem hiding this comment.
Approving 3391edd.
Measured on PG17: native_dml.sh 15/15, and test_native_dml.py 15 checks. The tsv at this head is 3687 / 2703 / 984, matching its budget.
Your central claim, with the mutant run myself: if (deleted > 0) → if (deleted > 0 && !needZones) in columnar_vector.c, so a group that holds deletes answers min/max from its zone maps. Both twins go red on min/max after delete: got [1|5000|10|50000] want [2|4999|20|49990]. Deleting ids 1 and 5000 moves the extremes, and k (which the primary key doesn't order) shows the zone map's stale 10 and 50000. Before this PR the deletes moved neither extreme, so the arm couldn't see this.
I took the count and index-fetch mutants from your table.
…es ALTER EXTENSION UPDATE (#1341 item 66) Between installing a new library and running ALTER EXTENSION pgcolumnar UPDATE, a database runs the 1.0-alpha5 library over its 1.0-alpha4 catalog. The library deletes from pgcolumnar.retired_group, which that catalog lacks, so DROP TABLE, TRUNCATE and pgcolumnar.vacuum() on a columnar table fail there with 42P01 'columnar metadata table "pgcolumnar.retired_group" does not exist', which names neither the cause nor the remedy. Measured: each refusal changes nothing, reads and writes work throughout, and after the update all three succeed on the same rows. open_columnar_table's error now carries a hint naming the update. docs/installation.md's "If you skipped step 3" shows the error and the hint. Red first, both twins, PG18: native_upgrade_converge gains the window's arms in a database at the previous release's catalog (the newest fixture), and the three refusal arms fail on main with `got [42P01|no-hint]`. test_native_upgrade_converge.py is their partial twin, declared INCOMPLETE; it stages the previous release's base script from test/fixtures as the shell does. Green on 15-19 in both twins. cluster_tests 660 -> 661 by collection; TESTS.md gets section 143. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… still be set to a row store (#1341) test_fk_referencing.py asserts every shell check by name; compare_to_bash grades it as covering all 28 (27 checks on PG17+, and the pre-17 branch's own), and it joins COMPLETE. The conversion guard's over-rejection control converted a table with no foreign key back to heap, so it could not see a guard that refused every SET ACCESS METHOD on a referenced table whatever the target: with the guard's columnar-target test removed, the suite stayed 26/26. A new arm sets the referenced table itself to heap; under that mutant both twins now fail it. cluster_tests 661 -> 666 by collection; TESTS.md gets section 144. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tations of the foreign-key guards (#1341) 28 rows from five-major runs (27/27 on 17-19, 26/26 on 15-16). Dated on PG18 by: the referenced-side refusal removed (5), the guard keyed on the referencing table (8), the conversion refusal removed (8), and the conversion refused whatever the target (1). suites_not_covered 219 -> 218; the census is 3647 rows, 2686 never, 961 dated, restated on main db56168. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_truncate_cleanup.py asserts every shell check by name; compare_to_bash grades it as covering all 25, and it joins COMPLETE. It runs in a private database, since the snapshot counts every pgcolumnar catalog row and the shared one holds every other test's. The shell suite needed no change: each of its arms already goes red under a mutation of the path it names. cluster_tests 666 -> 672 by collection; TESTS.md gets section 145. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mutations of the truncate paths (#1341) 25 rows from five-major runs (25/25 on each of 15-19). Dated on PG18 by a rewrite that keeps the retired storage's rows (7) or its stale write state (1), an in-place truncate that skips the projections' storages (8), and a storage-tree delete that skips them (1). suites_not_covered 218 -> 217; the census is 3672 rows, 2695 never, 977 dated, restated on main db56168. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…zone-map answer that ignores deletes (#1341) test_native_dml.py asserts every shell check by name; compare_to_bash grades it as covering all 15, and it joins COMPLETE. The min/max arm could not fail as its comment claimed. The deletes (id % 7 = 0) removed neither the smallest nor the largest id, so min and max were the same whether or not the deleted rows counted; and measured with groups that hold deletes left on the zone-map path, min/max(id) still came out right while min/max(k) came out wrong (10|50000 for 20|49990). The suite stayed 13/13. The deletes now include ids 1 and 5000, a premise asserts on heap that min and max moved, and the arm takes min/max over k as well as id; under that mutant both twins fail it. The index arms gain a premise that the lookups take the primary-key index, so they test the index fetch's delete check rather than the scan. cluster_tests 672 -> 673 by collection; TESTS.md gets section 146. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ons (#1341) 15 rows from five-major runs (15/15 on each of 15-19). Dated on PG18 by: a row group with deletes still answering min/max from zone maps (1), count(*) from metadata not subtracting deleted rows (3), and the index fetch ignoring the delete vector (5). suites_not_covered 217 -> 216; the census is 3687 rows, 2702 never, 985 dated, restated on main db56168. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OffgridwithJD
force-pushed
the
test/1341-native-dml-twin
branch
from
October 16, 2026 05:01
3391edd to
091f982
Compare
jdatcmd
approved these changes
Oct 16, 2026
jdatcmd
left a comment
jdatcmd
left a comment
Collaborator
There was a problem hiding this comment.
Re-approving 091f982, the restate onto main db56168. I compared every commit's patch with the approved stack using git range-diff. The test and source commits are unchanged, and the ledger-seed commits changed only by re-anchoring on #1470. The census at this head matches its budget paragraph, and the counts step consistently. At #1467's head: the guards pass (438), 687 collected (matching the file), and harness_selftest is 1245/1245.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#1341, the twin-less suites:
native_dml.shhad no pytest twin. This PR adds one, every check by the shell's name, and seeds the suite into the ledger. The port found that the suite's min/max arm couldn't see the bug its comment names, and that's fixed in both twins.Stacked on #1453, at the top of my chain. This branch is #1453's head plus two commits.
What changes
test/pytest/test_native_dml.py, the twin: one test in a private schema that runs DELETE, UPDATE and MERGE on a heap table and a columnar one and compares every result.compare_to_bash.pygrades it as covering all 15 names, and it joinsCOMPLETE.id % 7 = 0) removed neither the smallest nor the largest id, so min and max were the same whether or not the deleted rows counted.min/max(id)still came out right under that mutant.min/max(k)came out wrong,10|50000for20|49990, becausekis a column the primary key doesn't order.kas well asid.native_dmlis seeded with 15 rows from five-major runs, and 8 of the 15 rows dated.suites_not_coveredgoes 217 → 216, and the census is 3672 + 15 = 3687 rows, 2696 + 7 = 2703 never and 976 + 8 = 984 dated, matching the tsv. The gate gives rc=0 against origin/main.Red first
Each mutation was run one at a time on PG18, through both twins, with the
.somd5 printed every time.native_dml.shtest_native_dml.pycolumnar_vector.c,if (deleted > 0 && !needZones))min/max after delete:got [1|5000|10|50000] want [2|4999|20|49990]got [5000] want [4284]PgColumnarRowIsLive→ true)index skips deleted row:got [1], and the rows, sum and count after UPDATE and MERGEThe pytest twin is one test, so it stops at each mutation's first red.
Never red (7): the two premises, the rows and sum after the delete, the deleted rows being gone, the live row's index lookup, and the primary key arm.
Runs
native_dml.shtest_native_dml.pyAlso measured: the guard half (438 tests), harness_selftest 1245/1245 and docs_style.
cluster_testsgoes 672 → 673 by collection, and TESTS.md gets §146.🤖 Generated with Claude Code