Skip to content

test: audit.sh's two index arms read the index they name (#1339) - #1507

Merged
jdatcmd merged 7 commits into
mainfrom
test/1339-audit-index-arms
Oct 16, 2026
Merged

jdatcmd merged 7 commits into
mainfrom
test/1339-audit-index-arms

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

@jdatcmd The last #1339 custom-scan audit entry. Stacked on #1506 (#1504 → #1506 → this). This PR's own commit is the top one.

audit.sh's two index arms

"add-column index scan" and "create index rows indexed once" forced their index with enable_seqscan=off alone. That leaves the columnar custom scan on, and bitmap scans too, which would also collapse a duplicate index entry. Both lookups now turn off the custom scan and bitmap scans, and each asserts an Index Scan or Index Only Scan through its index.

Red first, with the index build (pgcolumnar_index_build_range_scan) handing every row to its callback twice:

arm old suite new suite
"add-column index scan" green: it wasn't reading ac_c_idx red
"create index rows indexed once" red: its plan happened to read the index red, with its plan now asserted

Five-major runs at head:

  • 33/33 on PG17, 18 and 19;
  • 29 pass + 1 skip on PG15 and 16, which is the suite's existing "partitioned-parent arm" skip. Both new premises pass there.

audit has no ledger rows and no pytest twin, so nothing else changes.

That completes the audit I posted on #1339:

PR suite
#1499 native_index
#1500 native_delete_visibility_paths
#1502 maintenance_rollback and maintenance_crash
#1503 native_fetch_bigcap
#1504 partition_rows
#1506 sorted_projection
this audit

🤖 Generated with Claude Code

jdatcmd and others added 7 commits October 16, 2026 18:11
…dium projections assert their plans (#1341, #1339)

test_native_fetch_bigcap.py carries all 15 checks by the shell's name in
two tests, measuring buffers as the shell's awk sums them.

Two premises are new, in both twins: the narrow (sum(tag)) and medium
(m1 || m2 || m3) projections are fetched through an index scan. Only the
payload query's plan was asserted. Both plan as Index Scans today, so these
guard against a planner change rather than fix a red. FORCE_INDEX also turns
the custom scan off, which enable_seqscan does not (#1339).

Measured and left alone:
- "the fetched rows are the right rows" compares an unforced read with an
  ORDER BY id LIMIT 10 read, which take different paths, so it reddens when
  the deferred decode corrupts a column (shell and twin). Forcing both sides
  through the index made it agree with itself, so it stays as it was.
- the two #433 ratios do not move when the cache misses every lookup or the
  cap drops every column, because those re-read only the projected columns.
  #433's own regression re-read the group's raw bytes per fetch, and there is
  no single-gate mutation for that here.

COMPLETE gains native_fetch_bigcap. TESTS.md section 173, cluster_tests
742 -> 744.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by a deferred-decode mutation (#1341)

The suite had no rows. suites_not_covered 193 -> 192; census 4347 rows /
3031 never / 1316 dated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"the primary key finds a moved row at its new key, and not at its old one,
as on heap" forced the index with enable_seqscan off alone, which leaves the
columnar custom scan on. Measured: with pgcolumnar_index_fetch_tuple finding
no row, the arm stayed green, because the columnar parent was answered by its
partitions' custom scans. IDX now also turns the custom scan off, and a
premise asserts the columnar lookup reads the primary key. It plans as an
Index Only Scan, which on a columnar table still asks each row whether it is
live, so the premise accepts either index node. With the same mutation the
arm is now red, in both twins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…te-key arms are dated (#1339)

Census 4348 rows / 3029 never / 1319 dated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the index, and each refusal must be its own (#1341, #1339)

test_sorted_projection.py carries all 29 checks by the shell's name in
three tests. Under each of four mutations the twin reddens exactly the
shell's set: the post-sort reindex removed, the compaction not sorting, and
vacuum_sorted's columnar check removed.

Three shell changes, each measured red first:
- "index point lookup after sort" set only enable_seqscan off. With the
  reindex removed, so the index points at the old row numbers, it stayed
  green: it never read the index. The lookups now turn the custom scan off,
  with a premise each that they read t_col_id_idx (an Index Scan or, for the
  count, an Index Only Scan).
- "reject non-columnar table" counted any error. With vacuum_sorted's
  columnar check removed, the heap table still raised -- XX001, reading the
  heap's file as columnar storage -- and the arm stayed green. Each refusal
  now requires its own SQLSTATE (42809, 42703, 22023, 42883).
- expect_error recorded a red as "<label> (expected error)", a name the
  passing run never records, so the ledger's row could not be dated by it.
  It records under the label.

COMPLETE gains sorted_projection, and its literal tables join the
loop-reader guard's list. TESTS.md section 174, cluster_tests 744 -> 747.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… vacuum_sorted mutations (#1341)

The suite had no rows. suites_not_covered 192 -> 191; census 4377 rows /
3053 never / 1324 dated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"add-column index scan" and "create index rows indexed once" forced their
index with enable_seqscan off alone, which leaves the columnar custom scan
on, and left bitmap scans on (which would also collapse a duplicate index
entry). Both lookups now turn off the custom scan and bitmap scans, and each
asserts an Index Scan or Index Only Scan through its index.

Red first, with the index build handing every row to its callback twice:
the old suite's "add-column index scan" stayed green, because it was not
reading ac_c_idx. It is red now. "create index rows indexed once" was
already red under it (its plan happened to read the index), and now its
plan is asserted.

audit has no ledger rows and no pytest twin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve, at 481becf (its own commit, on #1506).

My own mutant, the opposite of yours: the index build skips every tenth row (rowNumber % 10 == 3 → continue before FormIndexDatum) instead of indexing each twice. PG17, audit.sh:

  • add-column index scan: got [4] want [5];
  • create index rows indexed once: got [45000] want [50000].

So both arms now read the index they name, in both directions: too many entries and too few. Turning bitmap scans off as well is the right call, since a bitmap would collapse duplicates.

@jdatcmd

jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

@OffgridwithJD, merging #1507 now: approved at 481becf, 15/15 green, CLEAN.

@jdatcmd
jdatcmd merged commit ed4d2da into main Oct 16, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants