Repository navigation
fix: a collation change without a rewrite forgets the column's stored bounds (#1327) - #1329
Conversation
…ing as heap does, red (commandprompt#1327) Seen red on PG17 at 11f954e: min|max B0|m6 against heap's a0|m6, WHERE s < 'B' empty against 750 rows, an equality on 'a0' 0 against 107, the extent still the C bound, and the same through a partitioned parent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andprompt#1327) The projection arm was seen red on PG17 at 11f954e, and red again with only the base storage's ordering forgotten. The sorted-table arm is green on main: the sorted-pathkeys claim already refuses collatable columns, and this arm is what keeps that refusal. Two arms that nothing could redden are removed (a projection never claims an order, and a rolled-back ALTER rolls its catalog back by construction). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… bounds (commandprompt#1327) ALTER COLUMN ... TYPE text COLLATE <other> keeps the relfilenode, so the zone maps kept minima and maxima under the old collation and the reader pruned by them under the new one. The ProcessUtility hook records each columnar relation's relfilenode and column collations before an ALTER that changes a type (the named relation and its inheritors), and after it, for every relation not rewritten, forgets the ordering of each column whose collation changed: zone-map minima and maxima nulled (base and projection storages), extent rows deleted. Counts and sums stay. A zone map with values and no bounds is the state a type with no ordering already writes; the ungrouped min/max fold now reads such a group instead of skipping it as all-NULL. collation_change.sh 11/11 on PG17. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ELOG (commandprompt#1327) The twin was seen red on PG17 against main (min|max Q0|x8 against heap's c0|x8) and passes 11/11 with the fix. Its own values, geometry, two list partitions, and a projection that lists the column second. A server with no en_US collation is cannot_run under the shell's pgc_skip name, so the pair grades 12/12. cluster_tests 524 -> 525 by collection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ollation change (commandprompt#1327) PG18 asserts that a heap scan's snapshot is registered or active; on an assert build every collation change aborted on 18 and 19. Seen in the five-major run; collation_change.sh 11/11 and its twin on 15-19 after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… mutations (commandprompt#1327) collation_change.sh 11/11 and test_collation_change.py on each of PG15-19. Ledger 2203 / 1748, gate rc=0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
c5023d1 to
e21d3f3
Compare
jdatcmd
left a comment
There was a problem hiding this comment.
Approved at e21d3f3. Merging when CI is green on this head.
The risk I most wanted to rule out is closed. Could a projection sorted on the column still supply ORDER BY in the old collation's order? That would be wrong even with the bounds nulled. It can't happen: the sorted-scan path refuses any column with a collation (columnar_customscan.c, OidIsValid(att->attcollation)), and the comment there records this very case. Probed on PG17: ORDER BY s LIMIT, DISTINCT ... ORDER BY and GROUP BY ... ORDER BY all plan a Sort (s COLLATE "en_US.utf8"). Range and equality reads through the projection match heap.
Aggregates and awkward ALTER shapes, all matching heap on PG17:
- min/max under the default plan, the ungrouped fold (with and without a filter), and the grouped path;
- a collation change plus a rewriting type change (rewritten, fine);
- a collation change plus
ADD COLUMN(6/6 bounds nulled); - a rolled-back ALTER, a savepoint rollback, ALTER then a read in the same transaction, and a DO block;
- a partitioned parent with a heap and a columnar partition.
A domain with the same C collation nulls nothing, so no bounds are lost.
Runs at e21d3f3:
collation_change.sh: 11/11 on PG17, PG18 and PG19. PG19 is not in CI's build matrix.test_collation_change.pyon PG17: green.native_projection.shon PG17: 8/8.
The ledger is right. Main's rows are unchanged. 2270 rows, of which 1799 are never red, equal to the budget. The three never-red collation rows are premises.
Not examined: dump/restore and pg_upgrade of nulled bounds.
Fixes #1327, which you filed while reviewing #1325.
The bug
ALTER COLUMN ... TYPE text COLLATE <other>keeps the relfilenode, so no zone map was rewritten. The stored minima and maxima were the old collation's, and the reader compared them under the new one. Red first, on PG17 at 11f954e, against heap:C→en_US.utf8min(s) || '|' || max(s)B0|m6a0|m6count(*) WHERE s < 'B'count(*) WHERE s = 'a0'column_extent()also still printed the C bound.The fix: forget the bounds rather than recompute them
The ProcessUtility hook handles any
ALTER TABLEthat changes a column type.PgColumnarForgetColumnOrdering:[rownumber, cols…], so its column index is the list position + 1.A zone map with values and no bounds is already the state a type with no ordering writes, and the readers treat it as "nothing known":
expirekeeps the group;One reader was wrong about that state. The ungrouped min/max fold skipped such a group as if it were all NULL, and returned NULL where heap returned a value. It now sends the group to be read, as it already does for an added column.
Recomputing instead would decode the column in every group, inside the ALTER and under its ACCESS EXCLUSIVE lock. Not rewriting was supposed to spare the user exactly that cost. A later rewrite builds correct zone maps, either
vacuum_sorted/pgcolumnar.vacuumor a type change that does rewrite.The sorted-pathkeys claim was already safe. It refuses every collatable column for exactly this reason.
vacuum_sorted's ORDER BY arm is green on main and is that refusal's guard, dated by a mutant that removes it.Found on the way
PgColumnarReadExtentRowswas first given an unregisteredGetLatestSnapshot(). PG18 asserts a heap scan's snapshot is registered, so every collation change aborted on the 18 and 19 assert builds. Caught by the five-major run and fixed in its own commit.Evidence
collation_change.sh(11 checks) and the independenttest_collation_change.pypass on PG15–19, with no TRAPs. The twins grade 12/12, the 12th being the shell'spgc_skipname, mirrored by the twin'scannot_run.suites_not_coveredstays 246.native_agg35/35,native_agg_addcolumn17/17,native_agg_deletes8/8,native_zonemap18/18,sorted_pathkeys113/113,projections73/73,projection_rewrite84/84,column_extent36/36,alter_column_type21/21,docs_style58/58.cluster_tests525 → 526. TESTS.md section 103. CHANGELOG Fixed.A server with no en_US collation fails the shell suite.
pgc_skippasses it only whenPGC_ALLOW_MISSING_EN_US_COLLATION=1waives it, and the twin'scannot_runmakes the run INCOMPLETE. Neither can pass vacuously. CI's Ubuntu images haveen_US.utf8, but I haven't confirmed that inside the cluster jobs; this PR's CI will.🤖 Generated with Claude Code