fix: a collation change without a rewrite forgets the column's stored bounds (#1327)

CHANGELOG.md

@@ -356,6 +356,15 @@ true until the next version shipped.
 
 ### Fixed
 
+- Changing a text column's collation without a rewrite no longer gives wrong
+  answers (#1327). `ALTER COLUMN ... TYPE text COLLATE <other>` keeps the
+  table's files, so the zone maps kept minima and maxima ordered under the old
+  collation. Ranges, equalities and `min()`/`max()` then skipped rows that the
+  new collation orders differently. The column's stored bounds are now
+  forgotten after such a change, in the table and in its projections, and its
+  extent is dropped. Queries stay correct and read those groups until a
+  rewrite builds new zone maps. `test/collation_change.sh` and an independent
+  `test_collation_change.py`. Reported by @jdatcmd.
 - `test/native_upgrade_converge.sh` no longer leaves its PostgreSQL server
   running and its workdir in `/tmp` after every run (#1321). It armed its own
   `trap cleanup EXIT` after `pgc_setup`, which replaced the harness's

src/columnar.h

@@ -629,6 +629,7 @@ extern void PgColumnarInsertExtentRow(uint64 storageId, int columnIndex, bool co
 									  const char *maximum, int maximumLen);
 extern List *PgColumnarReadExtentRows(uint64 storageId, int columnIndex, Snapshot snapshot);
 extern void PgColumnarDeleteExtentRow(ItemPointer tid);
+extern void PgColumnarForgetColumnOrdering(uint64 storageId, int columnIndex);
 extern Oid	PgColumnarStorageRelationOid(uint64 storageId, Snapshot snapshot);
 
 /* the extent's lifecycle: columnar_extent.c */

src/columnar_metadata.c

@@ -4761,6 +4761,84 @@ PgColumnarDeleteExtentRow(ItemPointer tid)
 	table_close(rel, RowExclusiveLock);
 }
 
+/*
+ * PgColumnarForgetColumnOrdering
+ *		The column's stored minima and maxima were computed under an ordering that
+ *		no longer holds: its collation changed and nothing was rewritten (#1327).
+ *
+ * Every zone-map row of the column, whole-chunk and per-vector, loses its
+ * minimum and maximum, and the column's extent rows are deleted. The rows stay,
+ * with their counts and sums, which no ordering decides. A zone map with values
+ * and no minimum and maximum is the state a type with no ordering already
+ * writes, and every reader already treats it as "nothing is known": no group is
+ * pruned, min() and max() decode, expire keeps the group, and a recompute of the
+ * extent refuses, so the extent stays absent until a rewrite builds new zone
+ * maps.
+ *
+ * Forgetting rather than recomputing: a recompute decodes the column in every
+ * group, inside the ALTER and under its ACCESS EXCLUSIVE lock, which is the cost
+ * the collation change was spared by not rewriting.
+ */
+void
+PgColumnarForgetColumnOrdering(uint64 storageId, int columnIndex)
+{
+	Relation	rel = open_columnar_table("zone_map", RowExclusiveLock);
+	TupleDesc	td = RelationGetDescr(rel);
+	ScanKeyData key[1];
+	SysScanDesc scan;
+	HeapTuple	tuple;
+	Oid			idx;
+	Datum		values[Natts_zone_map];
+	bool		nulls[Natts_zone_map];
+	bool		replace[Natts_zone_map];
+
+	memset(values, 0, sizeof(values));
+	memset(nulls, 0, sizeof(nulls));
+	memset(replace, 0, sizeof(replace));
+	nulls[Anum_zone_map_minimum - 1] = replace[Anum_zone_map_minimum - 1] = true;
+	nulls[Anum_zone_map_maximum - 1] = replace[Anum_zone_map_maximum - 1] = true;
+
+	ScanKeyInit(&key[0], Anum_zone_map_storage_id, BTEqualStrategyNumber,
+				F_INT8EQ, Int64GetDatum((int64) storageId));
+	idx = pgcolumnar_scan_index_oid(rel, "zone_map_pkey");
+	scan = systable_beginscan(rel, idx, OidIsValid(idx), NULL, 1, key);
+	while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+	{
+		bool		isnull;
+		HeapTuple	updated;
+
+		if (DatumGetInt16(heap_getattr(tuple, Anum_zone_map_column_index, td,
+									   &isnull)) != columnIndex)
+			continue;
+		(void) heap_getattr(tuple, Anum_zone_map_minimum, td, &isnull);
+		if (isnull)
+			continue;			/* already no bound: an all-NULL unit */
+		updated = heap_modify_tuple(tuple, td, values, nulls, replace);
+		CatalogTupleUpdate(rel, &updated->t_self, updated);
+		heap_freetuple(updated);
+	}
+	systable_endscan(scan);
+	table_close(rel, RowExclusiveLock);
+
+	if (PgColumnarExtentCatalogPresent())
+	{
+		ListCell   *lc;
+		Snapshot	snap;
+
+		/*
+		 * Registered: PG18 asserts that a snapshot used for a heap scan is
+		 * registered or active, and an assert build aborts on an unregistered
+		 * one (measured: every collation change crashed on 18 and 19).
+		 */
+		CommandCounterIncrement();
+		snap = RegisterSnapshot(GetLatestSnapshot());
+		foreach(lc, PgColumnarReadExtentRows(storageId, columnIndex, snap))
+			PgColumnarDeleteExtentRow(&((NativeExtentRow *) lfirst(lc))->tid);
+		UnregisterSnapshot(snap);
+	}
+	CommandCounterIncrement();
+}
+
 /*
  * The relation a storage id belongs to, from pgcolumnar.storage, or InvalidOid.
  * A projection's storage names its base relation too; callers that need the

src/columnar_tableam.c

@@ -3187,13 +3187,153 @@ pgcolumnar_reject_drop_projected_column(AlterTableStmt *stmt)
 	list_free(relations);
 }
 
+/*
+ * A COLLATION CHANGE THAT REWRITES NOTHING (#1327).
+ *
+ * ALTER COLUMN ... TYPE text COLLATE <other> keeps the relfilenode when the type
+ * is unchanged, so no new zone maps are written, and the old ones hold minima
+ * and maxima under the old collation. The reader compares them under the new
+ * one, and pruned rows the new collation orders differently: @jdatcmd measured
+ * min|max B|m where heap said a|m, and WHERE s < 'B' returning nothing.
+ *
+ * Before the statement, each columnar relation it could alter -- the named one
+ * and every inheritor, which is how an ALTER on a partitioned parent reaches a
+ * columnar partition -- has its relfilenode and its columns' collations
+ * recorded from the syscache. Opening the relations here would take a lock the
+ * ALTER then upgrades. After the statement, a relation with the same
+ * relfilenode was not rewritten, and any column whose collation differs has its
+ * ordering forgotten (PgColumnarForgetColumnOrdering). A rewritten relation
+ * wrote new zone maps under the new collation and needs nothing.
+ */
+typedef struct PgColumnarCollationsBefore
+{
+	Oid			relid;
+	Oid			relfilenode;
+	int			natts;
+	Oid		   *collations;		/* by attnum - 1 */
+} PgColumnarCollationsBefore;
+
+static Oid
+pgcolumnar_attcollation(Oid relid, AttrNumber attnum)
+{
+	HeapTuple	tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(relid), Int16GetDatum(attnum));
+	Oid			coll = InvalidOid;
+
+	if (HeapTupleIsValid(tp))
+	{
+		coll = ((Form_pg_attribute) GETSTRUCT(tp))->attcollation;
+		ReleaseSysCache(tp);
+	}
+	return coll;
+}
+
+static List *
+pgcolumnar_collations_before(AlterTableStmt *stmt)
+{
+	List	   *out = NIL;
+	List	   *rels;
+	ListCell   *lc;
+	bool		changesType = false;
+	Oid			relid;
+
+	foreach(lc, stmt->cmds)
+		if (((AlterTableCmd *) lfirst(lc))->subtype == AT_AlterColumnType)
+			changesType = true;
+	if (!changesType || stmt->relation == NULL)
+		return NIL;
+	relid = RangeVarGetRelid(stmt->relation, NoLock, true);
+	if (!OidIsValid(relid))
+		return NIL;
+	rels = find_all_inheritors(relid, NoLock, NULL);
+	foreach(lc, rels)
+	{
+		Oid			r = lfirst_oid(lc);
+		HeapTuple	tp;
+		Form_pg_class cls;
+		PgColumnarCollationsBefore *b;
+		int			a;
+
+		if (!PgColumnarIsColumnarRelation(r))
+			continue;
+		tp = SearchSysCache1(RELOID, ObjectIdGetDatum(r));
+		if (!HeapTupleIsValid(tp))
+			continue;
+		cls = (Form_pg_class) GETSTRUCT(tp);
+		b = palloc0(sizeof(PgColumnarCollationsBefore));
+		b->relid = r;
+		b->relfilenode = cls->relfilenode;
+		b->natts = cls->relnatts;
+		ReleaseSysCache(tp);
+		b->collations = palloc0(sizeof(Oid) * Max(b->natts, 1));
+		for (a = 1; a <= b->natts; a++)
+			b->collations[a - 1] = pgcolumnar_attcollation(r, a);
+		out = lappend(out, b);
+	}
+	return out;
+}
+
+static void
+pgcolumnar_forget_changed_collations(List *before)
+{
+	ListCell   *lc;
+
+	foreach(lc, before)
+	{
+		PgColumnarCollationsBefore *b = (PgColumnarCollationsBefore *) lfirst(lc);
+		HeapTuple	tp;
+		Oid			relfilenode;
+		Relation	rel;
+		uint64		storageId;
+		int			a;
+
+		tp = SearchSysCache1(RELOID, ObjectIdGetDatum(b->relid));
+		if (!HeapTupleIsValid(tp))
+			continue;
+		relfilenode = ((Form_pg_class) GETSTRUCT(tp))->relfilenode;
+		ReleaseSysCache(tp);
+		if (relfilenode != b->relfilenode || !PgColumnarIsColumnarRelation(b->relid))
+			continue;			/* rewritten: its zone maps are new */
+		rel = relation_open(b->relid, AccessShareLock);
+		storageId = PgColumnarStorageId(rel);
+		for (a = 1; a <= Min(b->natts, RelationGetDescr(rel)->natts); a++)
+		{
+			Form_pg_attribute att = TupleDescAttr(RelationGetDescr(rel), a - 1);
+
+			ListCell   *pc;
+
+			if (att->attisdropped || att->attcollation == b->collations[a - 1])
+				continue;
+			PgColumnarForgetColumnOrdering(storageId, a - 1);
+
+			/*
+			 * A projection stores the column in its own storage, with zone maps
+			 * of its own, laid out [rownumber, projcol1..projcolK]: the column at
+			 * position j of the projection's list is column index j + 1.
+			 */
+			foreach(pc, PgColumnarListProjections(storageId))
+			{
+				PgColumnarProjection *proj = (PgColumnarProjection *) lfirst(pc);
+				int			j;
+
+				if (proj->projectionId == 0)
+					continue;
+				for (j = 0; j < proj->columnsLen; j++)
+					if (proj->columns[j] == a)
+						PgColumnarForgetColumnOrdering(proj->projStorageId, j + 1);
+			}
+		}
+		relation_close(rel, AccessShareLock);
+	}
+}
+
 static void
 pgcolumnar_process_utility(PlannedStmt *pstmt, const char *queryString,
 						 bool readOnlyTree, ProcessUtilityContext context,
 						 ParamListInfo params, QueryEnvironment *queryEnv,
 						 DestReceiver *dest, QueryCompletion *qc)
 {
 	Node	   *parsetree = pstmt->utilityStmt;
+	List	   *collationsBefore = NIL;
 
 	/*
 	 * Start from empty, but only for the OUTERMOST statement: a statement that
@@ -3211,6 +3351,7 @@ pgcolumnar_process_utility(PlannedStmt *pstmt, const char *queryString,
 
 		pgcolumnar_reject_drop_projected_column(stmt);
 		pgcolumnar_reject_set_am_to_columnar(stmt);
+		collationsBefore = pgcolumnar_collations_before(stmt);
 	}
 
 	PG_TRY();
@@ -3229,6 +3370,9 @@ pgcolumnar_process_utility(PlannedStmt *pstmt, const char *queryString,
 	}
 	PG_END_TRY();
 
+	/* #1327: after the statement, so a change that errored leaves nothing */
+	pgcolumnar_forget_changed_collations(collationsBefore);
+
 	/*
 	 * A rewrite loses this relation's declared projections, so re-record them
 	 * (#876, #887).

src/columnar_vector.c

@@ -3785,6 +3785,21 @@ pgcolumnar_fill_native_metadata_agg(PgColumnarAggScanState *state, int *ndirty)
 					missingColumn = true;
 					break;
 				}
+
+				/*
+				 * Values with no minimum and maximum: a type with no ordering,
+				 * or a column whose collation changed without a rewrite, whose
+				 * bounds were forgotten (#1327). The zone map cannot answer
+				 * min() or max() for it, and skipping it as if all NULL
+				 * returned NULL where heap returned a value.
+				 */
+				if ((state->specs[a].kind == COLUMNAR_AGG_MIN ||
+					 state->specs[a].kind == COLUMNAR_AGG_MAX) &&
+					!byCol[ai]->hasMinMax && byCol[ai]->valueCount > 0)
+				{
+					missingColumn = true;
+					break;
+				}
 			}
 
 			if (missingColumn)

test/check_ledger.tsv

@@ -204,6 +204,17 @@ catalog_scan_index	catalog_scan_index	premise: truncating a projected table reac
 catalog_scan_index	catalog_scan_index	premise: truncating a projected table took effect	15;16;17;18;19	never	-
 catalog_scan_index	catalog_scan_index	renaming a projected column does not read pgcolumnar.projection_declaration whole	15;16;17;18;19	2026-10-12	the twelve converted sites reverted to InvalidOid, so each reads its catalog whole
 catalog_scan_index	catalog_scan_index	truncating a projected table does not read pgcolumnar.projection_declaration whole	15;16;17;18;19	2026-10-12	the twelve converted sites reverted to InvalidOid, so each reads its catalog whole
+collation_change	collation_change	a collation change made through a partitioned parent reaches the columnar partition	15;16;17;18;19	2026-10-13	nothing forgotten after a collation change (main's behaviour);only the named relation, not its inheritors
+collation_change	collation_change	a projection sorted on the column answers a range as heap does	15;16;17;18;19	2026-10-13	nothing forgotten after a collation change (main's behaviour);the projection column forgotten at its list position, without the rownumber column's offset;the projection storages' bounds not forgotten
+collation_change	collation_change	a table sorted under the old collation orders ORDER BY ... LIMIT as heap does	15;16;17;18;19	2026-10-13	the sorted-pathkeys claim's refusal of collatable columns removed
+collation_change	collation_change	after the collation change min and max are heap's	15;16;17;18;19	2026-10-13	nothing forgotten after a collation change (main's behaviour);the min/max fold skips a group with values and no bounds as all-NULL
+collation_change	collation_change	and a range below an upper-case bound returns heap's rows	15;16;17;18;19	2026-10-13	nothing forgotten after a collation change (main's behaviour)
+collation_change	collation_change	and an equality on a value C sorts first returns heap's rows	15;16;17;18;19	2026-10-13	nothing forgotten after a collation change (main's behaviour)
+collation_change	collation_change	premise: the collation change rewrote nothing	15;16;17;18;19	never	-
+collation_change	collation_change	premise: the columnar table has several row groups	15;16;17;18;19	never	-
+collation_change	collation_change	premise: the new collation orders a lower-case letter before an upper-case one that C puts first	15;16;17;18;19	never	-
+collation_change	collation_change	rows written after the change join heap's answer	15;16;17;18;19	2026-10-13	nothing forgotten after a collation change (main's behaviour)
+collation_change	collation_change	the column's extent is not the old collation's bound	15;16;17;18;19	2026-10-13	nothing forgotten after a collation change (main's behaviour);the column's extent rows kept
 column_extent	column_extent	DROP TABLE removes them	15;16;17;18;19	2026-10-13	PgColumnarDeleteMetadata leaves the extent rows
 column_extent	column_extent	TRUNCATE removes the old storage's extent rows	15;16;17;18;19	2026-10-13	PgColumnarDeleteMetadata leaves the extent rows
 column_extent	column_extent	a column added after older groups has no complete extent (those groups have no zone map for it)	15;16;17;18;19	2026-10-13	a recompute that ignores a group with no zone map for the column;pgcolumnar.column_extent returns a row with no complete extent;the extent trusted without a complete row

test/check_ledger_budget.txt

@@ -1559,4 +1559,21 @@ suites_not_covered 246
 #     never         1745 + 51 = 1796
 #     dated          447 + 16 =  463
 #     1796 + 463 == 2259
-checks_never_observed_red 1796
+#
+# #1327, a collation change without a rewrite (OffgridwithJD): 1796 -> 1799.
+# collation_change enters the ledger with 11 checks from a five-major run
+# (collation_change.sh 11/11 on each of 15-19). Eight arms are dated by seven
+# mutations, each merged one log at a time with --mutation and --target:
+# nothing forgotten (main's behaviour, seven arms), the projection storages
+# skipped, the projection column at the wrong index, the named relation only,
+# the min/max fold treating a group with values and no bounds as all-NULL, the
+# extent rows kept, and the sorted-pathkeys claim's refusal of collatable
+# columns removed. The three never rows are premises about the fixture and the
+# server: the collation orders differently from C, several row groups, and no
+# rewrite.
+#
+#     ledger rows   2259 + 11 = 2270
+#     never         1796 +  3 = 1799
+#     dated          463 +  8 =  471
+#     1799 + 471 == 2270
+checks_never_observed_red 1799