From d0ca319fb50b474ac5f143f4edaef2a69eb18c08 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Thu, 15 Oct 2026 19:16:33 +0000 Subject: [PATCH 1/6] test: the ledger refuses a mutation split on ';', and the 12 split rows are reworded The mutation column is a ';'-separated set, written back sorted. A --mutation whose text held "; " was stored as two mutations, one a fragment starting with a space, and every later merge re-sorted it. #1427's recovery_minimal_wal text did this to 8 rows, and #1423, #1424 and #1426 each carried them as a change of their own. Four older harness_selftest rows were split the same way. - merge refuses a --mutation holding ';' (rc 2, ledger untouched). - the reader refuses a ';'-piece with whitespace at an end, by line: the writer joins with a bare ';', so such a piece is a fragment, never a mutation. - the 12 rows are reworded without ';', from their original text in history. Only field 6 changes; the census stays 3007 / 2282 / 725. Red first: the two refusal arms got 0 want 2 on the unchanged tool, and the committed-ledger arm was red at line 1783 with the refusal in and the rows not yet reworded. guard_tests 435 -> 438 by collection. Co-Authored-By: Claude Opus 5.5 --- test/check_ledger.tsv | 24 +++++++-------- test/pgc_ledger.py | 16 ++++++++++ test/pytest/TESTS.md | 20 ++++++++++++ test/pytest/expected_tests.txt | 5 ++- test/pytest/test_mutation_ledger.py | 48 +++++++++++++++++++++++++++++ 5 files changed, 100 insertions(+), 13 deletions(-) diff --git a/test/check_ledger.tsv b/test/check_ledger.tsv index 734b4a672..694fb5d6e 100644 --- a/test/check_ledger.tsv +++ b/test/check_ledger.tsv @@ -1805,8 +1805,8 @@ harness_selftest 540-an-arm-must-carry-its-measurement a new arm that cannot say harness_selftest 540-an-arm-must-carry-its-measurement a repaired arm must have its row removed, so the list only shrinks 15;16;17;18;19 2026-10-12 part 540's awk sweep never reports an indeterminate verdict (the print guarded by 0);part 540's sweep stops skipping arms whose text ends in $ (the awk 'if (arms ~ /\$/) next' made 'if (0) next') harness_selftest 540-an-arm-must-carry-its-measurement control: a determinate or carrying arm is not swept up 15;16;17;18;19 2026-10-12 part 540's sweep stops skipping arms whose text ends in $ (the awk 'if (arms ~ /\$/) next' made 'if (0) next') harness_selftest 540-an-arm-must-carry-its-measurement control: a determinate or carrying awk arm is not swept up 15;16;17;18;19 2026-10-12 part 540's awk sweep never reports an indeterminate verdict (the print guarded by 0);part 540's awk_determinate treats a comparison against > 0 as indeterminate (return 1 made return 0) -harness_selftest 540-an-arm-must-carry-its-measurement control: the carve-outs hold and the innocent arm above the helper is not named 15;16;17;18;19 2026-09-24 else echo 0; fi)` is invisible again (mutation B, the same-line recorder conjunct removed from the `[` rule, reddens the SAME two arms -- they are not independent); then echo 1;the shell if/then/else spelling removed from the matcher, so `$(if [ C ] -harness_selftest 540-an-arm-must-carry-its-measurement control: the sweep sees a shell if/then/else that discards its operand 15;16;17;18;19 2026-09-24 else echo 0; fi)` is invisible again (mutation B, the same-line recorder conjunct removed from the `[` rule, reddens the SAME two arms -- they are not independent); then echo 1;the shell if/then/else spelling removed from the matcher, so `$(if [ C ] +harness_selftest 540-an-arm-must-carry-its-measurement control: the carve-outs hold and the innocent arm above the helper is not named 15;16;17;18;19 2026-09-24 the shell if/then/else spelling removed from the matcher, so a one-line `$(if [ C ]... then echo 1... else echo 0... fi)` is invisible again (mutation B, the same-line recorder conjunct removed from the `[` rule, reddens the SAME two arms -- they are not independent) +harness_selftest 540-an-arm-must-carry-its-measurement control: the sweep sees a shell if/then/else that discards its operand 15;16;17;18;19 2026-09-24 the shell if/then/else spelling removed from the matcher, so a one-line `$(if [ C ]... then echo 1... else echo 0... fi)` is invisible again (mutation B, the same-line recorder conjunct removed from the `[` rule, reddens the SAME two arms -- they are not independent) harness_selftest 540-an-arm-must-carry-its-measurement control: the sweep sees a threshold arm and a floor arm 15;16;17;18;19 2026-10-12 part 540's sweep stops skipping arms whose text ends in $ (the awk 'if (arms ~ /\$/) next' made 'if (0) next') harness_selftest 540-an-arm-must-carry-its-measurement control: the sweep sees an awk verdict that discards its operands 15;16;17;18;19 2026-10-12 part 540's awk sweep never reports an indeterminate verdict (the print guarded by 0);part 540's awk_determinate treats a comparison against > 0 as indeterminate (return 1 made return 0) harness_selftest 540-an-arm-must-carry-its-measurement premise: the comment fixture quotes a whole recorder call, not a fragment 15;16;17;18;19 never - @@ -1852,10 +1852,10 @@ harness_selftest 570-a-checkout-test-must-ask-git without the trailing boundary harness_selftest 580-a-hand-rebuild-must-record a PostgreSQL symbol is NOT exempt, so the check still has a subject 15;16;17;18;19 2026-10-12 pgc_symbol_is_toolchain deleted from the real lib.sh;the symbol exemption matches every name, so the check has no subject left and a genuinely mislinked major would resolve clean harness_selftest 580-a-hand-rebuild-must-record a log that is not there also produces a diagnosis 15;16;17;18;19 2026-09-24 _hr_diagnose restored to its silent `[ -s ] || return 0` form, so an empty or missing rebuild.log prints nothing at all harness_selftest 580-a-hand-rebuild-must-record a log with content produces a diagnosis to print 15;16;17;18;19 2026-09-24 rebuild.sh forced to fail after its stamp AND _hr_diagnose gutted, so the failure prints no reason -harness_selftest 580-a-hand-rebuild-must-record a rebuild that failed skips them rather than reading a stamp it did not write 15;16;17;18;19 2026-09-24 driven alone, with no forced rebuild failure, because these two arms call the gate with literals and the rebuild's status never reaches them;the gate reverted so a failed rebuild still runs the arms beneath it +harness_selftest 580-a-hand-rebuild-must-record a rebuild that failed skips them rather than reading a stamp it did not write 15;16;17;18;19 2026-09-24 the gate reverted so a failed rebuild still runs the arms beneath it, driven alone with no forced rebuild failure, because these two arms call the gate with literals and the rebuild's status never reaches them harness_selftest 580-a-hand-rebuild-must-record a rebuild that succeeded runs the arms that read its stamp 15;16;17;18;19 2026-09-24 the gate always skips, so a successful rebuild runs none of the arms that read its stamp harness_selftest 580-a-hand-rebuild-must-record an empty log still produces a diagnosis, because silence is the bug 15;16;17;18;19 2026-09-24 _hr_diagnose restored to its silent `[ -s ] || return 0` form, so an empty or missing rebuild.log prints nothing at all -harness_selftest 580-a-hand-rebuild-must-record and any other status skips too, rather than being read as success 15;16;17;18;19 2026-09-24 driven alone, with no forced rebuild failure, because these two arms call the gate with literals and the rebuild's status never reaches them;the gate reverted so a failed rebuild still runs the arms beneath it +harness_selftest 580-a-hand-rebuild-must-record and any other status skips too, rather than being read as success 15;16;17;18;19 2026-09-24 the gate reverted so a failed rebuild still runs the arms beneath it, driven alone with no forced rebuild failure, because these two arms call the gate with literals and the rebuild's status never reaches them harness_selftest 580-a-hand-rebuild-must-record and it distinguishes missing from empty, so the two causes do not merge 15;16;17;18;19 2026-09-24 _hr_diagnose restored to its silent `[ -s ] || return 0` form, so an empty or missing rebuild.log prints nothing at all harness_selftest 580-a-hand-rebuild-must-record and it names the log as empty rather than printing a bare banner 15;16;17;18;19 2026-09-24 _hr_diagnose restored to its silent `[ -s ] || return 0` form, so an empty or missing rebuild.log prints nothing at all harness_selftest 580-a-hand-rebuild-must-record and it never reaches the pg_config check it would otherwise fail on 15;16;17;18;19 2026-09-25 rebuild.sh's fail-closed assertion is removed, so a missing exemption function leaves the undefined list empty and the check reports success @@ -2609,14 +2609,14 @@ range_pruning range_pruning premise: the range type collates differently from it range_pruning range_pruning so does a containment probe 15;16;17;18;19 never - range_pruning range_pruning the overlap qual is pushed down, where it used to be dropped 15;16;17;18;19 never - range_pruning range_pruning the same for a containment probe far to its right 15;16;17;18;19 never - -recovery_minimal_wal recovery_minimal_wal a table created and loaded in one transaction at wal_skip_threshold 0 committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered -recovery_minimal_wal recovery_minimal_wal a table created and loaded in one transaction at wal_skip_threshold 64MB committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered -recovery_minimal_wal recovery_minimal_wal a table truncated and reloaded in one transaction at wal_skip_threshold 0 committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered -recovery_minimal_wal recovery_minimal_wal a table truncated and reloaded in one transaction at wal_skip_threshold 64MB committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered -recovery_minimal_wal recovery_minimal_wal and after the truncate the columnar table holds what its heap mirror holds, at wal_skip_threshold 0 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered -recovery_minimal_wal recovery_minimal_wal and after the truncate the columnar table holds what its heap mirror holds, at wal_skip_threshold 64MB 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered -recovery_minimal_wal recovery_minimal_wal and the columnar table holds what its heap mirror holds, at wal_skip_threshold 0 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered -recovery_minimal_wal recovery_minimal_wal and the columnar table holds what its heap mirror holds, at wal_skip_threshold 64MB 15;16;17;18;19 2026-10-15 this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c);the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered +recovery_minimal_wal recovery_minimal_wal a table created and loaded in one transaction at wal_skip_threshold 0 committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) +recovery_minimal_wal recovery_minimal_wal a table created and loaded in one transaction at wal_skip_threshold 64MB committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) +recovery_minimal_wal recovery_minimal_wal a table truncated and reloaded in one transaction at wal_skip_threshold 0 committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) +recovery_minimal_wal recovery_minimal_wal a table truncated and reloaded in one transaction at wal_skip_threshold 64MB committed its rows, and the server came back through recovery 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) +recovery_minimal_wal recovery_minimal_wal and after the truncate the columnar table holds what its heap mirror holds, at wal_skip_threshold 0 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) +recovery_minimal_wal recovery_minimal_wal and after the truncate the columnar table holds what its heap mirror holds, at wal_skip_threshold 64MB 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) +recovery_minimal_wal recovery_minimal_wal and the columnar table holds what its heap mirror holds, at wal_skip_threshold 0 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) +recovery_minimal_wal recovery_minimal_wal and the columnar table holds what its heap mirror holds, at wal_skip_threshold 64MB 15;16;17;18;19 2026-10-15 the relfilenode created with smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered, and this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c) recovery_minimal_wal recovery_minimal_wal premise: the cluster runs at wal_level=minimal 15;16;17;18;19 never - rewrite_storage_oid rewrite_storage_oid a matview created WITH DATA leaves its storage row pointing at itself 15;16;17;18;19 2026-10-12 the repair gate skips every relation, as if no inheritor were locked by the statement;the rewrite repair gate no longer covers CreateTableAsStmt, so a matview created WITH DATA keeps the transient's OID rewrite_storage_oid rewrite_storage_oid a type rewrite leaves the storage row pointing at the live table 15;16;17;18;19 2026-10-12 the repair gate skips every relation, as if no inheritor were locked by the statement;the rewrite hook does not retarget storage.relation_oid diff --git a/test/pgc_ledger.py b/test/pgc_ledger.py index 1d880f395..83c6666dc 100755 --- a/test/pgc_ledger.py +++ b/test/pgc_ledger.py @@ -312,6 +312,16 @@ def read_ledger(path): f"{path}:{n}: major {m!r} is neither a number nor " f"{MAJOR_UNKNOWN!r}, so this row names no version it applies to") muts = set() if f[5] == NONE else {m for m in f[5].split(";") if m} + # A PIECE WITH WHITESPACE AT AN END IS A FRAGMENT, not a mutation. The writer + # joins the set with a bare ";", so it never produces one; a text that held + # "; " does, when it is split on the ";" (12 rows on the shipped ledger, 8 of + # them from one recovery_minimal_wal mutation, re-sorted by every later merge). + for m in sorted(muts): + if m != m.strip(): + raise LedgerError( + f"{path}:{n}: mutation {m!r} has whitespace at an end, so it is a " + f"fragment of one mutation's text split on ';'. Reword that mutation " + f"without ';' and keep the row's other mutations as they are.") key = (f[0], f[1], f[2]) # A REPEATED KEY, REFUSED RATHER THAN OVERWRITTEN. This was # `rows[key] = [...]`, so a duplicated row collapsed silently and the LAST line @@ -530,6 +540,12 @@ def cmd_merge(args): "--target names what a mutation killed, so it needs --mutation to say " "which mutation. On its own it attributes nothing.") + if args.mutation and ";" in args.mutation: + raise LedgerError( + f"--mutation {args.mutation!r} holds ';', which separates mutations in the " + f"ledger's mutation column, so it would be stored as several. Reword it " + f"without ';' (', and' reads the same).") + if args.mutation: failed = sorted({key for _p, seen in runs for key, vs in seen.items() if any(v == "FAIL" for v, _m in vs)}) diff --git a/test/pytest/TESTS.md b/test/pytest/TESTS.md index 4425ba7e6..ed83fde0e 100644 --- a/test/pytest/TESTS.md +++ b/test/pytest/TESTS.md @@ -2445,6 +2445,26 @@ Last-write-wins records the most recent attack rather than the catalogue the col exists to become. One `--mutation` copied across several logs attributes a deliberate change to failures it had nothing to do with, and is refused. +### `test_a_mutation_named_with_a_semicolon_is_refused_rather_than_split` + +The mutation column is a `;`-separated set, written back sorted, so a `--mutation` +whose text held `; ` was stored as two mutations, one a fragment starting with a space. +Every later merge re-sorted it, and three branches in a row carried the same eight +recovery_minimal_wal rows as a change of their own. `merge` now refuses such a text with +rc 2 and leaves the ledger as it was; the same text with `, and` is recorded whole. + +### `test_a_ledger_row_whose_mutation_holds_a_fragment_is_refused` + +The writer joins the set with a bare `;`, so a piece with whitespace at an end is a +fragment of a split sentence, never a mutation it wrote. The reader refuses the ledger +by line. Two whole mutations separated by a bare `;` still read. + +### `test_the_committed_ledger_names_no_mutation_fragment` + +The committed ledger, read by the same reader. Red on the tree before this change at +line 1783, the first of the 12 rows it rewords: 8 from #1427's recovery_minimal_wal +mutation and 4 older harness_selftest rows. + ### `test_a_mutation_with_two_targets_can_be_recorded_when_both_are_named` A mutation with two GENUINE targets is ordinary: reverting the diff --git a/test/pytest/expected_tests.txt b/test/pytest/expected_tests.txt index 783fc01c6..8e812d734 100644 --- a/test/pytest/expected_tests.txt +++ b/test/pytest/expected_tests.txt @@ -285,7 +285,10 @@ # tests, the twin of selftest part 640, by collection over the NO_CLUSTER files. # guard_tests 433 -> 435 for #1420's two arms in test_writes_wrote_rows.py (a setting # on a watched connection or cursor is refused), by collection over NO_CLUSTER. -guard_tests 435 +# guard_tests 435 -> 438 for three arms in test_mutation_ledger.py: a mutation text holding +# ';' is refused at merge, a ledger piece with whitespace at an end is refused, and the +# committed ledger holds none, by collection over NO_CLUSTER. +guard_tests 438 # The complement: tests that need the driver and a throwaway cluster. Until #1016 these ran # in no CI job at all -- a quarter of the corpus, green when somebody ran them by hand and diff --git a/test/pytest/test_mutation_ledger.py b/test/pytest/test_mutation_ledger.py index a06e83064..07650ec7f 100644 --- a/test/pytest/test_mutation_ledger.py +++ b/test/pytest/test_mutation_ledger.py @@ -125,6 +125,54 @@ def test_the_mutation_column_accumulates_rather_than_overwriting(tmp_path, expec "one mutation cannot be attributed across several runs at once") +def test_a_mutation_named_with_a_semicolon_is_refused_rather_than_split(tmp_path, expect): + """The mutation column is a ";"-separated SET, written back sorted. A `--mutation` + whose text held "; " was stored as two "mutations", one a sentence fragment that + starts with a space, and the next merge sorted the fragment first. Measured on the + shipped ledger: 12 rows, 8 of them from one recovery_minimal_wal mutation (#1427), + re-sorted by three later branches until each looked like a change of its own. + """ + ledger = _w(tmp_path, "l.tsv", "") + red = _w(tmp_path, "r.log", RED) + out, rc = _run("merge", "--ledger", ledger, "--date", "2026-10-15", + "--mutation", "the sync not registered; this build then asserts", red) + expect.num(rc, 2, "a mutation whose text holds ';' is refused, not split in two") + expect.num(out.count("separates mutations"), 1, "and the refusal says why") + expect.num(len(_rows(ledger)), 0, "and the ledger is left as it was") + + out, rc = _run("merge", "--ledger", ledger, "--date", "2026-10-15", + "--mutation", "the sync not registered, and this build then asserts", red) + expect.num(rc, 0, f"control: the same text without ';' is recorded: {out.strip()}") + expect.text({r[2]: r[5] for r in _rows(ledger)}["first check"], + "the sync not registered, and this build then asserts", + "and it is recorded whole, as one mutation") + + +def test_a_ledger_row_whose_mutation_holds_a_fragment_is_refused(tmp_path, expect): + """A ";"-piece with whitespace at either end cannot be a mutation the writer + produced: it joins the set with a bare ";". It is the fragment of a split sentence, + so the ledger is refused by line rather than read as two mutations.""" + green = _w(tmp_path, "g.log", GREEN) + split = _w(tmp_path, "split.tsv", + "demo\tpart1\tfirst check\t18\t2026-10-15\tthe sync not registered; this build asserts\n") + out, rc = _run("merge", "--ledger", split, green) + expect.num(rc, 2, "a mutation piece that starts with a space is refused") + expect.num(out.count("split.tsv:1:"), 1, "and the refusal names the line") + + two = _w(tmp_path, "two.tsv", + "demo\tpart1\tfirst check\t18\t2026-10-15\tbloom neutered;SAOP 128 -> 0\n") + out, rc = _run("merge", "--ledger", two, green) + expect.num(rc, 0, f"control: two whole mutations separated by a bare ';' still read: {out.strip()}") + + +def test_the_committed_ledger_names_no_mutation_fragment(tmp_path, expect): + """The committed ledger read by the same reader, so a split row on main is an + integrity failure of the file every gate reads, not a cosmetic one.""" + copy = _w(tmp_path, "ledger.tsv", (REPO / "test" / "check_ledger.tsv").read_text()) + out, rc = _run("merge", "--ledger", copy, _w(tmp_path, "g.log", GREEN)) + expect.num(rc, 0, f"the committed ledger reads with no fragment in it: {out.strip()[:400]}") + + def test_a_mutation_with_two_targets_can_be_recorded_when_both_are_named(tmp_path, expect): """#1014. A mutation with TWO GENUINE TARGETS is ordinary, not exotic. From 05fe29e4d579126e7b067831bd97865557132bfa Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Thu, 15 Oct 2026 00:48:54 +0000 Subject: [PATCH 2/6] test: recluster recomputes the column extent, in both twins (#1341 item 75) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho --- test/column_extent.sh | 15 +++++++++++++++ test/pytest/test_column_extent.py | 16 ++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/test/column_extent.sh b/test/column_extent.sh index bba291034..066a15d78 100755 --- a/test/column_extent.sh +++ b/test/column_extent.sh @@ -213,6 +213,21 @@ q "CREATE TABLE vr (ts timestamptz, v int) USING pgcolumnar; q "VACUUM vr;" >/dev/null check "after VACUUM retires a fully deleted group the extent is the live rows'" "$(ext vr ts)" "$(truth vr ts)" +# recluster rewrites every group too, and the changelog names it with the three +# above (#1341 item 75): the rows a DELETE removed are not written back, so its +# extent must be the live rows', not wider by rows no longer stored. The deleted +# rows hold the minimum, so a stale extent keeps it. +q "CREATE TABLE rc (ts timestamptz, v int) USING pgcolumnar; + SELECT pgcolumnar.set_options('rc', stripe_row_limit => 1000, chunk_group_row_limit => 1000); + INSERT INTO rc SELECT '2026-01-01'::timestamptz + g * interval '1 min', (g * 7919) % 3001 FROM generate_series(1, 3000) g; + DELETE FROM rc WHERE ts < '2026-01-01'::timestamptz + interval '400 min';" >/dev/null +rc_before="$(ext rc ts)" +reclustered="$(q "SELECT pgcolumnar.recluster('rc', 'ts', 'v');")" +check "recluster rewrites the table's row groups" "$([ "${reclustered:-0}" -ge 1 ] && echo yes || echo "no ($reclustered)")" "yes" +check "after recluster the extent is the live rows'" "$(ext rc ts)" "$(truth rc ts)" +check "and it did move: the deleted rows held the old minimum" \ + "$([ -n "$rc_before" ] && [ "$rc_before" != "$(ext rc ts)" ] && echo yes || echo "no ($rc_before)")" "yes" + # ---- no complete extent where the zone maps cannot give one ---------------------- q "CREATE TABLE ac (ts timestamptz) USING pgcolumnar; SELECT pgcolumnar.set_options('ac', stripe_row_limit => 1000, chunk_group_row_limit => 1000); diff --git a/test/pytest/test_column_extent.py b/test/pytest/test_column_extent.py index add40cc65..f0074520a 100644 --- a/test/pytest/test_column_extent.py +++ b/test/pytest/test_column_extent.py @@ -247,6 +247,22 @@ def test_a_shrink_is_recomputed_never_guessed(pgc_own_db, expect): expect.text(_ext(conn, "vr", "ts"), _truth(conn, "vr", "ts"), "after VACUUM retires a fully deleted group the extent is the live rows'") + # recluster, named by the changelog with the three above (#1341 item 75): the + # deleted rows are not written back, and they held the minimum. + _run(conn, "CREATE TABLE rc (ts timestamptz, v int) USING pgcolumnar") + _small_groups(conn, "rc") + _run(conn, "INSERT INTO rc SELECT '2025-05-05'::timestamptz + g * interval '3 min', (g * 104729) % 2711" + " FROM generate_series(1, 3600) g", + "DELETE FROM rc WHERE ts < '2025-05-05'::timestamptz + interval '1500 min'") + before = _ext(conn, "rc", "ts") + reclustered = _one(conn, "SELECT pgcolumnar.recluster('rc', 'ts', 'v')") + expect.text("yes" if (reclustered or 0) >= 1 else f"no ({reclustered})", "yes", + "recluster rewrites the table's row groups") + after = _ext(conn, "rc", "ts") + expect.text(after, _truth(conn, "rc", "ts"), "after recluster the extent is the live rows'") + expect.text("yes" if before and before != after else f"no ({before})", "yes", + "and it did move: the deleted rows held the old minimum") + def test_no_complete_extent_where_the_zone_maps_cannot_give_one(pgc_own_db, expect): conn = pgc_own_db From 639fc193b96d885c0603bf36d71d2c11e6dbd96f Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Thu, 15 Oct 2026 00:51:14 +0000 Subject: [PATCH 3/6] test: the recluster extent arms enter the ledger, dated by the retirement-mark mutation (#1341 item 75) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho --- test/check_ledger.tsv | 11 +++++++---- test/check_ledger_budget.txt | 14 +++++++++++++- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/test/check_ledger.tsv b/test/check_ledger.tsv index 694fb5d6e..54b8eaed2 100644 --- a/test/check_ledger.tsv +++ b/test/check_ledger.tsv @@ -393,11 +393,13 @@ column_extent column_extent a transaction's own flushed groups are inside the ex column_extent column_extent a type with no ordering gets no extent 15;16;17;18;19 2026-10-13 a recompute that skips values without an ordering;pgcolumnar.column_extent returns a row with no complete extent column_extent column_extent a writer that finds no complete extent recomputes one over every group 15;16;17;18;19 2026-10-14 a writer that finds no complete extent does not recompute one;no extent row is ever written (PgColumnarExtentPreCommit returns at once) column_extent column_extent a writer that rolls back leaves the extent as the data has it 15;16;17;18;19 2026-10-14 a writer that finds no complete extent does not recompute one;no extent row is ever written (PgColumnarExtentPreCommit returns at once);no extent row written at pre-commit;no extent rows written at PRE_PREPARE;the writer's fold keeps the largest minimum -column_extent column_extent after VACUUM retires a fully deleted group the extent is the live rows' 15;16;17;18;19 2026-10-14 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;no extent row is ever written (PgColumnarExtentPreCommit returns at once) -column_extent column_extent after compact_rewrite the extent is the live rows' 15;16;17;18;19 2026-10-14 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;compact_rewrite returning 0 before rewriting anything;no extent row is ever written (PgColumnarExtentPreCommit returns at once) -column_extent column_extent after expire the extent is recomputed to the data that is left 15;16;17;18;19 2026-10-14 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;no extent row is ever written (PgColumnarExtentPreCommit returns at once) +column_extent column_extent after VACUUM retires a fully deleted group the extent is the live rows' 15;16;17;18;19 2026-10-15 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;no extent row is ever written (PgColumnarExtentPreCommit returns at once) +column_extent column_extent after compact_rewrite the extent is the live rows' 15;16;17;18;19 2026-10-15 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;compact_rewrite returning 0 before rewriting anything;no extent row is ever written (PgColumnarExtentPreCommit returns at once) +column_extent column_extent after expire the extent is recomputed to the data that is left 15;16;17;18;19 2026-10-15 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;no extent row is ever written (PgColumnarExtentPreCommit returns at once) +column_extent column_extent after recluster the extent is the live rows' 15;16;17;18;19 2026-10-15 PgColumnarRetireGroup without marking the storage for a recompute column_extent column_extent and a plan made in that transaction estimates as the zone maps do 15;16;17;18;19 2026-10-13 the extent a transaction reads leaves out its own pending fold (the tree before the fix) -column_extent column_extent and it did move: the retired groups held the old minimum 15;16;17;18;19 2026-10-14 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;no extent row is ever written (PgColumnarExtentPreCommit returns at once) +column_extent column_extent and it did move: the deleted rows held the old minimum 15;16;17;18;19 2026-10-15 PgColumnarRetireGroup without marking the storage for a recompute +column_extent column_extent and it did move: the retired groups held the old minimum 15;16;17;18;19 2026-10-15 PgColumnarRetireGroup without marking the storage for a recompute;a recompute that leaves the rows it folded;no extent row is ever written (PgColumnarExtentPreCommit returns at once) column_extent column_extent and the estimate is the one the zone maps give 15;16;17;18;19 2026-10-13 the planner takes the extent's minimum as its upper bound too;the planner's zone-map fallback removed: with no complete extent it reads nothing and bounds nothing column_extent column_extent and the extent covers what they loaded 15;16;17;18;19 2026-10-14 a writer that finds no complete extent does not recompute one;no extent row is ever written (PgColumnarExtentPreCommit returns at once);no extent row written at pre-commit;no extent rows written at PRE_PREPARE;the writer's fold keeps the largest minimum column_extent column_extent and the extent still covers all fifty 15;16;17;18;19 2026-10-14 a writer that finds no complete extent does not recompute one;no extent row is ever written (PgColumnarExtentPreCommit returns at once);no extent row written at pre-commit @@ -414,6 +416,7 @@ column_extent column_extent planning a bucketed GROUP BY scans zone_map not at a column_extent column_extent premise: the extent function exists 15;16;17;18;19 never - column_extent column_extent premise: the transaction's plans gave two estimates 15;16;17;18;19 never - column_extent column_extent premise: the typed fixture spans several row groups 15;16;17;18;19 never - +column_extent column_extent recluster rewrites the table's row groups 15;16;17;18;19 never - column_extent column_extent the extent catalog is not readable by an ordinary role (it holds column values) 15;16;17;18;19 2026-10-13 the column_extent catalog granted to PUBLIC column_extent column_extent the extent of ty.d is the data's min and max, in the type's order 15;16;17;18;19 2026-10-14 a writer that finds no complete extent does not recompute one;no extent row is ever written (PgColumnarExtentPreCommit returns at once) column_extent column_extent the extent of ty.i is the data's min and max, in the type's order 15;16;17;18;19 2026-10-14 a writer that finds no complete extent does not recompute one;no extent row is ever written (PgColumnarExtentPreCommit returns at once) diff --git a/test/check_ledger_budget.txt b/test/check_ledger_budget.txt index cea363447..e8678ba9d 100644 --- a/test/check_ledger_budget.txt +++ b/test/check_ledger_budget.txt @@ -2533,4 +2533,16 @@ suites_not_covered 230 # never 2344 + 1 = 2345 # dated 767 + 3 = 770 # 2345 + 770 == 3115 -checks_never_observed_red 2345 +# +# #1341 item 75, the changelog names recluster with expire, compact_rewrite and +# VACUUM as recomputing the column extent, and only the other three had arms: +# 2345 -> 2346. column_extent gains 3 checks from five-major runs (42/42 on each +# of 15-19). Two are dated on PG17 by the existing mutation "PgColumnarRetireGroup +# without marking the storage for a recompute", which also re-dates the four +# arms it already held. "recluster rewrites the table's row groups" arrives never. +# +# ledger rows 3115 + 3 = 3118 +# never 2345 + 1 = 2346 +# dated 770 + 2 = 772 +# 2346 + 772 == 3118 +checks_never_observed_red 2346 From 14f6aeb94c6e29ee04673f74e5cb809b3fa81975 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Wed, 14 Oct 2026 16:05:52 +0000 Subject: [PATCH 4/6] test: tier_partition and partition_create_ahead run on the caller's own privileges, in both twins (#1341) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho --- test/partition_lifecycle.sh | 31 ++++++++++++++ test/pytest/TESTS.md | 2 + test/pytest/expected_tests.txt | 4 +- test/pytest/test_partition_lifecycle.py | 38 +++++++++++++++++ test/pytest/test_tier_partition.py | 54 +++++++++++++++++++++++++ test/tier_partition.sh | 44 ++++++++++++++++++++ 6 files changed, 172 insertions(+), 1 deletion(-) diff --git a/test/partition_lifecycle.sh b/test/partition_lifecycle.sh index 9c649e886..5eb52be5f 100755 --- a/test/partition_lifecycle.sh +++ b/test/partition_lifecycle.sh @@ -241,6 +241,37 @@ check "a parent with a DEFAULT partition is refused" \ check "and nothing was created for it" \ "$(kids td)|$(q "SELECT count(*) FROM pg_proc WHERE proname = 'partition_create_ahead';")" "1|1" +# ---- the caller's own privileges (#1341) ------------------------------------- +# +# partition_create_ahead is not SECURITY DEFINER, so its CREATE TABLE ... PARTITION OF +# runs as the caller, and core refuses a role that does not own the parent. Made +# SECURITY DEFINER, it would run as the extension's owner and let any caller add +# partitions, and the access-exclusive lock that comes with them, to a table it does +# not own. Both roles hold USAGE and CREATE on the schema, so ownership is all that +# separates them. +for r in pl_own pl_peer; do + psql_run "DROP ROLE IF EXISTS $r; CREATE ROLE $r LOGIN NOSUPERUSER; + GRANT USAGE, CREATE ON SCHEMA public TO $r; GRANT USAGE ON SCHEMA pgcolumnar TO $r;" +done +as_role() { # as_role ROLE SQL -> the single value SQL returns as ROLE, or the SQLSTATE it fails with + local out + out=$(env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U "$1" -d "$PGC_DB" \ + -X -At -v ON_ERROR_STOP=1 -v VERBOSITY=sqlstate -c "$2" 2>&1) + case "$out" in + *ERROR:*|*FATAL:*) sed -n 's/^\(ERROR\|FATAL\): *\([0-9A-Z]\{5\}\).*/\2/p' <<<"$out" | head -1 ;; + *) printf '%s\n' "$out" | tail -1 ;; + esac +} +q "CREATE TABLE pv (ts timestamptz NOT NULL, v int) PARTITION BY RANGE (ts); ALTER TABLE pv OWNER TO pl_own;" >/dev/null +check "premise: pl_own owns the parent, and neither role is a superuser" \ + "$(q "SELECT pg_get_userbyid(relowner) FROM pg_class WHERE oid = 'pv'::regclass;")|$(q "SELECT count(*) FROM pg_roles WHERE rolname IN ('pl_own', 'pl_peer') AND rolsuper;")" \ + "pl_own|0" +check "premise: pl_peer can open a session" "$(as_role pl_peer "SELECT 1")" "1" +check "a role that does not own the parent is refused by its own privileges (42501), and creates nothing" \ + "$(as_role pl_peer "SELECT pgcolumnar.partition_create_ahead('pv', '1 day', 2)")|$(kids pv)" "42501|0" +check "a non-superuser owner creates partitions ahead on its own table" \ + "$(as_role pl_own "SELECT pgcolumnar.partition_create_ahead('pv', '1 day', 2)")|$(kids pv)" "3|3" + # ==== the drop half: partition_drop_plan (#1297) ================================ # # partition_drop_plan(parent regclass, cutoff timestamptz) -> SETOF text diff --git a/test/pytest/TESTS.md b/test/pytest/TESTS.md index ed83fde0e..effcdd57c 100644 --- a/test/pytest/TESTS.md +++ b/test/pytest/TESTS.md @@ -6643,6 +6643,7 @@ Removal proofs, each red in both twins: the prototype's zone handling; `CREATE T | --- | --- | | `test_partition_create_ahead` | from nothing it reaches now plus the lead, columnar, contiguous and one width wide; a second call creates nothing, and asking further ahead creates only the difference; rows land in columnar partitions; the parent is never held ACCESS EXCLUSIVE; west of UTC, for both key types, partitions continue a hand-made one contiguously and a row at every boundary routes; east of UTC, a timestamp parent covers the local now and starts on the width's boundary; inputs that broke the first version (a generated column, a 63-character name, more than 1000 partitions, a MAXVALUE bound) are handled or refused before any DDL; a DEFAULT partition is refused and nothing is created | | `test_partition_drop_plan` | the drop half: `partition_drop_plan` lists `DETACH ... CONCURRENTLY` and `DROP` for the partitions wholly before the cutoff, in range order, from partitions created newest first. A cutoff on an upper bound includes the partition, and one before them all plans nothing. Running the plan, each statement top-level, leaves the rest with their rows and drops the storage rows. A `timestamp` key with a `timestamp` cutoff plans the same in Tokyo and Denver, and a cutoff of the other type is refused (`42804`). MINVALUE is listed once past, and MAXVALUE never. Quoted schema and partition names run as returned. A sub-partitioned partition is planned whole and dropped with its children. A tiered partition is planned with `DROP FOREIGN TABLE`, and its Parquet file stays. A partition whose name holds the drop block's dollar-quote tag drops as planned. A detach cancelled in its second phase is planned as `FINALIZE` and dropped. The DEFAULT and non-timestamp parents are refused (`55000`) | +| `test_partition_create_ahead_runs_on_the_callers_own_privileges` | not SECURITY DEFINER: a role that does not own the parent, holding USAGE and CREATE on the schema, is refused `42501` and creates nothing; a non-superuser owner creates its partitions ahead | ## 94. test_approx_aggregates.py: hll and tdigest over columnar tables, the compatibility statement @@ -6791,6 +6792,7 @@ On a private database, because foreign servers are per database. A local path st | `test_tier_partition_moves_a_partition_exactly_once` | the move returns the rows moved; the parent's answers are the same; the cold range is a foreign partition with the same name and bound, reading the exported file; no pgcolumnar catalog rows are left; pruning keeps the ranges apart; an insert into the tiered range is refused; five refusals move nothing; the parent is held ACCESS EXCLUSIVE until commit, and a rollback leaves the rows local; a write in flight is exported; a fault after each of the five steps leaves the rows exactly once, still local | | `test_tier_partition_refuses_what_it_cannot_move_and_survives_concurrency` | at each isolation level a row committed between the snapshot and the lock survives, and only READ COMMITTED moves; a UNIQUE parent and four kinds of dependent are refused with no file left, and the partition moves once they are gone; grants and comment carry over; a racing insert leaves every row once; lock_wait times a held parent out with nothing attached; the caller's lock_timeout survives; a retry over a failed attempt's file moves the partition | | `test_tier_partition_moves_a_virtual_generated_column` | PostgreSQL 18+: a partition whose parent has a virtual generated column moves and reads back the same; below 18, `UNSUPPORTED_MAJOR`, listed in `expected_unrunnable.txt` | +| `test_tier_partition_runs_on_the_callers_own_privileges` | not SECURITY DEFINER: a role that does not own the table, holding the server-file role, USAGE on the server, CREATE on the schema and SELECT on the partition, is refused `42501`, and the partition stays local and attached with no foreign table behind it; a non-superuser owner tiers its own partition | ## 101. test_prepared_writes.py: a write prepared and finished in one session leaves the backend alive diff --git a/test/pytest/expected_tests.txt b/test/pytest/expected_tests.txt index 8e812d734..cb3b19a84 100644 --- a/test/pytest/expected_tests.txt +++ b/test/pytest/expected_tests.txt @@ -705,4 +705,6 @@ guard_tests 438 # cluster_tests 586 -> 587 for #1341's test_serializable.py, one test, by collection # cluster_tests 587 -> 588 for #1341's daemon TimeZone test in test_autovacuum_expire.py, # by collection over the cluster files on this tree. -cluster_tests 588 +# cluster_tests 588 -> 590 for #1341's caller-privilege tests in test_tier_partition.py and +# test_partition_lifecycle.py, one each, by collection over the cluster files on this tree. +cluster_tests 590 diff --git a/test/pytest/test_partition_lifecycle.py b/test/pytest/test_partition_lifecycle.py index c7bcad322..814a912c2 100644 --- a/test/pytest/test_partition_lifecycle.py +++ b/test/pytest/test_partition_lifecycle.py @@ -595,3 +595,41 @@ def name(t): for t in ("xnl", "xls", "xr2")), "|".join(f"55000 {name(t)} is not RANGE-partitioned on one column" for t in ("xnl", "xls", "xr2")), "a drop plan for a parent not RANGE-partitioned on one column is refused, LIST and two-column RANGE included") + + +def test_partition_create_ahead_runs_on_the_callers_own_privileges(pgc_conn, pgc_cluster, expect): + """partition_create_ahead is not SECURITY DEFINER, so its CREATE TABLE ... PARTITION + OF runs as the caller and core refuses a role that does not own the parent (#1341). + Made SECURITY DEFINER, it would run as the extension's owner and let any caller add + partitions to a table it does not own. Both roles hold USAGE and CREATE on the + schema, so ownership is all that separates them.""" + conn = pgc_conn + schema = _one(conn, "SELECT current_schema()") + with conn.cursor() as cur: + for r in ("pl_own_py", "pl_peer_py"): + cur.execute(f"DO $$ BEGIN CREATE ROLE {r} LOGIN NOSUPERUSER; " + "EXCEPTION WHEN duplicate_object THEN NULL; END $$") + cur.execute(f'GRANT USAGE, CREATE ON SCHEMA "{schema}" TO {r}') + cur.execute(f"GRANT USAGE ON SCHEMA pgcolumnar TO {r}") + cur.execute("CREATE TABLE pv (ts timestamptz NOT NULL, v int) PARTITION BY RANGE (ts)") + cur.execute("ALTER TABLE pv OWNER TO pl_own_py") + + def as_role(role, sql): + try: + with psycopg.connect(f"host=127.0.0.1 port={pgc_cluster.port} user={role} dbname=postgres", + autocommit=True) as c: + c.execute(f'SET search_path TO "{schema}", pgcolumnar, public') + return str(c.execute(sql).fetchone()[0]) + except psycopg.Error as e: + return e.sqlstate or "?" + + ahead_pv = "SELECT pgcolumnar.partition_create_ahead('pv', '1 day', 2)" + owner = _one(conn, "SELECT pg_get_userbyid(relowner) FROM pg_class WHERE oid = 'pv'::regclass") + supers = _one(conn, "SELECT count(*) FROM pg_roles WHERE rolname IN ('pl_own_py', 'pl_peer_py') AND rolsuper") + expect.text(f"{owner}|{supers}", "pl_own_py|0", + "premise: pl_own owns the parent, and neither role is a superuser") + expect.text(as_role("pl_peer_py", "SELECT 1"), "1", "premise: pl_peer can open a session") + expect.text(f"{as_role('pl_peer_py', ahead_pv)}|{_kids(conn, 'pv')}", + "42501|0", "a role that does not own the parent is refused by its own privileges (42501), and creates nothing") + expect.text(f"{as_role('pl_own_py', ahead_pv)}|{_kids(conn, 'pv')}", + "3|3", "a non-superuser owner creates partitions ahead on its own table") diff --git a/test/pytest/test_tier_partition.py b/test/pytest/test_tier_partition.py index ad306382a..b994d3480 100644 --- a/test/pytest/test_tier_partition.py +++ b/test/pytest/test_tier_partition.py @@ -530,3 +530,57 @@ def test_tier_partition_moves_a_virtual_generated_column(pgc_own_db, server_dir, kind = _one(conn, "SELECT relkind::text FROM pg_class WHERE oid = 'cv_mar'::regclass") expect.text(f"{moved}|{kind}|{'same' if before and before == _one(conn, digest) else 'differs'}", "71|f|same", "a virtual generated column moves and reads back the same") + + +def test_tier_partition_runs_on_the_callers_own_privileges(pgc_own_db, pgc_cluster, server_dir, expect): + """tier_partition is not SECURITY DEFINER, so a role that does not own the table is + refused by its own privileges (#1341). Made SECURITY DEFINER, it would run as the + extension's owner and let any caller replace a partition it does not own. Both roles + hold every other privilege the call needs, so ownership is all that separates them.""" + conn = pgc_own_db + db = _one(conn, "SELECT current_database()") + with conn.cursor() as cur: + cur.execute("SET timezone = 'UTC'") + cur.execute("CREATE SERVER tq FOREIGN DATA WRAPPER pgcolumnar_parquet") + for r in ("tier_own_py", "tier_peer_py"): + cur.execute(f"DO $$ BEGIN CREATE ROLE {r} LOGIN NOSUPERUSER; " + "EXCEPTION WHEN duplicate_object THEN NULL; END $$") + cur.execute(f"GRANT pg_write_server_files TO {r}") + cur.execute(f"GRANT USAGE ON FOREIGN SERVER tq TO {r}") + cur.execute(f"GRANT USAGE, CREATE ON SCHEMA public TO {r}") + cur.execute(f"GRANT USAGE ON SCHEMA pgcolumnar TO {r}") + _build(conn, "po") + with conn.cursor() as cur: + for t in ("po", "po_mar", "po_apr"): + cur.execute(f"ALTER TABLE {t} OWNER TO tier_own_py") + cur.execute("GRANT SELECT ON po, po_mar TO tier_peer_py") + before = _digest(conn, "po") + + def as_role(role, sql): + try: + with psycopg.connect(f"host=127.0.0.1 port={pgc_cluster.port} user={role} dbname={db}", + autocommit=True) as c: + c.execute("SET timezone = 'UTC'") + c.execute(sql) + return "ok" + except psycopg.Error as e: + return e.sqlstate or "?" + + owners = _one(conn, "SELECT string_agg(DISTINCT pg_get_userbyid(relowner), ',') FROM pg_class" + " WHERE relname IN ('po', 'po_mar', 'po_apr')") + supers = _one(conn, "SELECT count(*) FROM pg_roles WHERE rolname IN ('tier_own_py', 'tier_peer_py') AND rolsuper") + expect.text(f"{owners}|{supers}", "tier_own_py|0", + "premise: tier_own owns the parent and its partitions, and neither role is a superuser") + expect.text(as_role("tier_peer_py", "SELECT count(*) FROM po_mar"), "ok", + "premise: tier_peer can open a session and read the partition") + peer_path = server_dir / "po_peer.parquet" + expect.text(as_role("tier_peer_py", f"SELECT pgcolumnar.tier_partition('po_mar', 'tq', '{peer_path}')"), + "42501", "a role that does not own the table is refused by its own privileges (42501)") + expect.text(f"{_digest(conn, 'po')}|{_mar_shape(conn, 'po')}", f"{before}|r:pgcolumnar:{BOUND}", + "and the refused call leaves the partition local and attached, every row once") + expect.num(_one(conn, "SELECT count(*) FROM pg_foreign_table t JOIN pg_class c ON c.oid = t.ftrelid" + " WHERE c.relname LIKE 'po\\_%'"), 0, "with no foreign table behind it") + print(f"-- the refused call's export file: {'written' if peer_path.exists() else 'absent'}") + expect.text(as_role("tier_own_py", f"SELECT pgcolumnar.tier_partition('po_mar', 'tq', '{server_dir / 'po_own.parquet'}')") + + f"|{_mar_shape(conn, 'po')}", f"ok|f:-:{BOUND}", + "a non-superuser owner tiers its own partition") diff --git a/test/tier_partition.sh b/test/tier_partition.sh index 03b3540d8..e543a5e3a 100755 --- a/test/tier_partition.sh +++ b/test/tier_partition.sh @@ -399,6 +399,50 @@ else "PostgreSQL $PGC_MAJOR has no virtual generated columns" fi +# ---- the caller's own privileges (#1341) ------------------------------------- +# +# tier_partition is not SECURITY DEFINER: every step runs as the caller, so the +# owner's ALTER TABLE ... DETACH is what stops a role that does not own the table. +# Made SECURITY DEFINER, it would run as the extension's owner and let any caller +# detach, drop and replace a partition it does not own. Two roles, each holding +# every other privilege the call needs (the server-file role for the export, USAGE +# on the server and CREATE on the schema for the foreign table, and SELECT on the +# partition), so ownership is the only thing between them. +for r in tier_own tier_peer; do + psql_run "DROP ROLE IF EXISTS $r; CREATE ROLE $r LOGIN NOSUPERUSER; + GRANT pg_write_server_files TO $r; GRANT USAGE ON FOREIGN SERVER tier_pq TO $r; + GRANT USAGE, CREATE ON SCHEMA public TO $r; GRANT USAGE ON SCHEMA pgcolumnar TO $r;" +done +sqlstate_as() { # sqlstate_as ROLE SQL -> the SQLSTATE SQL fails with as ROLE, or ok + local out + out=$(env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U "$1" -d "$PGC_DB" \ + -X -At -v ON_ERROR_STOP=1 -v VERBOSITY=sqlstate -c "$2" 2>&1 >/dev/null) + [ -z "$out" ] && { echo ok; return; } + echo "$out" | sed -n 's/^\(ERROR\|FATAL\): *\([0-9A-Z]\{5\}\).*/\2/p' | head -1 +} +build po +psql_run "ALTER TABLE po OWNER TO tier_own; ALTER TABLE po_jan OWNER TO tier_own; + ALTER TABLE po_feb OWNER TO tier_own; GRANT SELECT ON po, po_jan TO tier_peer;" +pb="$(digest po)" +check "premise: tier_own owns the parent and its partitions, and neither role is a superuser" \ + "$(q "SELECT string_agg(DISTINCT pg_get_userbyid(relowner), ',') FROM pg_class WHERE relname IN ('po', 'po_jan', 'po_feb');")|$(q "SELECT count(*) FROM pg_roles WHERE rolname IN ('tier_own', 'tier_peer') AND rolsuper;")" \ + "tier_own|0" +check "premise: tier_peer can open a session and read the partition" \ + "$(sqlstate_as tier_peer "SELECT count(*) FROM po_jan")" "ok" +check "a role that does not own the table is refused by its own privileges (42501)" \ + "$(sqlstate_as tier_peer "SELECT pgcolumnar.tier_partition('po_jan', 'tier_pq', '$W/po_peer.parquet')")" "42501" +check "and the refused call leaves the partition local and attached, every row once" \ + "$(digest po)|$(jan_shape po)" "$pb|r:pgcolumnar:$JAN_BOUND" +check_num "with no foreign table behind it" \ + "$(q "SELECT count(*) FROM pg_foreign_table t JOIN pg_class c ON c.oid = t.ftrelid WHERE c.relname LIKE 'po\\_%';")" "0" +# Measured, not asserted: the export runs before the DETACH that refuses, so the +# file is written first. A role holding the server-file role and SELECT can write +# that file with export_parquet directly, so this grants nothing new. +echo "-- the refused call's export file: $([ -e "$W/po_peer.parquet" ] && echo written || echo absent)" +check "a non-superuser owner tiers its own partition" \ + "$(sqlstate_as tier_own "SELECT pgcolumnar.tier_partition('po_jan', 'tier_pq', '$W/po_own.parquet')")|$(jan_shape po)" \ + "ok|f:-:$JAN_BOUND" + # ---- a failure after each step ---------------------------------------------- for step in export detach drop create attach; do build f From a4b0ac6be4f89106e11baeaaaf32d6c49ef0871e Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Wed, 14 Oct 2026 16:15:14 +0000 Subject: [PATCH 5/6] test: the caller-privilege owner controls get tables of their own, and the refused tier writes no file, in both twins (#1341) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho --- test/partition_lifecycle.sh | 9 ++++++--- test/pytest/TESTS.md | 4 ++-- test/pytest/test_partition_lifecycle.py | 13 +++++++++---- test/pytest/test_tier_partition.py | 20 ++++++++++++------- test/tier_partition.sh | 26 +++++++++++++++---------- 5 files changed, 46 insertions(+), 26 deletions(-) diff --git a/test/partition_lifecycle.sh b/test/partition_lifecycle.sh index 5eb52be5f..c8c89a1d9 100755 --- a/test/partition_lifecycle.sh +++ b/test/partition_lifecycle.sh @@ -262,15 +262,18 @@ as_role() { # as_role ROLE SQL -> the single value SQL returns as ROLE, or the S *) printf '%s\n' "$out" | tail -1 ;; esac } -q "CREATE TABLE pv (ts timestamptz NOT NULL, v int) PARTITION BY RANGE (ts); ALTER TABLE pv OWNER TO pl_own;" >/dev/null +# The owner's control has a parent of its own (pw2), so a refusal that let the peer +# through cannot make the control fail for the peer's reason. +q "CREATE TABLE pv (ts timestamptz NOT NULL, v int) PARTITION BY RANGE (ts); ALTER TABLE pv OWNER TO pl_own; + CREATE TABLE pw2 (ts timestamptz NOT NULL, v int) PARTITION BY RANGE (ts); ALTER TABLE pw2 OWNER TO pl_own;" >/dev/null check "premise: pl_own owns the parent, and neither role is a superuser" \ - "$(q "SELECT pg_get_userbyid(relowner) FROM pg_class WHERE oid = 'pv'::regclass;")|$(q "SELECT count(*) FROM pg_roles WHERE rolname IN ('pl_own', 'pl_peer') AND rolsuper;")" \ + "$(q "SELECT string_agg(DISTINCT pg_get_userbyid(relowner), ',') FROM pg_class WHERE oid IN ('pv'::regclass, 'pw2'::regclass);")|$(q "SELECT count(*) FROM pg_roles WHERE rolname IN ('pl_own', 'pl_peer') AND rolsuper;")" \ "pl_own|0" check "premise: pl_peer can open a session" "$(as_role pl_peer "SELECT 1")" "1" check "a role that does not own the parent is refused by its own privileges (42501), and creates nothing" \ "$(as_role pl_peer "SELECT pgcolumnar.partition_create_ahead('pv', '1 day', 2)")|$(kids pv)" "42501|0" check "a non-superuser owner creates partitions ahead on its own table" \ - "$(as_role pl_own "SELECT pgcolumnar.partition_create_ahead('pv', '1 day', 2)")|$(kids pv)" "3|3" + "$(as_role pl_own "SELECT pgcolumnar.partition_create_ahead('pw2', '1 day', 2)")|$(kids pw2)" "3|3" # ==== the drop half: partition_drop_plan (#1297) ================================ # diff --git a/test/pytest/TESTS.md b/test/pytest/TESTS.md index effcdd57c..bb1d13ebb 100644 --- a/test/pytest/TESTS.md +++ b/test/pytest/TESTS.md @@ -6643,7 +6643,7 @@ Removal proofs, each red in both twins: the prototype's zone handling; `CREATE T | --- | --- | | `test_partition_create_ahead` | from nothing it reaches now plus the lead, columnar, contiguous and one width wide; a second call creates nothing, and asking further ahead creates only the difference; rows land in columnar partitions; the parent is never held ACCESS EXCLUSIVE; west of UTC, for both key types, partitions continue a hand-made one contiguously and a row at every boundary routes; east of UTC, a timestamp parent covers the local now and starts on the width's boundary; inputs that broke the first version (a generated column, a 63-character name, more than 1000 partitions, a MAXVALUE bound) are handled or refused before any DDL; a DEFAULT partition is refused and nothing is created | | `test_partition_drop_plan` | the drop half: `partition_drop_plan` lists `DETACH ... CONCURRENTLY` and `DROP` for the partitions wholly before the cutoff, in range order, from partitions created newest first. A cutoff on an upper bound includes the partition, and one before them all plans nothing. Running the plan, each statement top-level, leaves the rest with their rows and drops the storage rows. A `timestamp` key with a `timestamp` cutoff plans the same in Tokyo and Denver, and a cutoff of the other type is refused (`42804`). MINVALUE is listed once past, and MAXVALUE never. Quoted schema and partition names run as returned. A sub-partitioned partition is planned whole and dropped with its children. A tiered partition is planned with `DROP FOREIGN TABLE`, and its Parquet file stays. A partition whose name holds the drop block's dollar-quote tag drops as planned. A detach cancelled in its second phase is planned as `FINALIZE` and dropped. The DEFAULT and non-timestamp parents are refused (`55000`) | -| `test_partition_create_ahead_runs_on_the_callers_own_privileges` | not SECURITY DEFINER: a role that does not own the parent, holding USAGE and CREATE on the schema, is refused `42501` and creates nothing; a non-superuser owner creates its partitions ahead | +| `test_partition_create_ahead_runs_on_the_callers_own_privileges` | not SECURITY DEFINER: a role that does not own the parent, holding USAGE and CREATE on the schema, is refused `42501` and creates nothing; a non-superuser owner creates partitions ahead on a parent of its own | ## 94. test_approx_aggregates.py: hll and tdigest over columnar tables, the compatibility statement @@ -6792,7 +6792,7 @@ On a private database, because foreign servers are per database. A local path st | `test_tier_partition_moves_a_partition_exactly_once` | the move returns the rows moved; the parent's answers are the same; the cold range is a foreign partition with the same name and bound, reading the exported file; no pgcolumnar catalog rows are left; pruning keeps the ranges apart; an insert into the tiered range is refused; five refusals move nothing; the parent is held ACCESS EXCLUSIVE until commit, and a rollback leaves the rows local; a write in flight is exported; a fault after each of the five steps leaves the rows exactly once, still local | | `test_tier_partition_refuses_what_it_cannot_move_and_survives_concurrency` | at each isolation level a row committed between the snapshot and the lock survives, and only READ COMMITTED moves; a UNIQUE parent and four kinds of dependent are refused with no file left, and the partition moves once they are gone; grants and comment carry over; a racing insert leaves every row once; lock_wait times a held parent out with nothing attached; the caller's lock_timeout survives; a retry over a failed attempt's file moves the partition | | `test_tier_partition_moves_a_virtual_generated_column` | PostgreSQL 18+: a partition whose parent has a virtual generated column moves and reads back the same; below 18, `UNSUPPORTED_MAJOR`, listed in `expected_unrunnable.txt` | -| `test_tier_partition_runs_on_the_callers_own_privileges` | not SECURITY DEFINER: a role that does not own the table, holding the server-file role, USAGE on the server, CREATE on the schema and SELECT on the partition, is refused `42501`, and the partition stays local and attached with no foreign table behind it; a non-superuser owner tiers its own partition | +| `test_tier_partition_runs_on_the_callers_own_privileges` | not SECURITY DEFINER: a role that does not own the table, holding the server-file role, USAGE on the server, CREATE on the schema and SELECT on the partition, is refused `42501` before any file is written, and the partition stays local and attached with no foreign table behind it; a non-superuser owner tiers a partition of its own table | ## 101. test_prepared_writes.py: a write prepared and finished in one session leaves the backend alive diff --git a/test/pytest/test_partition_lifecycle.py b/test/pytest/test_partition_lifecycle.py index 814a912c2..67791e7c0 100644 --- a/test/pytest/test_partition_lifecycle.py +++ b/test/pytest/test_partition_lifecycle.py @@ -611,8 +611,11 @@ def test_partition_create_ahead_runs_on_the_callers_own_privileges(pgc_conn, pgc "EXCEPTION WHEN duplicate_object THEN NULL; END $$") cur.execute(f'GRANT USAGE, CREATE ON SCHEMA "{schema}" TO {r}') cur.execute(f"GRANT USAGE ON SCHEMA pgcolumnar TO {r}") - cur.execute("CREATE TABLE pv (ts timestamptz NOT NULL, v int) PARTITION BY RANGE (ts)") - cur.execute("ALTER TABLE pv OWNER TO pl_own_py") + # The owner's control has a parent of its own (pw2), so a refusal that let the + # peer through cannot make the control fail for the peer's reason. + for t in ("pv", "pw2"): + cur.execute(f"CREATE TABLE {t} (ts timestamptz NOT NULL, v int) PARTITION BY RANGE (ts)") + cur.execute(f"ALTER TABLE {t} OWNER TO pl_own_py") def as_role(role, sql): try: @@ -624,12 +627,14 @@ def as_role(role, sql): return e.sqlstate or "?" ahead_pv = "SELECT pgcolumnar.partition_create_ahead('pv', '1 day', 2)" - owner = _one(conn, "SELECT pg_get_userbyid(relowner) FROM pg_class WHERE oid = 'pv'::regclass") + owner = _one(conn, "SELECT string_agg(DISTINCT pg_get_userbyid(relowner), ',') FROM pg_class" + " WHERE oid IN ('pv'::regclass, 'pw2'::regclass)") supers = _one(conn, "SELECT count(*) FROM pg_roles WHERE rolname IN ('pl_own_py', 'pl_peer_py') AND rolsuper") expect.text(f"{owner}|{supers}", "pl_own_py|0", "premise: pl_own owns the parent, and neither role is a superuser") expect.text(as_role("pl_peer_py", "SELECT 1"), "1", "premise: pl_peer can open a session") expect.text(f"{as_role('pl_peer_py', ahead_pv)}|{_kids(conn, 'pv')}", "42501|0", "a role that does not own the parent is refused by its own privileges (42501), and creates nothing") - expect.text(f"{as_role('pl_own_py', ahead_pv)}|{_kids(conn, 'pv')}", + ahead_pw2 = "SELECT pgcolumnar.partition_create_ahead('pw2', '1 day', 2)" + expect.text(f"{as_role('pl_own_py', ahead_pw2)}|{_kids(conn, 'pw2')}", "3|3", "a non-superuser owner creates partitions ahead on its own table") diff --git a/test/pytest/test_tier_partition.py b/test/pytest/test_tier_partition.py index b994d3480..929b99a17 100644 --- a/test/pytest/test_tier_partition.py +++ b/test/pytest/test_tier_partition.py @@ -549,10 +549,14 @@ def test_tier_partition_runs_on_the_callers_own_privileges(pgc_own_db, pgc_clust cur.execute(f"GRANT USAGE ON FOREIGN SERVER tq TO {r}") cur.execute(f"GRANT USAGE, CREATE ON SCHEMA public TO {r}") cur.execute(f"GRANT USAGE ON SCHEMA pgcolumnar TO {r}") - _build(conn, "po") + # The owner's control has a table of its own (pw), so a refusal that let the peer + # through cannot make the control fail for the peer's reason. + for t in ("po", "pw"): + _build(conn, t) + with conn.cursor() as cur: + for r in (t, f"{t}_mar", f"{t}_apr"): + cur.execute(f"ALTER TABLE {r} OWNER TO tier_own_py") with conn.cursor() as cur: - for t in ("po", "po_mar", "po_apr"): - cur.execute(f"ALTER TABLE {t} OWNER TO tier_own_py") cur.execute("GRANT SELECT ON po, po_mar TO tier_peer_py") before = _digest(conn, "po") @@ -567,7 +571,7 @@ def as_role(role, sql): return e.sqlstate or "?" owners = _one(conn, "SELECT string_agg(DISTINCT pg_get_userbyid(relowner), ',') FROM pg_class" - " WHERE relname IN ('po', 'po_mar', 'po_apr')") + " WHERE relname IN ('po', 'po_mar', 'po_apr', 'pw', 'pw_mar', 'pw_apr')") supers = _one(conn, "SELECT count(*) FROM pg_roles WHERE rolname IN ('tier_own_py', 'tier_peer_py') AND rolsuper") expect.text(f"{owners}|{supers}", "tier_own_py|0", "premise: tier_own owns the parent and its partitions, and neither role is a superuser") @@ -580,7 +584,9 @@ def as_role(role, sql): "and the refused call leaves the partition local and attached, every row once") expect.num(_one(conn, "SELECT count(*) FROM pg_foreign_table t JOIN pg_class c ON c.oid = t.ftrelid" " WHERE c.relname LIKE 'po\\_%'"), 0, "with no foreign table behind it") - print(f"-- the refused call's export file: {'written' if peer_path.exists() else 'absent'}") - expect.text(as_role("tier_own_py", f"SELECT pgcolumnar.tier_partition('po_mar', 'tq', '{server_dir / 'po_own.parquet'}')") - + f"|{_mar_shape(conn, 'po')}", f"ok|f:-:{BOUND}", + # The LOCK ... IN SHARE MODE ahead of the export is the first statement to need + # more than SELECT, so the refusal comes before any file is written. + expect.text("written" if peer_path.exists() else "absent", "absent", "and the refused call wrote no export file") + expect.text(as_role("tier_own_py", f"SELECT pgcolumnar.tier_partition('pw_mar', 'tq', '{server_dir / 'pw_own.parquet'}')") + + f"|{_mar_shape(conn, 'pw')}", f"ok|f:-:{BOUND}", "a non-superuser owner tiers its own partition") diff --git a/test/tier_partition.sh b/test/tier_partition.sh index e543a5e3a..186370e20 100755 --- a/test/tier_partition.sh +++ b/test/tier_partition.sh @@ -402,7 +402,8 @@ fi # ---- the caller's own privileges (#1341) ------------------------------------- # # tier_partition is not SECURITY DEFINER: every step runs as the caller, so the -# owner's ALTER TABLE ... DETACH is what stops a role that does not own the table. +# owner-only steps (its LOCK ... IN SHARE MODE first) stop a role that does not own +# the table. # Made SECURITY DEFINER, it would run as the extension's owner and let any caller # detach, drop and replace a partition it does not own. Two roles, each holding # every other privilege the call needs (the server-file role for the export, USAGE @@ -420,12 +421,17 @@ sqlstate_as() { # sqlstate_as ROLE SQL -> the SQLSTATE SQL fails with as ROLE, o [ -z "$out" ] && { echo ok; return; } echo "$out" | sed -n 's/^\(ERROR\|FATAL\): *\([0-9A-Z]\{5\}\).*/\2/p' | head -1 } -build po -psql_run "ALTER TABLE po OWNER TO tier_own; ALTER TABLE po_jan OWNER TO tier_own; - ALTER TABLE po_feb OWNER TO tier_own; GRANT SELECT ON po, po_jan TO tier_peer;" +# The owner's control has a table of its own (pw), so a refusal that let the peer +# through cannot make the control fail for the peer's reason. +for t in po pw; do + build $t + psql_run "ALTER TABLE $t OWNER TO tier_own; ALTER TABLE ${t}_jan OWNER TO tier_own; + ALTER TABLE ${t}_feb OWNER TO tier_own;" +done +psql_run "GRANT SELECT ON po, po_jan TO tier_peer;" pb="$(digest po)" check "premise: tier_own owns the parent and its partitions, and neither role is a superuser" \ - "$(q "SELECT string_agg(DISTINCT pg_get_userbyid(relowner), ',') FROM pg_class WHERE relname IN ('po', 'po_jan', 'po_feb');")|$(q "SELECT count(*) FROM pg_roles WHERE rolname IN ('tier_own', 'tier_peer') AND rolsuper;")" \ + "$(q "SELECT string_agg(DISTINCT pg_get_userbyid(relowner), ',') FROM pg_class WHERE relname IN ('po', 'po_jan', 'po_feb', 'pw', 'pw_jan', 'pw_feb');")|$(q "SELECT count(*) FROM pg_roles WHERE rolname IN ('tier_own', 'tier_peer') AND rolsuper;")" \ "tier_own|0" check "premise: tier_peer can open a session and read the partition" \ "$(sqlstate_as tier_peer "SELECT count(*) FROM po_jan")" "ok" @@ -435,12 +441,12 @@ check "and the refused call leaves the partition local and attached, every row o "$(digest po)|$(jan_shape po)" "$pb|r:pgcolumnar:$JAN_BOUND" check_num "with no foreign table behind it" \ "$(q "SELECT count(*) FROM pg_foreign_table t JOIN pg_class c ON c.oid = t.ftrelid WHERE c.relname LIKE 'po\\_%';")" "0" -# Measured, not asserted: the export runs before the DETACH that refuses, so the -# file is written first. A role holding the server-file role and SELECT can write -# that file with export_parquet directly, so this grants nothing new. -echo "-- the refused call's export file: $([ -e "$W/po_peer.parquet" ] && echo written || echo absent)" +# The first statement to need more than SELECT is the LOCK ... IN SHARE MODE ahead +# of the export, so the refusal comes before any file is written (measured on 15-19). +check "and the refused call wrote no export file" \ + "$([ -e "$W/po_peer.parquet" ] && echo written || echo absent)" "absent" check "a non-superuser owner tiers its own partition" \ - "$(sqlstate_as tier_own "SELECT pgcolumnar.tier_partition('po_jan', 'tier_pq', '$W/po_own.parquet')")|$(jan_shape po)" \ + "$(sqlstate_as tier_own "SELECT pgcolumnar.tier_partition('pw_jan', 'tier_pq', '$W/pw_own.parquet')")|$(jan_shape pw)" \ "ok|f:-:$JAN_BOUND" # ---- a failure after each step ---------------------------------------------- From bfa284c960061776b15ec263cae2937e03835cb6 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Wed, 14 Oct 2026 16:22:33 +0000 Subject: [PATCH 6/6] test: the caller-privilege arms enter the ledger, five dated by SECURITY DEFINER (#1341) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho --- test/check_ledger.tsv | 11 +++++++++++ test/check_ledger_budget.txt | 16 +++++++++++++++- 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/test/check_ledger.tsv b/test/check_ledger.tsv index 54b8eaed2..098b45e26 100644 --- a/test/check_ledger.tsv +++ b/test/check_ledger.tsv @@ -2399,6 +2399,7 @@ partition_lifecycle partition_lifecycle a cutoff on a partition's upper bound in partition_lifecycle partition_lifecycle a drop plan for a parent not RANGE-partitioned on one column is refused, LIST and two-column RANGE included 15;16;17;18;19 2026-10-14 partition_drop_candidates refusing only a parent with no partition key, not LIST or a multi-column RANGE;partition_drop_candidates without its RANGE-on-one-column check partition_lifecycle partition_lifecycle a drop plan for a parent not partitioned on a timestamp is refused 15;16;17;18;19 2026-10-14 a parent partitioned on a non-timestamp key planned instead of refused;partition_drop_candidates without its timestamp-key check partition_lifecycle partition_lifecycle a drop plan for a parent with a DEFAULT partition is refused 15;16;17;18;19 2026-10-13 a parent with a DEFAULT partition planned instead of refused +partition_lifecycle partition_lifecycle a non-superuser owner creates partitions ahead on its own table 15;16;17;18;19 never - partition_lifecycle partition_lifecycle a parent partitioned on a key that is not a timestamp is refused before any DDL 15;16;17;18;19 2026-10-13 partition_create_ahead without its timestamp key check partition_lifecycle partition_lifecycle a parent that is not RANGE-partitioned on one column is refused before any DDL 15;16;17;18;19 2026-10-13 partition_create_ahead without its RANGE-on-one-column check partition_lifecycle partition_lifecycle a parent with a 63-byte name gets partitions whose names keep their dates 15;16;17;18;19 2026-10-14 partition_create_ahead creating its last partition without attaching it;partition_create_ahead stopping one width short of now plus the lead;partition_create_ahead without shortening a long parent name;partition_create_ahead's name bound one byte too long @@ -2410,6 +2411,7 @@ partition_lifecycle partition_lifecycle a partition named with the drop block's partition_lifecycle partition_lifecycle a partition that is itself partitioned is planned whole 15;16;17;18;19 2026-10-13 a partition that is itself partitioned left out of the plan;a partition whose upper bound equals the cutoff left out;a plain DROP, not refusing while the partition is still attached;the cutoff filter removed: every partition planned;the lower bound read where the upper is meant;the plan detaches but drops nothing;the plan detaches without CONCURRENTLY partition_lifecycle partition_lifecycle a pending detach is finalised, not started again 15;16;17;18;19 2026-10-13 a partition whose upper bound equals the cutoff left out;a pending detach planned as a new concurrent detach, not FINALIZE;a plain DROP, not refusing while the partition is still attached;the cutoff filter removed: every partition planned;the lower bound read where the upper is meant;the plan detaches but drops nothing partition_lifecycle partition_lifecycle a plan for names that need quoting runs as returned 15;16;17;18;19 2026-10-13 a partition whose upper bound equals the cutoff left out;names in the plan with their quotes removed;the cutoff filter removed: every partition planned;the lower bound read where the upper is meant;the plan detaches but drops nothing +partition_lifecycle partition_lifecycle a role that does not own the parent is refused by its own privileges (42501), and creates nothing 15;16;17;18;19 2026-10-14 partition_create_ahead made SECURITY DEFINER partition_lifecycle partition_lifecycle a second call creates nothing 15;16;17;18;19 2026-10-14 partition_create_ahead creating its last partition without attaching it;partition_create_ahead starting from now, not the highest upper bound partition_lifecycle partition_lifecycle a second plan for the same cutoff is empty, and the partitions it kept are still attached 15;16;17;18;19 2026-10-13 the cutoff filter removed: every partition planned;the lower bound read where the upper is meant partition_lifecycle partition_lifecycle a tiered partition is planned with DROP FOREIGN TABLE 15;16;17;18;19 2026-10-13 a partition whose upper bound equals the cutoff left out;a plain DROP, not refusing while the partition is still attached;a tiered partition planned with DROP TABLE;the cutoff filter removed: every partition planned;the lower bound read where the upper is meant;the plan detaches but drops nothing;the plan detaches without CONCURRENTLY @@ -2441,6 +2443,8 @@ partition_lifecycle partition_lifecycle premise: a partition is named with the d partition_lifecycle partition_lifecycle premise: a timestamp parent continued from a hand-made partition, in a session west of UTC 15;16;17;18;19 never - partition_lifecycle partition_lifecycle premise: a timestamp parent was given partitions from a session east of UTC 15;16;17;18;19 never - partition_lifecycle partition_lifecycle premise: a timestamptz parent continued from a hand-made partition, in a session west of UTC 15;16;17;18;19 never - +partition_lifecycle partition_lifecycle premise: pl_own owns the parent, and neither role is a superuser 15;16;17;18;19 never - +partition_lifecycle partition_lifecycle premise: pl_peer can open a session 15;16;17;18;19 never - partition_lifecycle partition_lifecycle premise: the January partition was tiered to Parquet 15;16;17;18;19 never - partition_lifecycle partition_lifecycle premise: the call inside the transaction created partitions 15;16;17;18;19 never - partition_lifecycle partition_lifecycle premise: the drop fixture has several partitions, each holding rows 15;16;17;18;19 never - @@ -2884,6 +2888,7 @@ tier_partition tier_partition a failure after drop leaves the rows exactly once, tier_partition tier_partition a failure after export leaves the rows exactly once, still local 15;16;17;18;19 never - tier_partition tier_partition a fault step that is not one of tier_partition's is refused 15;16;17;18;19 2026-10-13 the tier_fault_after check hook accepting any value tier_partition tier_partition a grant to a role whose name needs quoting is carried to the foreign partition 15;16;17;18;19 2026-10-13 tier_partition naming the foreign table with a suffix;tier_partition not carrying the grants;tier_partition returning 0 rows moved +tier_partition tier_partition a non-superuser owner tiers its own partition 15;16;17;18;19 never - tier_partition tier_partition a parent held by a long reader times the move out, as lock_wait says 15;16;17;18;19 2026-10-13 tier_partition without the lock_wait bound tier_partition tier_partition a parent with a UNIQUE index is refused before anything moves 15;16;17;18;19 2026-10-13 tier_partition without the UNIQUE-parent refusal tier_partition tier_partition a partition a view depends on is refused before anything moves 15;16;17;18;19 2026-10-13 tier_partition without the dependents check @@ -2896,6 +2901,7 @@ tier_partition tier_partition a partition with a trigger of its own is refused b tier_partition tier_partition a query on the other range scans only the local partition, beside a foreign one 15;16;17;18;19 never - tier_partition tier_partition a query on the tiered range scans only the foreign partition 15;16;17;18;19 never - tier_partition tier_partition a retry over the failed attempt's file moves the partition, exactly once 15;16;17;18;19 2026-10-13 tier_partition naming the foreign table with a suffix;tier_partition refusing every existing path (the first version's rule);tier_partition returning 0 rows moved +tier_partition tier_partition a role that does not own the table is refused by its own privileges (42501) 15;16;17;18;19 2026-10-14 tier_partition made SECURITY DEFINER tier_partition tier_partition a rollback after the whole call leaves the rows local, exactly once 15;16;17;18;19 never - tier_partition tier_partition a server of another wrapper is refused 15;16;17;18;19 2026-10-14 tier_partition without its wrong-wrapper branch, so the iceberg wrapper refuses only after the export;tier_partition without the wrapper check tier_partition tier_partition a server that does not exist is refused before anything moves 15;16;17;18;19 2026-10-13 tier_partition without its missing-server branch, so core refuses only after the export @@ -2913,6 +2919,8 @@ tier_partition tier_partition and no foreign table behind it after detach 15;16; tier_partition tier_partition and no foreign table behind it after drop 15;16;17;18;19 never - tier_partition tier_partition and no foreign table behind it after export 15;16;17;18;19 never - tier_partition tier_partition and the refusals moved nothing 15;16;17;18;19 2026-10-13 tier_partition without the path refusal +tier_partition tier_partition and the refused call leaves the partition local and attached, every row once 15;16;17;18;19 2026-10-14 tier_partition made SECURITY DEFINER +tier_partition tier_partition and the refused call wrote no export file 15;16;17;18;19 2026-10-14 tier_partition made SECURITY DEFINER tier_partition tier_partition and the timed-out move leaves the rows local, with nothing attached 15;16;17;18;19 2026-10-13 tier_partition without the lock_wait bound tier_partition tier_partition at read committed a row committed between the caller's snapshot and the lock survives 15;16;17;18;19 never - tier_partition tier_partition at read committed the move goes ahead and exports that row 15;16;17;18;19 2026-10-13 tier_partition naming the foreign table with a suffix;tier_partition returning 0 rows moved @@ -2937,6 +2945,8 @@ tier_partition tier_partition premise: the first attempt failed after export and tier_partition tier_partition premise: the move started while the insert was in flight, and waited for it 15;16;17;18;19 never - tier_partition tier_partition premise: the parent holds every row 15;16;17;18;19 never - tier_partition tier_partition premise: the storage id was read before the move 15;16;17;18;19 never - +tier_partition tier_partition premise: tier_own owns the parent and its partitions, and neither role is a superuser 15;16;17;18;19 never - +tier_partition tier_partition premise: tier_peer can open a session and read the partition 15;16;17;18;19 never - tier_partition tier_partition the answers over the parent are the same after the move 15;16;17;18;19 2026-10-13 tier_partition naming the foreign table with a suffix tier_partition tier_partition the call holds ACCESS EXCLUSIVE on the parent until commit, as the docs say 15;16;17;18;19 never - tier_partition tier_partition the cold partition is now a foreign partition with its own name and bound 15;16;17;18;19 2026-10-13 tier_partition attaching with the upper bound shifted a minute;tier_partition naming the foreign table with a suffix @@ -2947,6 +2957,7 @@ tier_partition tier_partition the local partition leaves no pgcolumnar storage r tier_partition tier_partition the move ran inside the caller's transaction 15;16;17;18;19 2026-10-13 tier_partition returning 0 rows moved tier_partition tier_partition the other partition is still local after the refusals 15;16;17;18;19 2026-10-13 tier_partition without the path refusal tier_partition tier_partition tier_partition moves the cold partition and returns the rows it moved 15;16;17;18;19 2026-10-13 tier_partition returning 0 rows moved +tier_partition tier_partition with no foreign table behind it 15;16;17;18;19 2026-10-14 tier_partition made SECURITY DEFINER tier_partition tier_partition with the dependents gone, the same partition moves 15;16;17;18;19 2026-10-13 tier_partition returning 0 rows moved;tier_partition without the dependents check timeseries_recipes timeseries_recipes premise: the readings table is columnar 15;16;17;18;19 never - timeseries_recipes timeseries_recipes premise: the reference has 24 hours, 6 of them empty and one all-NULL 15;16;17;18;19 never - diff --git a/test/check_ledger_budget.txt b/test/check_ledger_budget.txt index e8678ba9d..7f5b72915 100644 --- a/test/check_ledger_budget.txt +++ b/test/check_ledger_budget.txt @@ -2545,4 +2545,18 @@ suites_not_covered 230 # never 2345 + 1 = 2346 # dated 770 + 2 = 772 # 2346 + 772 == 3118 -checks_never_observed_red 2346 +# +# #1341, tier_partition and partition_create_ahead on the caller's own privileges +# (OffgridwithJD): 2346 -> 2352. Eleven new checks, all in already-covered suites. +# Five are dated, by each function made SECURITY DEFINER: tier_partition's refusal, +# the partition left local, no foreign table and no export file (four), and +# partition_create_ahead's refusal (one). The two owner controls run on tables of +# their own, so neither mutation reddens them; they and four premises arrive never. +# Five-major runs: tier_partition 79/79 (80 on 18-19, which run the virtual column) +# and partition_lifecycle 64/64 on 15-19. +# +# ledger rows 3118 + 11 = 3129 +# never 2346 + 6 = 2352 +# dated 772 + 5 = 777 +# 2352 + 777 == 3129 +checks_never_observed_red 2352