Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions test/check_ledger.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -2453,12 +2453,15 @@ ttl_expire ttl_expire a group whose only NULLs have been deleted is retired 15;1
ttl_expire ttl_expire a later expire on a column of only NULLs reads no zone map 15;16;17;18;19 2026-10-13 expire's early exit removed (the tree before #1310);the early exit not taken for a column with no value
ttl_expire ttl_expire a negative ttl_interval is refused with 22023 15;16;17;18;19 never -
ttl_expire ttl_expire a no-op expire reads no zone map when the extent's minimum is past the cutoff 15;16;17;18;19 2026-10-13 expire's early exit removed (the tree before #1310)
ttl_expire ttl_expire a retention column without an interval is no retention: refused (55000), every row kept 15;16;17;18;19 2026-10-14 a retention column with no interval read as retention with a zero interval
ttl_expire ttl_expire a role that does not own the table cannot expire it, even holding DELETE (42501) 15;16;17;18;19 2026-10-14 expire without its owner check
ttl_expire ttl_expire a rolled-back expire records nothing 15;16;17;18;19 never -
ttl_expire ttl_expire a row dated exactly on the cutoff is kept, so truncation errs toward keeping 15;16;17;18;19 never -
ttl_expire ttl_expire a table with no declared retention is an error, not a silent success 15;16;17;18;19 2026-10-14 the no-retention refusal raised as XX000 rather than 55000
ttl_expire ttl_expire and VACUUM really did set visibility-map bits to clear 15;16;17;18;19 never -
ttl_expire ttl_expire and a zero ttl_interval is refused too 15;16;17;18;19 never -
ttl_expire ttl_expire and an interval without a column 15;16;17;18;19 2026-10-14 set_options refuses a half retention declaration
ttl_expire ttl_expire and an interval without a column is no retention either 15;16;17;18;19 2026-10-14 the retention reader drops its column test, so an interval alone is read (the backend then crashed)
ttl_expire ttl_expire and its rows are gone 15;16;17;18;19 never -
ttl_expire ttl_expire and no row older than UTC's own cutoff survives 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire and no row older than a zone on another date's own cutoff survives 15;16;17;18;19 2026-10-14 expire's date cutoff taken from the UTC wall clock, not the session's date;the early exit compares the extent's maximum to the cutoff instead of its minimum
Expand Down Expand Up @@ -2517,6 +2520,7 @@ ttl_expire ttl_expire premise: the timestamp fixture is six groups of 1,000 rows
ttl_expire ttl_expire running it again retires nothing, because nothing new expired 15;16;17;18;19 never -
ttl_expire ttl_expire seqscan agrees the expired rows are gone 15;16;17;18;19 never -
ttl_expire ttl_expire set_options accepts an int retention column, so the refusal is expire's 15;16;17;18;19 never -
ttl_expire ttl_expire set_options takes a retention column without an interval 15;16;17;18;19 2026-10-14 set_options refuses a half retention declaration
ttl_expire ttl_expire the index-only scan is still the plan after expire 15;16;17;18;19 never -
ttl_expire ttl_expire the retention is recorded where the other options live 15;16;17;18;19 never -
ttl_expire ttl_expire with the extent, expire retires the same groups 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
Expand Down
24 changes: 24 additions & 0 deletions test/check_ledger_budget.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2037,4 +2037,28 @@ suites_not_covered 241
# never 1963 - 1 = 1962
# dated 586 + 1 = 587
# 1962 + 587 == 2549
#
# #1341, half a retention declaration is no retention (OffgridwithJD): 1962 -> 1965.
# Four new ttl_expire checks. The column-only arm is dated by a reader that fills a
# missing interval with zero: expire then retired all 3000 aged rows instead of
# refusing. Three arrive never: two premises on what set_options accepts, and the
# interval-only arm, which no non-crashing mutation reaches (a column filled in
# would name a column that does not exist). Five-major runs: 74/74 on 15-19.
#
# ledger rows 2549 + 4 = 2553
# never 1962 + 3 = 1965
# dated 587 + 1 = 588
# 1965 + 588 == 2553
#
# #1370 review (OffgridwithJD): 1965 -> 1962. The two set_options checks lose the
# premise prefix and are dated, with the interval-only expire arm:
# - set_options refusing a half declaration reddens both set_options checks, while
# the expire arms stay green for an unrelated reason;
# - the retention reader without its column test crashed the backend on the
# interval-only table, so that arm now runs last in the suite.
#
# ledger rows 2553 = 2553
# never 1965 - 3 = 1962
# dated 588 + 3 = 591
# 1962 + 591 == 2553
checks_never_observed_red 1962
1 change: 1 addition & 0 deletions test/pytest/TESTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6235,6 +6235,7 @@ restored byte-identical.
| --- | --- |
| `test_expire_retires_expired_groups_and_keeps_the_straddling_one` | the feature, and the straddling group that constrains it |
| `test_expire_refuses_what_it_cannot_honour` | #1341: a non-owner holding DELETE (`42501`), a retention column that is not a time type (`42804`) and a dropped one (`42703`) are refused with every row still there, on tables whose rows are all past retention; the owner's `expire` is the control that retires them |
| `test_a_half_declaration_is_no_retention` | #1341: a retention column without an interval, or an interval without a column, is accepted by `set_options` and refused by `expire` with `55000`, every aged row kept |
| `test_a_null_retention_value_keeps_its_group` | a live NULL the maximum cannot speak for |
| `test_an_index_only_scan_does_not_return_retired_rows` | the visibility-map bits over a group `expire` removed |
| `test_a_non_positive_retention_is_refused` | a cutoff in the future would retire live groups: `22023`, with a positive control |
Expand Down
4 changes: 3 additions & 1 deletion test/pytest/expected_tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -649,4 +649,6 @@ guard_tests 433
# test/fdw_plan_privilege.sh, by collection over the cluster files on this branch.
# cluster_tests 543 -> 549 for #1341's test_arrow_import.py, the twin of arrow_import.sh:
# six tests, by collection over the cluster files on this tree.
cluster_tests 549
# cluster_tests 549 -> 550 for #1341's test_a_half_declaration_is_no_retention in
# test_ttl_expire.py, by collection over the cluster files on this tree.
cluster_tests 550
29 changes: 29 additions & 0 deletions test/pytest/test_ttl_expire.py
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,35 @@ def groups(t):
"expire refuses a retention column that no longer exists (42703)")


def test_a_half_declaration_is_no_retention(pgc_conn, expect):
"""A column with no interval names nothing to compare against, and an interval
with no column has nothing to compare: either alone reads as "no retention", so
a half declaration cannot drop rows (#1341). Every row is ten days old, so a
reader that filled in the missing half with a default would retire them all."""
with pgc_conn.cursor() as cur:
for t in ("ttl_hcol", "ttl_hiv"):
cur.execute(f"CREATE TABLE {t} (id int, ts timestamptz) USING pgcolumnar")
cur.execute(f"SELECT pgcolumnar.set_options('{t}', stripe_row_limit => 1000)")
cur.execute(f"INSERT INTO {t} SELECT g, now() - interval '10 days' FROM generate_series(1, 3000) g")

def declared(t):
return _one(pgc_conn, "SELECT coalesce(ttl_column::text, '-') || '|' || coalesce(ttl_interval::text, '-')"
" FROM pgcolumnar.options WHERE regclass = %s::regclass", (t,))

col_state = _sqlstate(pgc_conn, "SELECT pgcolumnar.set_options('ttl_hcol', ttl_column => 'ts')")
expect.text(f"{col_state}|{declared('ttl_hcol')}", "|ts|-",
"set_options takes a retention column without an interval")
iv_state = _sqlstate(pgc_conn, "SELECT pgcolumnar.set_options('ttl_hiv', ttl_interval => '1 day')")
expect.text(f"{iv_state}|{declared('ttl_hiv')}", "|-|1 day",
"and an interval without a column")
hcol = _sqlstate(pgc_conn, "SELECT pgcolumnar.expire('ttl_hcol')")
expect.text(f"{hcol}|{_one(pgc_conn, 'SELECT count(*) FROM ttl_hcol')}", "55000|3000",
"a retention column without an interval is no retention: refused (55000), every row kept")
hiv = _sqlstate(pgc_conn, "SELECT pgcolumnar.expire('ttl_hiv')")
expect.text(f"{hiv}|{_one(pgc_conn, 'SELECT count(*) FROM ttl_hiv')}", "55000|3000",
"and an interval without a column is no retention either")


def test_a_deleted_null_stops_pinning_its_group(pgc_conn, expect):
"""`null_count` is recorded at WRITE time and never revised.

Expand Down
27 changes: 27 additions & 0 deletions test/ttl_expire.sh
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,27 @@ check "premise: the declaration still names the dropped column" \
check "expire refuses a retention column that no longer exists (42703)" \
"$(ttl_state "SELECT pgcolumnar.expire('ttl_gone');")|$(q 'SELECT count(*) FROM ttl_gone')" "42703|3000"

# ---- half a declaration is no retention (#1341) -----------------------------
#
# A column with no interval names nothing to compare against, and an interval
# with no column has nothing to compare: either alone reads as "no retention", so
# a half declaration cannot drop rows. Every row here is ten days old, so a reader
# that filled in the missing half with a default would retire them all.
for _t in ttl_hcol ttl_hiv; do
psql_run "CREATE TABLE $_t (id int, ts timestamptz) USING pgcolumnar;
SELECT pgcolumnar.set_options('$_t', stripe_row_limit => 1000);
INSERT INTO $_t SELECT g, now() - interval '10 days' FROM generate_series(1, 3000) g;"
done
# Checks, not premises (#1236; #1370 review): set_options refusing a half declaration
# is a change to this repository's SQL, and it turns both red while the expire arms
# below stay green for an unrelated reason, so they are what notices it.
check "set_options takes a retention column without an interval" \
"$(ttl_state "SELECT pgcolumnar.set_options('ttl_hcol', ttl_column => 'ts');")|$(q "SELECT coalesce(ttl_column::text, '-') || '|' || coalesce(ttl_interval::text, '-') FROM pgcolumnar.options WHERE regclass = 'ttl_hcol'::regclass;")" "|ts|-"
check "and an interval without a column" \
"$(ttl_state "SELECT pgcolumnar.set_options('ttl_hiv', ttl_interval => '1 day');")|$(q "SELECT coalesce(ttl_column::text, '-') || '|' || coalesce(ttl_interval::text, '-') FROM pgcolumnar.options WHERE regclass = 'ttl_hiv'::regclass;")" "|-|1 day"
check "a retention column without an interval is no retention: refused (55000), every row kept" \
"$(ttl_state "SELECT pgcolumnar.expire('ttl_hcol');")|$(q 'SELECT count(*) FROM ttl_hcol')" "55000|3000"

# ---- a NULL that has been DELETED must stop pinning its group ---------------
#
# z->nullCount is recorded at WRITE time and never revised, so it still counts
Expand Down Expand Up @@ -667,4 +688,10 @@ check_num "expire retires the aged groups and keeps those of NULLs" \
"$(q "SELECT pgcolumnar.expire('ttl_n');")" "2"
check "a later expire on a column of only NULLs reads no zone map" "$(_expire_scans ttl_n)" "n=0 scans=0"

# The interval-only half (#1341), LAST IN THE SUITE ON PURPOSE: with the reader's
# column test removed, expire on ttl_hiv crashed the backend (PG17), and a crash
# mid-suite takes the checks after it down with it, premises included.
check "and an interval without a column is no retention either" \
"$(ttl_state "SELECT pgcolumnar.expire('ttl_hiv');")|$(q 'SELECT count(*) FROM ttl_hiv')" "55000|3000"

pgc_summary
Loading