Skip to content

test: ttl_expire, the #1339 findings in both twins: a timestamp column, a zone that always differs, accurate premises #1353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdatcmd merged 3 commits into from
Oct 14, 2026
Merged

test: ttl_expire, the #1339 findings in both twins: a timestamp column, a zone that always differs, accurate premises #1353

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
64fa006
test: ttl_expire, the #1339 findings in both twins
jdatcmd Oct 13, 2026
6204ebb
test: ledger for the ttl_expire fixes (#1339)
jdatcmd Oct 13, 2026
6786f2c
test: TESTS.md row and cluster_tests for the timestamp retention test…
jdatcmd Oct 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 18 additions & 15 deletions test/check_ledger.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2250,23 +2250,25 @@ ttl_expire ttl_expire NO row still inside the retention was dropped (#403 item 5
ttl_expire ttl_expire NO row still inside the retention was dropped, on a date column 15;16;17;18;19 never -
ttl_expire ttl_expire a compaction's retired groups are not recorded as retention 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire a date retention column is accepted, not refused (#1135) 15;16;17;18;19 never -
ttl_expire ttl_expire a group whose only NULLs have been deleted is retired 15;16;17;18;19 never -
ttl_expire ttl_expire a group whose only NULLs have been deleted is retired 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired
ttl_expire ttl_expire a later expire on a column of only NULLs reads no zone map 15;16;17;18;19 2026-10-13 expire's early exit removed (the tree before #1310);the early exit not taken for a column with no value
ttl_expire ttl_expire a negative ttl_interval is refused with 22023 15;16;17;18;19 never -
ttl_expire ttl_expire a no-op expire reads no zone map when the extent's minimum is past the cutoff 15;16;17;18;19 2026-10-13 expire's early exit removed (the tree before #1310)
ttl_expire ttl_expire a rolled-back expire records nothing 15;16;17;18;19 never -
ttl_expire ttl_expire a row dated exactly on the cutoff is kept, so truncation errs toward keeping 15;16;17;18;19 never -
ttl_expire ttl_expire a table with no declared retention is an error, not a silent success 15;16;17;18;19 never -
ttl_expire ttl_expire and VACUUM really did set visibility-map bits to clear 15;16;17;18;19 never -
ttl_expire ttl_expire and a zero ttl_interval is refused too 15;16;17;18;19 never -
ttl_expire ttl_expire and its rows are gone 15;16;17;18;19 never -
ttl_expire ttl_expire and no row older than Pacific/Midway's own cutoff survives 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire and no row older than UTC's own cutoff survives 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire and no row older than a zone on another date's own cutoff survives 15;16;17;18;19 2026-10-14 expire's date cutoff taken from the UTC wall clock, not the session's date;the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire and the date table really is smaller than it was 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire and the expired timestamps sharing the group were kept with them 15;16;17;18;19 never -
ttl_expire ttl_expire and the table lost exactly the groups it retired 15;16;17;18;19 never -
ttl_expire ttl_expire and the table lost exactly the groups it retired 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired
ttl_expire ttl_expire and the table really is smaller than it was 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire and they are the groups gone from the row-group catalog 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire and those NULL rows are still there 15;16;17;18;19 never -
ttl_expire ttl_expire but the zone map still records the deleted NULLs 15;16;17;18;19 never -
ttl_expire ttl_expire control: a group whose NULLs are still live is NOT retired 15;16;17;18;19 never -
ttl_expire ttl_expire control: and a positive one is still accepted 15;16;17;18;19 never -
ttl_expire ttl_expire control: and the rows are all still there 15;16;17;18;19 never -
Expand All @@ -2275,28 +2277,27 @@ ttl_expire ttl_expire dropping the table drops its retention record 15;16;17;18;
ttl_expire ttl_expire every remaining date row reads back its own value 15;16;17;18;19 never -
ttl_expire ttl_expire every remaining row reads back its own value 15;16;17;18;19 never -
ttl_expire ttl_expire expire does not retire a group that still holds NULL retention rows 15;16;17;18;19 never -
ttl_expire ttl_expire expire in Pacific/Midway keeps exactly the rows at or after that session's cutoff 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire in UTC keeps exactly the rows at or after that session's cutoff 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire in the transaction that wrote aged rows retires their groups 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum;the extent bound leaves out the transaction's own pending fold (#1304 option 5's fix)
ttl_expire ttl_expire expire on a date column retires at least one group 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire in a zone on another date keeps exactly the rows at or after that session's cutoff 15;16;17;18;19 2026-10-14 expire's date cutoff taken from the UTC wall clock, not the session's date;the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire in the transaction that wrote aged rows retires their groups 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired;the early exit compares the extent's maximum to the cutoff instead of its minimum;the extent bound leaves out the transaction's own pending fold (#1304 option 5's fix)
ttl_expire ttl_expire expire on a date column retires at least one group 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired;the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire on a timestamp column retires the groups wholly before the session's local cutoff 15;16;17;18;19 2026-10-14 expire's timestamp cutoff taken from the UTC wall clock, not the session's local time
ttl_expire ttl_expire expire records exactly the groups it retired 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire retired the three aged groups 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire retires at least one group (#403 item 5a) 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire retires the all-visible expired group 15;16;17;18;19 never -
ttl_expire ttl_expire expire retired the three aged groups 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired;the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire retires at least one group (#403 item 5a) 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired;the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire expire retires the aged groups and keeps those of NULLs 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired
ttl_expire ttl_expire expire retires the all-visible expired group 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired
ttl_expire ttl_expire index-only scan does not return rows expire already retired 15;16;17;18;19 never -
ttl_expire ttl_expire premise: an index-only scan is chosen on the all-visible table 15;16;17;18;19 never -
ttl_expire ttl_expire premise: and VACUUM really did set visibility-map bits to clear 15;16;17;18;19 never -
ttl_expire ttl_expire premise: and every live row is past the retention 15;16;17;18;19 never -
ttl_expire ttl_expire premise: and rows dated exactly ON the cutoff, which decide the rounding 15;16;17;18;19 never -
ttl_expire ttl_expire premise: and the SAME row group holds expired rows and NULL ones 15;16;17;18;19 never -
ttl_expire ttl_expire premise: but the zone map still records the deleted NULLs 15;16;17;18;19 never -
ttl_expire ttl_expire premise: compact_rewrite retired a group 15;16;17;18;19 never -
ttl_expire ttl_expire premise: expire retires the aged groups and keeps those of NULLs 15;16;17;18;19 never -
ttl_expire ttl_expire premise: in Pacific/Midway the fixture straddles that session's cutoff 15;16;17;18;19 never -
ttl_expire ttl_expire premise: in UTC the fixture straddles that session's cutoff 15;16;17;18;19 never -
ttl_expire ttl_expire premise: in a zone on another date the fixture straddles that session's cutoff 15;16;17;18;19 never -
ttl_expire ttl_expire premise: no LIVE row holds a NULL retention value any more 15;16;17;18;19 never -
ttl_expire ttl_expire premise: one date per row group in Pacific/Midway, so no group straddles the cutoff 15;16;17;18;19 never -
ttl_expire ttl_expire premise: one date per row group in UTC, so no group straddles the cutoff 15;16;17;18;19 never -
ttl_expire ttl_expire premise: one date per row group in a zone on another date, so no group straddles the cutoff 15;16;17;18;19 never -
ttl_expire ttl_expire premise: rows exist on both sides of the cutoff within one day of it 15;16;17;18;19 never -
ttl_expire ttl_expire premise: some rows are past the retention and some are not 15;16;17;18;19 never -
ttl_expire ttl_expire premise: the date fixture has rows on both sides of the cutoff 15;16;17;18;19 never -
Expand All @@ -2305,14 +2306,16 @@ ttl_expire ttl_expire premise: the fixture is ONE row group, so the NULLs share
ttl_expire ttl_expire premise: the fixture is laid out in several row groups 15;16;17;18;19 never -
ttl_expire ttl_expire premise: the no-op fixture has many row groups 15;16;17;18;19 never -
ttl_expire ttl_expire premise: the record fixture has three aged-out row groups ahead of the current ones 15;16;17;18;19 never -
ttl_expire ttl_expire premise: the second zone's date is not UTC's right now 15;16;17;18;19 never -
ttl_expire ttl_expire premise: the table holds NULL retention rows at all 15;16;17;18;19 never -
ttl_expire ttl_expire premise: without the extent, expire retires the aged groups 15;16;17;18;19 never -
ttl_expire ttl_expire premise: the timestamp fixture is six groups of 1,000 rows 15;16;17;18;19 never -
ttl_expire ttl_expire running it again retires nothing, because nothing new expired 15;16;17;18;19 never -
ttl_expire ttl_expire seqscan agrees the expired rows are gone 15;16;17;18;19 never -
ttl_expire ttl_expire the index-only scan is still the plan after expire 15;16;17;18;19 never -
ttl_expire ttl_expire the retention is recorded where the other options live 15;16;17;18;19 never -
ttl_expire ttl_expire with the extent, expire retires the same groups 15;16;17;18;19 2026-10-13 the early exit compares the extent's maximum to the cutoff instead of its minimum
ttl_expire ttl_expire without the extent the same expire probes every group's zone map 15;16;17;18;19 never -
ttl_expire ttl_expire without the extent, expire retires the aged groups 15;16;17;18;19 2026-10-14 expire returning 0 whatever it retired
validity_elision validity_elision a chunk claiming no bitmap while it holds fewer values than rows is refused 15;16;17;18;19 never -
validity_elision validity_elision a column with no nulls stores no validity bitmap 15;16;17;18;19 2026-09-18 writer: flush_one_column's presentCount == rowCount forced false, so the bitmap is written even when the chunk holds no null
validity_elision validity_elision a null-free column elides its bitmap beside a null-bearing one in the same row group 15;16;17;18;19 never -
Expand Down
17 changes: 16 additions & 1 deletion test/check_ledger_budget.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1803,4 +1803,19 @@ suites_not_covered 246
# never 1803 - 2 = 1801
# dated 533 + 4 + 2 = 539
# 1801 + 539 == 2340
checks_never_observed_red 1801
#
# #1339 audit, ttl_expire (OffgridwithJD): 1801 -> 1798. Three new rows: the
# timestamp retention arm, dated by expire's timestamp cutoff taken from the UTC
# wall clock; two premises (the second zone's date is not UTC's, the timestamp
# fixture's six groups), never. Eight rows renamed with their history: four checks
# named premise: that test expire, VACUUM or the zone map lose the prefix (#1236),
# and the four Pacific/Midway checks are named "a zone on another date", because the
# zone is now chosen at run time. Dated here: the two renamed expire arms and three
# existing arms first seen red (expire returning 0); the zone arms again by expire's
# date cutoff taken from the UTC wall clock. Five-major runs: 70/70 on each of 15-19.
#
# ledger rows 2340 + 3 = 2343
# never 1801 + 2 - 2 - 3 = 1798
# dated 539 + 1 + 2 + 3 = 545
# 1798 + 545 == 2343
checks_never_observed_red 1798
3 changes: 2 additions & 1 deletion test/pytest/TESTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6233,7 +6233,8 @@ restored byte-identical.
| `test_a_non_positive_retention_is_refused` | a cutoff in the future would retire live groups: `22023`, with a positive control |
| `test_a_deleted_null_stops_pinning_its_group` | `null_count` is a write-time record, not a live count |
| `test_a_date_retention_column_truncates_toward_keeping` | the rounding, which only this arm notices |
| `test_the_cutoff_is_the_sessions_own_date` | the cutoff is the session's, asserted as an identity rather than as a difference |
| `test_the_cutoff_is_the_sessions_own_date` | the cutoff is the session's, asserted as an identity rather than as a difference, in UTC and in a zone chosen at run time to be on another date (#1339) |
| `test_a_timestamp_retention_column_uses_the_sessions_local_time` | a `timestamp` column's cutoff is the session's local now: in UTC+14, exactly the groups wholly before it are retired (#1339) |
| `test_expire_records_what_it_retires_and_nothing_else` | `expire` writes exactly the groups it retires to `pgcolumnar.retired_group` (#1294); a rolled-back `expire` and a `compact_rewrite` record nothing; dropping the table drops its rows |
| `test_a_no_op_expire_reads_the_extent_not_every_zone_map` | #1310: with the retention column's extent (#1304 option 5) past the cutoff, a no-op `expire` makes 0 zone-map scans where it made one per group; the same groups retire with or without the extent; aged rows written in `expire`'s own transaction retire; a column left with only NULLs needs no reads. Counted in a fresh connection, with the test schema's `search_path` carried over. Seen red first: 40 scans |

Expand Down
4 changes: 3 additions & 1 deletion test/pytest/expected_tests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -627,4 +627,6 @@ guard_tests 429
#
# cluster_tests 530 -> 532 for #1339's two test_harness_deps.py arms (the needs-the-driver
# kind and the --partition mode), by collection over the cluster files on this branch.
cluster_tests 532
# cluster_tests 532 -> 533 for #1339's test_a_timestamp_retention_column_uses_the_sessions_local_time
# in test_ttl_expire.py, by collection over the cluster files on this branch.
cluster_tests 533
Loading
Loading