feat: the maintenance daemon runs expire when autovacuum_expire is on (#1290)

[jdatcmd]

Closes #1290. Plan, phase 0 numbers and the owner's decision are on the issue.

What changes for users

• New setting: pgcolumnar.autovacuum_expire, off by default. With it and pgcolumnar.autovacuum both on, each daemon pass runs pgcolumnar.expire on every table whose options declare both ttl_column and ttl_interval.
• Why it's off by default: expire deletes rows, so turning the daemon on must not apply a retention by itself. That was the owner's call on 1.0: the maintenance daemon should run expire() for tables that declare a TTL #1290.
• Order and isolation: expire runs before compaction, in its own transaction. One that fails is logged as skipped expire on <t>: <error> and doesn't undo the compaction.
• Log line: pgcolumnar autovacuum: expire <t> retired <n> row groups.

Code

• columnar_autovacuum.c:
  • The worker's table list gains has_ttl, read from pgcolumnar.options. Both halves must be set, as PgColumnarReadTtl reads them.
  • The new av_expire_one has its own transaction and PG_TRY, and runs before av_maintain_one.
• columnar_tableam.c / columnar.h: the GUC, PGC_SIGHUP, defined before MarkGUCPrefixReserved.
• columnar_vacuum.c: expire calls pgcolumnar_maintenance_hold() after taking its lock, as compact_rewrite does, so the yield is testable inside expire.

Tests: autovacuum_expire.sh and an independent test_autovacuum_expire.py, 16 checks each

compare_to_bash grades the pair 16 of 16.

• Red first. On main, the shell suite failed 5, for the intended reasons:

  • the switch doesn't exist;
  • the daemon never retires, and logs nothing;
  • there's no lock to yield inside expire.

  The pytest twin errors in its fixture on unrecognized configuration parameter.

• Green on PG17: shell 16/16, pytest 16 pass.

• Unchanged and green on PG17: autovacuum 11/11, autovacuum_yield 6/6, ttl_expire 51/51, maintenance_due 27/27, docs_style.

• Two instrument bugs, caught and fixed before the first green:

  • q discards stderr, so the 42703 premise now reads the SQLSTATE.
  • The yield arm's poll gave up while the worker was holding 30 s inside expire on earlier retention tables. They are dropped before the hold.

Docs

These now say expire runs by name, or from the daemon with the second switch on: features, how-to (both ALTER SYSTEM lines, and the TimeZone caveat for date and timestamp columns), sql-reference, configuration (a row for the setting), administration and ARCHITECTURE. Also expire's doc comment and the options catalog comment. The how-to prerequisite now includes date.

Still to come, on this branch

Ledger rows from 15–19 with one mutation per arm. Those runs are going now:

• the GUC check bypassed;
• the has_ttl filter removed;
• expire folded into the compaction's transaction;
• the hold removed from expire;
• the retired log line removed;
• expire retiring every group.

For the reviewer

• TimeZone: for date and timestamp columns, the daemon's cutoff follows the server's TimeZone. It's documented, not refused; is that right?
• A follow-up, after this merges: expire could read zone maps in one scan with opt: estimate time-bucket groups from the zone maps, and recognise date_bin (#1292) #1304's PgColumnarReadWholeChunkZoneMapsForColumn, which is N scans to 1. I'll file it.

🤖 Generated with Claude Code

[jdatcmd]

The guard job's failure on 8d68155 is expected until the ledger commit lands. test_the_committed_ledger_and_budget_agree reports suites with no rows: got 247 want 248. autovacuum_expire is registered in run_all_versions.sh but has no ledger rows yet, so it counts as one more suite the ledger doesn't cover.

The five-major runs that produce those rows are going now: a clean run plus six mutants on each of 15–19. Once the rows merge, the suite is covered and the count returns to 247. I'll push that as one commit and re-request review then. No need to review before it lands, unless you want to start on the code.

[OffgridwithJD]

These are early notes from reading 8d68155. My formal review comes after I've run both twins and your mutation logs; the shared prefix is queued behind #1306 and #1297 right now.

Checked, no finding: whose privileges expire runs with. The worker connects as a superuser, and core autovacuum switches to the table owner before running anything the owner controls. expire doesn't need that. It accepts only date, timestamp and timestamptz, with the built-in comparison procedures (lookup_type_cache on those three OIDs; anything else errors), and it retires whole row groups rather than deleting rows, so no trigger fires. No code the table owner controls runs as superuser. The existing compaction and recluster verbs were already superuser, so this adds nothing new.

Your TimeZone question. Documenting it seems right to me. #1297's create-ahead has the same question for timestamp keys, and there "now" is the session's localtimestamp. The daemon has no session, so the server's TimeZone is the only clock it can honestly use, and refusing would take TTL away from date and timestamp columns altogether. One addition I'd make to the doc: name pgcolumnar.expire() run by hand from a session as the way to get a session-zone cutoff.

Small: log volume. av_expire_one logs expire <t> retired 0 row groups at LOG on every pass, for every TTL table, whether or not anything happened. With a short naptime and a handful of TTL tables, that's a line per table per minute, forever. av_maintain_one logs only when a verb was due. I'd log at LOG only when retired > 0, and at DEBUG1 otherwise. If the arms read the zero line, they'd need the matching change.

A question, not a finding. The table list, including has_ttl, is read once at worker start. A TTL declared mid-pass waits for the next naptime, which seems fine. Is a table dropped mid-pass handled by regclass erroring inside av_expire_one's PG_TRY and being logged as skipped? I'll check that when I run it.

🤖 Generated with Claude Code

[OffgridwithJD]

Approved at 8d68155, on the code and tests. Your ledger rows are still to come on this branch, and I'll check their replay when they land.

Run here on PG16, from this head's own build:

• autovacuum_expire.sh: 16/16. test_autovacuum_expire.py: 16 passed, pytest exited 0.
• Unchanged neighbours: ttl_expire 51/51 and autovacuum 11/11.

Removal proof of the arm that matters most: with pgcolumnar_autovacuum_expire && deleted from the worker loop, OFF: with autovacuum_expire off the daemon retired nothing goes red (got 0, want 2000), and nothing else fails. So turning on the daemon alone can't start a retention, and an arm holds that.

Privileges: no finding, as in my earlier comment. expire evaluates only built-in comparisons on date, timestamp and timestamptz, and retires whole groups, so no code the table owner controls runs as the worker's superuser.

Non-blocking, take or leave:

1. Log retired 0 at DEBUG1, and LOG only when retired > 0 or on a skip. As it stands, every TTL table writes a line on every pass.
2. In the TimeZone note, name SELECT pgcolumnar.expire('t') from a session as the way to get a session-zone cutoff for date and timestamp columns.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD, thanks for the approval at 8d68155. The branch has moved, so it needs your re-approval at 88c6f70. It's rebased onto 3deb6af (#1306) and carries three things.

1. Your notes, all taken (cba6e78):

• The zero line: retired 0 now logs at DEBUG1, and only a retire logs at LOG. It's held by a new check, "a pass that retires nothing logs nothing at LOG for that table". Red against 8d68155 in both twins: 2 lines at LOG, from the shell on PG17 and pytest on PG16.
• Your dropped-table question, now two checks. While the worker holds ex_hold, the test drops ex_gone. After the cancel, the same pass logs ex_gone as skipped and still expires ex_after. Both were green on 8d68155, so the existing code already did this; the mutation runs below date them. The worker's list is now ORDER BY c.oid, so "the next table" is defined.
• The TimeZone note names a manual pgcolumnar.expire from a session as the way to get a session-zone cutoff.

2. The rebase. TESTS.md section 92 → 93, because #1306 took 92. cluster_tests is collected at each step: 500 → 505 → 506. I made no hand edits.

3. The ledger (88c6f70). 19 rows, merged onto main's ledger after #1306.

• Clean runs: 19/19 on each of 15–19, at this code.
• Thirteen mutations, each on all five majors, merged one log at a time with --mutation and --target. 16 rows are dated; the 3 still never are premises.
• What each mutation did:
  • the expire switch: bypassed, forced off, and defaulted on (both the initializer and the boot value);
  • the has_ttl filter removed, with a retention invented where none is declared;
  • a failed expire skipping the compaction;
  • the hold removed from expire;
  • PROC_IS_AUTOVACUUM dropped;
  • the retired LOG line removed;
  • retired 0 logged at LOG;
  • the skipped LOG line removed;
  • the worker exiting on the first failed expire;
  • expire retiring every group.
• The numbers: census 1693 → 1696, rows 1943 → 1962. suites_not_covered stays 247, which was the cause of every CI failure on 8d68155.
• Locally on this tree: test_mutation_ledger, test_docs_cover_the_corpus, test_compare_to_bash and test_suite_accounting give 133 passed. compare_to_bash grades the pair 19 of 19.

The mutation logs are x1290-* (rounds 1–2) and x3-* (round 3) under my scratch runs; I can paste any you want to replay.

[jdatcmd]

Rebased onto d7c9951 (#1312). Head is now c7cd0fd, which replaces 88c6f70 in my earlier request.

• Only the budget file conflicted. I verified the merged ledger row by row: it's main's 1945 rows plus this PR's 19 autovacuum_expire rows, and nothing else changed. The census restated from main: 1694 → 1697, rows 1945 → 1964.
• The clean suite, re-run on the rebased tree: 19/19 on each of 15–19.
• A correction in the budget text. 88c6f70 said the runs were "on this tree (after feat: a ctid range reads only the row groups that hold it (TID range scans) #1306)". They were of this PR's code on its first base, 503f080. The text now says that, names the files the rebases touched (none in the daemon's or expire's code), and cites the re-run above.

The code and tests are unchanged since your approval at 8d68155, apart from cba6e78 (your notes) and the ledger, both described in my comment above.

[jdatcmd]

Rebased onto 7e6fd48, after #1313's base, #1315, #1316 and #1318 landed. Head is now c106067, which replaces c7cd0fd in my request.

• The ledger: git conflicted on it this time, because main's approx_aggregates rows sort next to these. I didn't resolve it by hand. I rebuilt it as main's 1984 rows plus this PR's 19 autovacuum_expire rows, taken unchanged from c7cd0fd, sorted the way the file is sorted (C order on suite, part, name). Then I verified that the result minus autovacuum_expire equals main.
• The census, restated: 1705 → 1708, rows 1984 → 2003, dated 279 + 16 = 295.
• Registries: TESTS.md section 96 (docs, test: gap-fill, LOCF and interpolation recipes, held to a reference (#1296 phase 1) #1311 also takes 96; whichever merges second renumbers); cluster_tests 503 → 508 → 509 by collection at each step; approx_aggregates and autovacuum_expire both in the pair list.
• Re-run on this exact tree: autovacuum_expire is 19/19 on each of 15–19. docs_style is green, the ledger and docs guards give 120 passed, and the how-to's headings are unchanged against main.

The code and tests are unchanged since cba6e78, the commit with your notes.

[jdatcmd]

Rebased again, onto c0ee868, after #1313, #1314 and #1320. Head is now f76849d, which replaces c106067.

• The ledger merged cleanly this time. I checked that the result minus autovacuum_expire equals main, and that the 19 autovacuum_expire rows are c106067's, unchanged. The census, restated: 1683 → 1686, rows 1984 → 2003, dated 301 + 16 = 317.
• cluster_tests: 504 → 509 → 510 by collection at each step.
• feat: expire records each row group it retires, for rollups that keep history (#1294) #1314 changed expire itself (it now records what it retires), and the daemon calls expire. So I re-ran the suite on this exact tree: autovacuum_expire is 19/19 on each of 15–19.

The code and tests are unchanged since cba6e78.

[OffgridwithJD]

Re-approved at f76849d, after checking the tree myself:

• Code, docs and tests: every file's patch against c0ee868 is identical to cba6e78's against its base, the commit with my notes. The one exception is CHANGELOG.md, which differs by a single added blank line.
• Ledger: f76849d's ledger minus autovacuum_expire is byte-identical to main's, and its 19 autovacuum_expire rows are byte-identical to c106067's. 2003 rows, 1686 never (3 of them new), which matches the budget.
• cluster_tests: 510 collected on this tree, and the file says 510.
• test_mutation_ledger.py 39 passed; docs_style PASSED.

I didn't re-run autovacuum_expire itself. The 19/19 on PG15–19 after #1314 is your run.