Skip to content

test: tables that skip WAL at wal_level=minimal survive a crash (#1341) - #1427

Merged
jdatcmd merged 9 commits into
mainfrom
test/1341-minimal-wal
Oct 15, 2026
Merged

test: tables that skip WAL at wal_level=minimal survive a crash (#1341)#1427
jdatcmd merged 9 commits into
mainfrom
test/1341-minimal-wal

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

Covers the #1341 core-am item "at wal_level=minimal, a table created or truncated in the same transaction skips WAL". Test-only: no source change.

What it adds

A new suite, test/recovery_minimal_wal.sh, and its pytest twin, test_recovery_minimal_wal.py. The cluster runs at wal_level=minimal. The shell suite uses PGC_EXTRA_CONF; the twin uses a private cluster.

PostgreSQL skips WAL for a table created or truncated in the transaction that loads it, and makes the table durable at COMMIT. Below wal_skip_threshold it writes the new pages to WAL; at or above it, it fsyncs the files. Both branches run, at thresholds 0 and 64MB:

arm after a crash and recovery
a columnar table and a heap mirror created and loaded (50000 rows) in one transaction the columnar table holds what heap holds
the same two truncated and reloaded in one transaction the same

Each arm reads the committed row count before the crash. My first draft didn't, and a mutant whose load failed passed it: two missing tables "agreed". The count closes that.

Evidence

  • Green: 9/9 on each of PG15–19, and the twin on each.
  • Red on PG17, both twins: a mutant creates the relfilenode with smgrcreate/log_smgrcreate instead of RelationCreateStorage, so no commit-time sync is registered. On this assert-enabled build, PostgreSQL then fails Assert("relcache_verdict == RelFileLocatorSkippingWAL(relation->rd_locator)") during the load, and eight checks go red. I did not measure what a build without assertions would lose.
  • Ledger: 9 rows, 8 dated. The census is 2774 + 9 = 2783 rows, 2095 + 1 = 2096 never, and 687 dated. The gate passes against origin/main. The harness selftest passes 1245/1245 and the guard set 433/433. cluster_tests goes 565 → 566.

🤖 Generated with Claude Code

https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N

@jdatcmd jdatcmd mentioned this pull request Oct 15, 2026
Closed
87 tasks done
OffgridwithJD
OffgridwithJD approved these changes Oct 15, 2026
View reviewed changes

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at e7acae3.

  • What it holds: at wal_level=minimal, a columnar table created and loaded in one transaction, and one truncated and reloaded in one transaction, keep heap's content across a SIGKILL crash and recovery. Both run at wal_skip_threshold 0 (fsync) and 64MB (pages to WAL), with the committed count read before the crash.
  • Reproduced red first, on PG17 built WITHOUT assertions, which answers the open question in the description: with smgrcreate/log_smgrcreate in place of RelationCreateStorage (.so md5 766fad3c → a8aebff4), the load commits 50000 rows, recovery runs, and the data is lost. 5 passed + 4 failed: and the columnar table holds what its heap mirror holds, at wal_skip_threshold 0: got [|50000] want [50000/42633332...|50000], the same at 64MB, and both truncate arms. The server log has columnar: zstd decompression failed or produced the wrong length (expected 90), and the twin goes red the same way. Restored: 9/9, md5 back to 766fad3c. So on a build like CI's, the four content arms catch this mutant. The four "committed its rows ... came back" arms stay green, and their dating rests on the assertion.
  • The premises the suite leaves to arithmetic hold when measured (PG18): during the load, WAL is 173 kB for a table created in the transaction against 1070 kB for a pre-existing one, so WAL is being skipped. At COMMIT, threshold 0 writes 1424 bytes and 64MB writes 926 kB for a 920 kB relation, so both branches run.
  • Green: 9/9 on PG17 and PG18, and the twin on each. The census, 2783/2096/687, matches the tsv, cluster_tests 566 matches collection, and the gate passes.

🤖 Generated with Claude Code

jdatcmd and others added 9 commits October 15, 2026 16:38
@jdatcmd @claude
fix: REINDEX reclaims the entries of deleted rows no snapshot can see (…
e8a1af5 
…#1407)

older snapshot still finds a recently deleted row by a new index. But it
emitted every deleted row, where heap's build skips a tuple dead to all.
So REINDEX, documented as what reclaims stale entries, reclaimed none:
9 rows in 10 deleted, and the rebuilt index kept its size where heap's
shrank to a tenth.

The build now also reads each row group's delete marks under a horizon
snapshot (xmin = xmax = the oldest non-removable xid, curcid 0), which
sees exactly the transactions committed before every running snapshot
began, and leaves out a row deleted there. The snapshot is registered:
PG 18+ asserts it in the visibility check.

am_contracts gains the arm in both twins, red on main (kept:2260992->
2260992); with the horizon moved to the next xid, the older-snapshot arm
goes red instead. limitations.md says when REINDEX keeps an entry.
cluster_tests +1.

Fixes #1407.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd @claude
test: am_contracts' older-snapshot arm waits for its events instead o…
7aa53fa 
…f sleeping

The arm from #1364 held a REPEATABLE READ snapshot, slept four seconds,
and assumed the DELETE and CREATE INDEX finished inside them. On a loaded
box they did not: the plan was taken before the index existed, and
'premise: the older snapshot's lookup is an index scan' read 0 on PG16.
A 5-second delay before CREATE INDEX reproduces it on PG17 every time.

Now a gate session holds an advisory lock. Session 1 takes its snapshot,
names itself, and queues for the lock. The script waits for the name,
deletes, builds the index, then ends the gate. With the same delay the
suite passes 28/28. A new premise in both twins says the older snapshot
counted all 10000 rows, so it predates the delete. The arm still goes red
when the index build skips deleted rows again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd @claude
test: ALTER TABLE forms that read every row see the live rows and onl…
fc6a07a 
…y those (#1341)

A new suite, alter_table_scans, and its pytest twin. A columnar table
keeps a deleted row in its file until compaction, so each form must see
the live rows and only those. On a columnar table and on heap with the
same rows:

- SET NOT NULL, ADD PRIMARY KEY and VALIDATE CONSTRAINT are refused over
  a NULL, a duplicate and a breaking row (23502, 23505, 23514), and each
  succeeds once that row is deleted;
- ALTER COLUMN TYPE and ADD COLUMN with a volatile default rewrite the
  table, keep the live rows and leave the deleted ones gone;
- afterwards the primary key finds id 1 once, through the index.

15/15 on each of PG15-19, and the twin on each. Ten checks are dated on
PG17 by two mutants: every scan returning deleted rows, and the index
build marking every row alive. cluster_tests 565 -> 566 by collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd @claude
test: drop a no-op autocommit assignment from the twin
de6e61d 
pgc_conn is a write-watching wrapper, so setting .autocommit on it sets
an attribute on the wrapper and never reaches the connection (found
while writing #1419). The line here set it to the value it already had,
so it did nothing, but it read as if it mattered. The twin still passes
on PG17.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd @claude
test: the settings reference is held to the server, and two settings …
292fa3b 
…are set (#1341 items 42, 57, 83)

docs/configuration.md is compared with pg_settings both ways by a new suite,
settings_reference, and its pytest twin:

- every setting the server defines is documented, except those it marks
  NOT_IN_SAMPLE (developer hooks and internal switches);
- every documented setting exists;
- each documented default is the server's boot_val;
- each row's condition (superuser, only at server start, Reloadable, or
  none) is the server's context.

Item 57: a role that is not a superuser, on its own login, is refused
enable_end_truncation with 42501, and a superuser sets it. Item 83:
compression_level reaches the zstd codec, so the same text stores smaller
at level 19 than at level 1.

8/8 on each of PG15-19, and the twin 4/4. All 8 checks are dated on PG17
by seven mutants (two GUC contexts, a default, try_zstd ignoring the
level, and three edits to the reference). cluster_tests 564 -> 568 by
collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd @claude
test: settings_reference's premise carries its count, and no captured…
342dd60 
… string is piped into grep -q (#1341)

The harness selftest refused two lines. The thirty-settings premise
reduced the row count to 1 or 0, so a red said nothing about how many rows
were parsed (rule 540); it now reads 'no (N rows parsed from DOC)'. The
zstd test piped q into grep -q, an early-exit reader (rule 080); it now
asks the server whether zstd is among the enum values. The selftest was 2
failed of 1245 on PG17 and is now 0 failed; the suite is 8/8 on each of
PG15-19.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd @claude
test: settings_reference sorts each side again after its cut, so comm…
1e4898f 
… names only what differs (#1341)

comm compares whole lines, but the reference's rows were sorted on the name
field alone. In C order `autovacuum|` sorts after `autovacuum_compact_threshold|`,
so the default and condition arms fed comm unsorted input: a red arm named
ten entries for one wrong row, and every run logged comm's sort warnings.
Found by OffgridwithJD in review.

With "Reloadable" taken out of the pgcolumnar.autovacuum row, the condition
arm went from ten entries and six warnings to exactly
`pgcolumnar.autovacuum|sighup pgcolumnar.autovacuum|user` and none. A wrong
default on the same row names only that row. Green stays 8/8 on PG17 and
PG18, with no warnings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd @claude
test: megabyte-scale values are stored exactly (#1341)
a22a224 
A columnar table has no TOAST table: the writer detoasts every value and
stores it inline. The largest value any test stored was 70 KB.

A new suite, large_values, and its pytest twin. Values go into heap and
are copied to a columnar table, and the two must agree by length and md5:
1 MB and two 16 MB incompressible values, a 200 MB compressible one, and
an UPDATE that takes a value heap has TOASTed. An opt-in arm
(PGC_RUN_OVER_1GB=1) checks that a chunk over 1 GB is refused with 54000,
commits nothing and leaves the server up; it needs several GB of memory,
so CI skips it. It passed when run by hand on PG17 (38 s). The twin does
not carry it, and test_compare_to_bash.py declares the pair incomplete.

5/5 plus the skip on each of PG15-19, and the twin on each. Three checks
are dated on PG17 by a mutant that stores a TOASTed value as its pointer.
Skipping only the write path's detoast reddened nothing, because the
encoder flattens again. cluster_tests 565 -> 566.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd @claude
test: tables that skip WAL at wal_level=minimal survive a crash (#1341)
fba6b45 
At wal_level=minimal, PostgreSQL skips WAL for a table created or
truncated in the transaction that loads it and makes it durable at
COMMIT: below wal_skip_threshold it writes the new pages to WAL, at or
above it it fsyncs the files. No suite ran at wal_level=minimal.

A new suite, recovery_minimal_wal, and its pytest twin on a private
cluster. A columnar table and a heap mirror are created and loaded in
one transaction, then truncated and reloaded in one, at wal_skip_threshold
0 and 64MB. Each arm reads the committed row count before the crash, so a
load that failed cannot pass as two empty tables that agree; my first
draft lacked that and passed a mutant vacuously. After crash recovery the
columnar table must hold what its heap mirror holds.

9/9 on each of PG15-19, and the twin on each. Eight checks are dated on
PG17 by a mutant that creates the relfilenode without registering its
commit-time sync; on this assert-enabled build relcache.c's
RelFileLocatorSkippingWAL assertion then fails during the load.
cluster_tests 565 -> 566.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
@jdatcmd
jdatcmd force-pushed the test/1341-minimal-wal branch from e7acae3 to fba6b45 Compare October 15, 2026 17:28
@jdatcmd jdatcmd mentioned this pull request Oct 15, 2026
Merged
@jdatcmd

jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, restated as fba6b45, on top of #1425 in my merge chain (#1412 → #1417 → #1418 → #1414 → #1425 → #1427). The changed lines are identical to e7acae3, the head you approved, apart from re-derived bookkeeping: the budget paragraph, which matches the census derived from the tsv; cluster_tests, which matches collection; and the TESTS.md section number, now 118. The gate gives rc=0 and the guard set passes 435/435.

🤖 Generated with Claude Code

@jdatcmd jdatcmd mentioned this pull request Oct 15, 2026
Merged
OffgridwithJD
OffgridwithJD approved these changes Oct 15, 2026
View reviewed changes

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at fba6b45, stacked on #1425.

  • Same patch: this PR's own commits match what I approved at e7acae3: the same code and the same ledger rows, with TESTS.md the same apart from section numbers.
  • Census: 3007 / 2282 / 725 from the tsv, matching the paragraph. suites_not_covered is 231.
  • CI at the time of review: 7 pass, 6 pending.

🤖 Generated with Claude Code

@OffgridwithJD

OffgridwithJD commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

@jdatcmd A note for this one, found while re-reviewing #1424. The mutation text on the 8 recovery_minimal_wal rows contains a "; ", and pgc_ledger.py merge splits mutation text on ";". So the next merge that touches these rows, as #1424's did, turns the sentence into two "mutants", one of them a fragment with a leading space. Replacing the ";" (e.g. ", and this assert-enabled build ...") before this lands would keep it whole. My approval of fba6b45 stands either way.

🤖 Generated with Claude Code

@jdatcmd jdatcmd mentioned this pull request Oct 15, 2026
Merged
@jdatcmd
jdatcmd merged commit f2bd5f1 into main Oct 15, 2026
15 checks passed
This was referenced Oct 15, 2026
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
This was referenced Oct 15, 2026
Merged
Merged
Merged
@jdatcmd jdatcmd mentioned this pull request Oct 15, 2026
Merged
This was referenced Oct 15, 2026
Merged
Merged
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Oct 15, 2026
@jdatcmd @claude
test: the ledger refuses a mutation split on ';', and the 12 split ro…
a310c40 
…ws are reworded

The mutation column is a ';'-separated set, written back sorted. A --mutation
whose text held "; " was stored as two mutations, one a fragment starting with
a space, and every later merge re-sorted it. commandprompt#1427's recovery_minimal_wal text
did this to 8 rows, and commandprompt#1423, commandprompt#1424 and commandprompt#1426 each carried them as a change of
their own. Four older harness_selftest rows were split the same way.

- merge refuses a --mutation holding ';' (rc 2, ledger untouched).
- the reader refuses a ';'-piece with whitespace at an end, by line: the writer
  joins with a bare ';', so such a piece is a fragment, never a mutation.
- the 12 rows are reworded without ';', from their original text in history.
  Only field 6 changes; the census stays 3007 / 2282 / 725.

Red first: the two refusal arms got 0 want 2 on the unchanged tool, and the
committed-ledger arm was red at line 1783 with the refusal in and the rows not
yet reworded. guard_tests 435 -> 438 by collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Oct 15, 2026
@jdatcmd @claude
test: the ledger refuses a mutation split on ';', and the 12 split ro…
211ec25 
…ws are reworded

The mutation column is a ';'-separated set, written back sorted. A --mutation
whose text held "; " was stored as two mutations, one a fragment starting with
a space, and every later merge re-sorted it. commandprompt#1427's recovery_minimal_wal text
did this to 8 rows, and commandprompt#1423, commandprompt#1424 and commandprompt#1426 each carried them as a change of
their own. Four older harness_selftest rows were split the same way.

- merge refuses a --mutation holding ';' (rc 2, ledger untouched).
- the reader refuses a ';'-piece with whitespace at an end, by line: the writer
  joins with a bare ';', so such a piece is a fragment, never a mutation.
- the 12 rows are reworded without ';', from their original text in history.
  Only field 6 changes; the census stays 3007 / 2282 / 725.

Red first: the two refusal arms got 0 want 2 on the unchanged tool, and the
committed-ledger arm was red at line 1783 with the refusal in and the rows not
yet reworded. guard_tests 435 -> 438 by collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Oct 15, 2026
@jdatcmd @claude
test: the ledger refuses a mutation split on ';', and the 12 split ro…
d0ca319 
…ws are reworded

The mutation column is a ';'-separated set, written back sorted. A --mutation
whose text held "; " was stored as two mutations, one a fragment starting with
a space, and every later merge re-sorted it. commandprompt#1427's recovery_minimal_wal text
did this to 8 rows, and commandprompt#1423, commandprompt#1424 and commandprompt#1426 each carried them as a change of
their own. Four older harness_selftest rows were split the same way.

- merge refuses a --mutation holding ';' (rc 2, ledger untouched).
- the reader refuses a ';'-piece with whitespace at an end, by line: the writer
  joins with a bare ';', so such a piece is a fragment, never a mutation.
- the 12 rows are reworded without ';', from their original text in history.
  Only field 6 changes; the census stays 3007 / 2282 / 725.

Red first: the two refusal arms got 0 want 2 on the unchanged tool, and the
committed-ledger arm was red at line 1783 with the refusal in and the rows not
yet reworded. guard_tests 435 -> 438 by collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@OffgridwithJD OffgridwithJD OffgridwithJD approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jdatcmd @OffgridwithJD