Skip to content

test: the settings reference is held to the server (#1341 items 42, 57, 83) - #1414

Merged
jdatcmd merged 7 commits into
mainfrom
test/1341-settings-reference
Oct 15, 2026
Merged

jdatcmd merged 7 commits into
mainfrom
test/1341-settings-reference

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

Covers #1341 items 42, 57 and 83. Test-only: no source change.

What it adds

A new suite, test/settings_reference.sh, and its pytest twin, test_settings_reference.py. They hold docs/configuration.md to pg_settings in both directions:

  • Item 42. Every setting the server defines is documented. The exceptions are settings the server marks NOT_IN_SAMPLE: the developer hooks and internal switches (rollup_pause, tier_fault_after, bulk_parallel_writer). Every documented setting exists. Each documented default equals boot_val. Each row's stated condition matches context:
    • "superuser" → superuser;
    • "only at server start" → postmaster;
    • "Reloadable" → sighup;
    • none of these → user.
  • Item 57. A role that is not a superuser, on its own login, is refused SET pgcolumnar.enable_end_truncation with 42501. A superuser can set it.
  • Item 83. compression_level reaches the zstd codec: the same text stores smaller at level 19 than at level 1. A build without zstd cannot run this check.

A premise check needs at least thirty parsed rows, so a format change that parses nothing cannot pass the comparisons vacuously.

Evidence

  • Green: 8/8 on each of PG15–19, and the twin 4/4 on each.
  • All 8 checks were seen red on PG17, one mutant at a time. Both twins went red on the first four:
mutant checks that went red
enable_end_truncation defined PGC_USERSET condition; 42501 refusal (got ok)
enable_end_truncation defined PGC_POSTMASTER condition; refusal (got 55P02); superuser control
compression_level default 3 → 5 default
try_zstd passes level 1, not the setting level (1073152 vs 1073152)
the compression_level row deleted from the reference undocumented
the row renamed compression_levl existence, undocumented, default, condition
the backticks removed from every name the thirty-settings premise, undocumented
  • Ledger: 2774 + 8 = 2782 rows, 2095 never (unchanged), 687 dated. The gate passes against origin/main. The ceiling stays 234.
  • Registration: run_all_versions.sh, test_compare_to_bash.py COMPLETE, TESTS.md §115. cluster_tests goes 565 → 569 by collection. The guard set passes 433/433, including test_collation_pinned.py: every sort, comm and join sets LC_ALL=C itself.

🤖 Generated with Claude Code

https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Reviewed a9484e3.

  • What it holds: the reference and pg_settings name the same settings (NOT_IN_SAMPLE aside), each documented default is boot_val, and each documented condition is context. A plain login is refused enable_end_truncation with 42501 and a superuser can set it. zstd level 19 stores less than level 1. The two parsers agree: over today's configuration.md both give the same 45 rows.
  • Red (PG17): enable_end_truncation as PGC_USERSET reddens the condition arm and a role that is not a superuser cannot set enable_end_truncation (42501): got [ok] want [42501], in both twins. try_zstd passing level 1 gives got [1073152 vs 1073152] want [less] in both. The .so md5 changed for each mutant and came back on restore.
  • Green: 8/8 and 4 passed on PG17 and PG18. The guard half gives 433 passed, cluster_tests is 569 by collection, the census (2782 / 2095 / 687) matches, and the gate passes at 234.
  • Blocker: the shell's default and condition arms feed comm input it doesn't consider sorted, so a red arm names settings that are correct. doc_rows sorts on field 1 only (sort -t'|' -k1,1), but comm compares whole lines. In C order | sorts after _, so pgcolumnar.autovacuum|… belongs after pgcolumnar.autovacuum_compact_threshold|…. With "Reloadable" taken out of the pgcolumnar.autovacuum row only, pytest names one setting (pgcolumnar.autovacuum: doc user, server sighup). The shell names ten entries, five settings on each side, four of them unchanged: got [pgcolumnar.autovacuum|sighup pgcolumnar.autovacuum_compact_threshold|sighup pgcolumnar.autovacuum_expire|sighup pgcolumnar.autovacuum_naptime|sighup pgcolumnar.autovacuum_recluster_threshold|sighup pgcolumnar.autovacuum|user …]. Every run, green ones included, also logs comm: file 1 is not in sorted order … input is not in sorted order twice. The verdict is still right (I found no way for a difference to come out as none), but the failure text points at the wrong settings, and the stderr noise is constant. Sorting each side after the cut fixes it: comm -3 <(cut -d'|' -f1,3 <<<"$docd" | LC_ALL=C sort) <(cut -d'|' -f1,3 <<<"$srv_docd" | LC_ALL=C sort), and the same for -f1,2. With that change, the same mutation reports exactly got [pgcolumnar.autovacuum|sighup pgcolumnar.autovacuum|user] with no order warnings, and green stays 8/8.

🤖 Generated with Claude Code

@jdatcmd

jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, fixed in b1006ec as you suggested: each side of the default and condition comparisons is sorted again after its cut.

  • Your case, before the fix (PG17): with "Reloadable" taken out of the pgcolumnar.autovacuum row only, the condition arm named ten entries, and the run logged six comm sort-order warnings.
  • After: the same mutation gives exactly got [pgcolumnar.autovacuum|sighup pgcolumnar.autovacuum|user], with 0 warnings.
  • The default arm: with that row's documented default changed from off to on, it gives exactly got [pgcolumnar.autovacuum|off pgcolumnar.autovacuum|on], with 0 warnings.
  • Green: 8/8 on PG17 and PG18, with 0 warnings.
  • The other two comm calls (names only, lines 58 and 60) were already safe: each side is the name field alone, which sort -k1,1 and ORDER BY name COLLATE "C" both put in C order. I left them as they were.

No check names changed, so the ledger rows and the census don't move. The pytest twin is untouched.

🤖 Generated with Claude Code

jdatcmd and others added 4 commits October 15, 2026 16:38
…#1407)

older snapshot still finds a recently deleted row by a new index. But it
emitted every deleted row, where heap's build skips a tuple dead to all.
So REINDEX, documented as what reclaims stale entries, reclaimed none:
9 rows in 10 deleted, and the rebuilt index kept its size where heap's
shrank to a tenth.

The build now also reads each row group's delete marks under a horizon
snapshot (xmin = xmax = the oldest non-removable xid, curcid 0), which
sees exactly the transactions committed before every running snapshot
began, and leaves out a row deleted there. The snapshot is registered:
PG 18+ asserts it in the visibility check.

am_contracts gains the arm in both twins, red on main (kept:2260992->
2260992); with the horizon moved to the next xid, the older-snapshot arm
goes red instead. limitations.md says when REINDEX keeps an entry.
cluster_tests +1.

Fixes #1407.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
…f sleeping

The arm from #1364 held a REPEATABLE READ snapshot, slept four seconds,
and assumed the DELETE and CREATE INDEX finished inside them. On a loaded
box they did not: the plan was taken before the index existed, and
'premise: the older snapshot's lookup is an index scan' read 0 on PG16.
A 5-second delay before CREATE INDEX reproduces it on PG17 every time.

Now a gate session holds an advisory lock. Session 1 takes its snapshot,
names itself, and queues for the lock. The script waits for the name,
deletes, builds the index, then ends the gate. With the same delay the
suite passes 28/28. A new premise in both twins says the older snapshot
counted all 10000 rows, so it predates the delete. The arm still goes red
when the index build skips deleted rows again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
…y those (#1341)

A new suite, alter_table_scans, and its pytest twin. A columnar table
keeps a deleted row in its file until compaction, so each form must see
the live rows and only those. On a columnar table and on heap with the
same rows:

- SET NOT NULL, ADD PRIMARY KEY and VALIDATE CONSTRAINT are refused over
  a NULL, a duplicate and a breaking row (23502, 23505, 23514), and each
  succeeds once that row is deleted;
- ALTER COLUMN TYPE and ADD COLUMN with a volatile default rewrite the
  table, keep the live rows and leave the deleted ones gone;
- afterwards the primary key finds id 1 once, through the index.

15/15 on each of PG15-19, and the twin on each. Ten checks are dated on
PG17 by two mutants: every scan returning deleted rows, and the index
build marking every row alive. cluster_tests 565 -> 566 by collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
pgc_conn is a write-watching wrapper, so setting .autocommit on it sets
an attribute on the wrapper and never reaches the connection (found
while writing #1419). The line here set it to the value it already had,
so it did nothing, but it read as if it mattered. The twin still passes
on PG17.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N

@OffgridwithJD OffgridwithJD left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at b1006ec. The sort fix does what it says.

Measured on PG18, with "Reloadable" changed to "Restart-only" in the pgcolumnar.autovacuum row of docs/configuration.md:

tree result comm order warnings
b1006ec, docs clean 8 passed + 0 failed 0
b1006ec, docs mutated 7 + 1: each documented condition is the server's context for the setting: got [pgcolumnar.autovacuum|sighup pgcolumnar.autovacuum|user] want [none] 0
the sort reverted to a9484e3's, docs mutated 7 + 1: got [pgcolumnar.autovacuum|sighup pgcolumnar.autovacuum_compact_threshold|sighup pgcolumnar.autovacuum_expire|sighup ... (the ten-entry list again) 6
restored 8 passed + 0 failed 0

I agree the two name-only comm calls were already safe. No CI had reported on b1006ec at the time of review, probably because it conflicts with main; please rebase so it runs.

🤖 Generated with Claude Code

jdatcmd and others added 3 commits October 15, 2026 17:25
…are set (#1341 items 42, 57, 83)

docs/configuration.md is compared with pg_settings both ways by a new suite,
settings_reference, and its pytest twin:

- every setting the server defines is documented, except those it marks
  NOT_IN_SAMPLE (developer hooks and internal switches);
- every documented setting exists;
- each documented default is the server's boot_val;
- each row's condition (superuser, only at server start, Reloadable, or
  none) is the server's context.

Item 57: a role that is not a superuser, on its own login, is refused
enable_end_truncation with 42501, and a superuser sets it. Item 83:
compression_level reaches the zstd codec, so the same text stores smaller
at level 19 than at level 1.

8/8 on each of PG15-19, and the twin 4/4. All 8 checks are dated on PG17
by seven mutants (two GUC contexts, a default, try_zstd ignoring the
level, and three edits to the reference). cluster_tests 564 -> 568 by
collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
… string is piped into grep -q (#1341)

The harness selftest refused two lines. The thirty-settings premise
reduced the row count to 1 or 0, so a red said nothing about how many rows
were parsed (rule 540); it now reads 'no (N rows parsed from DOC)'. The
zstd test piped q into grep -q, an early-exit reader (rule 080); it now
asks the server whether zstd is among the enum values. The selftest was 2
failed of 1245 on PG17 and is now 0 failed; the suite is 8/8 on each of
PG15-19.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
… names only what differs (#1341)

comm compares whole lines, but the reference's rows were sorted on the name
field alone. In C order `autovacuum|` sorts after `autovacuum_compact_threshold|`,
so the default and condition arms fed comm unsorted input: a red arm named
ten entries for one wrong row, and every run logged comm's sort warnings.
Found by OffgridwithJD in review.

With "Reloadable" taken out of the pgcolumnar.autovacuum row, the condition
arm went from ten entries and six warnings to exactly
`pgcolumnar.autovacuum|sighup pgcolumnar.autovacuum|user` and none. A wrong
default on the same row names only that row. Green stays 8/8 on PG17 and
PG18, with no warnings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd
jdatcmd force-pushed the test/1341-settings-reference branch from b1006ec to 1e4898f Compare October 15, 2026 17:26
@jdatcmd

jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, restated as 1e4898f, on top of #1418 (de6e61d) in my merge chain (#1412 → #1417 → #1418 → this). The changed lines are identical to b1006ec, the head you approved, except for re-derived bookkeeping:

The ledger rows are unchanged. The gate gives rc=0 and the guard set passes 435/435.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 1e4898f, stacked on #1418.

  • Same patch: this PR's own commits match what I approved at b1006ec: the same code and the same ledger rows, with TESTS.md the same apart from section numbers.
  • Census: 2992 / 2278 / 714 from the tsv, matching the paragraph. suites_not_covered is 231.
  • CI at the time of review: 12 pass, 1 pending.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit 3db7b34 into main Oct 15, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants