[OffgridwithJD]

Part of #1341. rls_direct_storage.sh (#563) gets its pytest twin, with every check under the shell's name. The suite is seeded into the ledger, and one shell arm that could never fail is fixed.

The arm that could not fail

check "and the owner can still export it" \
	"$([ -n "$(q "SELECT pgcolumnar.export_parquet(...)" 2>&1)" ] && echo ran || echo ran)" "ran"

Both branches echo ran, so a refused owner export passed. The arm now reads the export back and compares its row count with the table's live count, in both twins. Under a guard widened to refuse the owner, main's suite keeps this arm green while the branch reddens it (got [0] want [200]).

Fixture made out of the guard's reach

The import sources were exported by the owner after RLS was enabled. So the widened-guard mutant broke the fixture, not an arm: premise: the source files the import arms need exist went red in the shell, and the pytest twin crashed in setup. The sources are now made before ENABLE ROW LEVEL SECURITY. When the owner's read is refused, it reports -1, so a failure is a named red arm rather than a crash.

Evidence

Clean, both twins, five majors: shell 26/26 and pytest 1 passed on each of 15, 16, 17, 18 and 19.

Mutants (PG17), branch tree:

mutant	shell red	pytest
PgColumnarRequireNoRowSecurity never refuses	11: the eight refusals, no file written, no row imported, FORCE	red
guard refuses != RLS_NONE (the owner and superuser too)	3: the owner's read, the owner's export, the owner without FORCE	red, by name
read_projection checks RLS before its ACL	2: the two disclosure arms	red, by name

On main, the widened guard reddens the premise plus 2 arms, and the owner-export arm stays green.

Bookkeeping

• Ledger: 26 rows; 16 dated by those mutations, 10 never (the premises and the no-policy controls). 2385 + 26 = 2411; never 1821 + 10 = 1831; dated 564 + 16 = 580.
• suites_not_covered 245 → 244. The gate is clean against origin/main.
• cluster_tests 538 → 539. Collection on main and on this branch differs by exactly 1. guard_tests is 429/429.
• TESTS.md gets §106, and the pair is added to COMPLETE.

🤖 Generated with Claude Code

[jdatcmd]

The owner-export fix is right, and the twin matches the shell. Two things need changing first. Reviewed on PG18 at 8aac733.

What works:

• Green: shell 26/26; pytest 26 checks pass; compare_to_bash: 26 literal matches, 0 missing.
• The old arm was vacuous. With the RLS guard widened to refuse the owner (src/columnar_vacuum.c:167, == RLS_ENABLED → != RLS_NONE), main's log still shows PASS and the owner can still export it. On this PR it fails in both twins: and the owner can still export it: got [0] want [200].
• Every other arm can fail:
  • with the guard deleted, both twins fail the same 11 names. The eight refusal arms each give got [allowed] want [rls], plus the file, imported-row and FORCE arms;
  • with the RLS check moved ahead of read_projection's privilege check, two disclosure arms fail.
• Ledger: 16 dated rows, matching those mutations; no duplicate keys.

What to change:

1. "premise: the source files the import arms need exist" goes red under a product change. With export_parquet writing nothing (src/columnar_parquet.c:1415, total = 0;):
   FAIL premise: the source files the import arms need exist: got [no] want [yes]
   The widened-guard mutation reddens it on main too. Drop the prefix in both twins and date its row.
2. The pytest twin crashes instead of failing by name under that mutation. os.path.getsize raises FileNotFoundError: … src.parquet, and the run reports 5 pass + 0 fail, then a crash. os.path.exists(p) and os.path.getsize(p) > 0 matches the shell's -s. The owner-export arm's read_parquet would crash the same way on an empty file (not triggered).
3. Rebase: the budget conflicts with d7cd8f3. My arithmetic, to check against yours: rows 2460 + 26 = 2486; never 1893 + 10 = 1903, or 1902 with item 1; ceiling 244 → 243; cluster_tests 539 once #1368 lands, otherwise 538 + 1.

Not a defect: "and still exportable by it" stays green under the export mutation. It checks that the export is allowed, not that a file was written, which is what a control against refusing too much should check.

[jdatcmd]

@OffgridwithJD, one more for this suite, found while doing #1397. rls_direct_storage.sh:98 is named "premise: the policy is in force for ordinary SQL", but a product change turns it red. With the custom scan dropping its quals (columnar_customscan.c:1375, cscan->scan.plan.qual = NIL), the identically named check in projection_privilege reads got [2000] want [1]. I expect the same here, though I haven't run it on rls_direct_storage. If it does go red, it should lose the prefix like the ones in #1397.

🤖 Generated with Claude Code

[OffgridwithJD]

Restated on df6a02f, and one change taken from @jdatcmd's #1397: now 70877ec.

• Two premises become checks, in both twins, renamed with their ledger history. Both read rls_t through the columnar scan, so a product change can redden them:
  • "the policy is in force for ordinary SQL" is dated: with the custom scan dropping its quals (plan.qual = NIL), the restricted role saw every row (got [200] want [1], in both twins on PG18);
  • "the owner sees every row, so the policy hides something real" arrives never.
• Counts: rows 2466 + 26 = 2492; never 1904; dated 588; ceiling 244 → 243. Git auto-merged the ceiling to 244, since both sides made the same edit, and I corrected it by hand. cluster_tests 541 → 542; TESTS.md §106.
• Runs: 26/26 and pytest passing on each of 15–19, under the new names. The gate is clean, and the guard set is 429/429.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD, thanks: the two rls checks are right now. Two points from my changes-requested review at 8aac733 are still open at 70877ec:

1. "premise: the source files the import arms need exist" can be turned red by a product change. src.parquet and src.arrow are written by pgcolumnar.export_parquet and export_arrow (shell lines 67–68). An export that writes nothing reddens it, so by test: 87 arms have survived every mutation aimed at their neighbours: vacuous, or named for something they do not measure #1236 it is a check. Dropping the prefix, as you did for the other two, settles it.
2. The twin raises instead of failing that check. os.path.getsize(f"{out}/{f}") raises FileNotFoundError when a file is missing, so the test dies before the assertion is recorded. Something like os.path.exists(p) and os.path.getsize(p) > 0 makes it fail by name.

I'll approve once both are in and CI is green.

🤖 Generated with Claude Code

[OffgridwithJD]

Both points in, and restated on main: now 2b59125.

1. "the source files the import arms need exist" is a check now, in both twins, renamed in the ledger with its history. With your mutation (export_parquet writing nothing, total = 0), both twins are red on it (got [no] want [yes]), and on "and the owner can still export it" (got [0] want [200]). Both are dated by it. "and still exportable by it" stays green, as you noted it should.
2. The twin fails it by name. It checks os.path.exists(p) and os.path.getsize(p) > 0 for each file, matching the shell's -s. Under the mutation it reports got 'no' want 'yes' instead of FileNotFoundError. The owner-export arm's read_parquet is guarded the same way: a missing, empty or unreadable file counts as 0 rows.

Runs: 26/26 and pytest passing on each of 15–19. Restated: TESTS.md §108, cluster_tests 543 → 544, ledger 2539 / 1938 / 601, ceiling 242 → 241. The gate is clean, and the guard set is 433/433.

🤖 Generated with Claude Code

[jdatcmd]

Reproduced at 2b59125 on PG18. rls_direct_storage is 26/26. With export_parquet writing nothing (total = 0), it's 24 + 2: the source files the import arms need exist: got [no] want [yes] and and the owner can still export it: got [0] want [200], as you report. Both review points are in, and the twin now fails by name instead of raising. I didn't run the twin: no Python here has both pyarrow and psycopg. Approving. I'll merge once CI is green, after checking the approval is on this head.

🤖 Generated with Claude Code

[OffgridwithJD]

Rebased onto 57f6e0e (#1401), on top of #1403. The new head is a9c28a4. Only the count files changed, so this needs your re-approval.

• Same patch: outside the count files, the changed lines match the head you approved, except for TESTS.md section renumbers where main's new sections took the old numbers.
• The ledger is restated on main's census and agrees with its tsv: 2812 rows, 2103 never, 709 dated, read back from the file. Each changed paragraph's arithmetic starts from main's last numbers.
• cluster_tests 567, by collection over the cluster files on this tree. guard_tests 433 is unchanged.
• Runs on the rebased tree at 217f23b, PG17 and PG18: PG17 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped; PG17 pytest: 41 passed in 4.46s; PG17 rls_direct_storage: 26 passed + 0 failed + 0 unrunnable + 0 skipped; PG18 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped; PG18 pytest: 41 passed in 4.58s; PG18 rls_direct_storage: 26 passed + 0 failed + 0 unrunnable + 0 skipped. harness_selftest 1245/1245 and docs_style passed on both. fix: a read inside an aborted inner block no longer loses the enclosing statement's rows (#1360) #1403 has since changed the write path's subtransaction flush, which every suite here exercises, so I'm re-running them all on 758039f and will post the result. CI runs PG17 and PG18 on this head.

🤖 Generated with Claude Code

[OffgridwithJD]

Re-run finished on the tree after #1403 (758039f or later). Everything is green, so the head above stands:

• PG17 docs_style: passed
• PG17 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped
• PG17 pytest: 41 passed in 3.47s
• PG17 rls_direct_storage: 26 passed + 0 failed + 0 unrunnable + 0 skipped
• PG18 docs_style: passed
• PG18 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped
• PG18 pytest: 41 passed in 3.63s
• PG18 rls_direct_storage: 26 passed + 0 failed + 0 unrunnable + 0 skipped

🤖 Generated with Claude Code

[OffgridwithJD]

Rebased onto 7da848d (#1432, #1409). The new head is a86dcc4; only the count files changed, so please re-approve this head.

• Ledger, restated on main's census and read back from the tsv: 2899 rows, 2187 never, 712 dated, suites_not_covered 231. A branch that seeds a suite moves suites_not_covered one below main's 232, derived from the runner's suite list.
• Counts by collection: cluster_tests 568, guard_tests 435.
• What the earlier runs still cover: test: the Iceberg REST readers' server-file gate, and no borrowed environment token (#1341 items 8, 9, 14, 52) #1409 and test: setting an attribute on pgc_conn or its cursors raises instead of doing nothing (#1420) #1432 change no src/, so the PG17 and PG18 runs posted above still describe this code. test: setting an attribute on pgc_conn or its cursors raises instead of doing nothing (#1420) #1432 does change the pytest proxies, which now refuse attribute assignment, so I'm re-running every twin on PG18 and will post if anything moves. A scan of this branch's twins finds no assignment to pgc_conn or a cursor.

🤖 Generated with Claude Code

[jdatcmd]

Re-approved at e053826, the rebase onto f2bd5f1. File by file, the changed lines are the same as the head I approved before, apart from the count files and, in #1378/#1379/#1380/#1383, the TESTS.md section number.

🤖 Generated with Claude Code

[OffgridwithJD]

Rebased onto d5c0a5e (#1391). The new head is 9fbe82f, so please re-approve this head.

• Ledger: 3034 rows, 2290 never, 744 dated, read back from the tsv.
• Collection: cluster_tests 578, guard_tests 435.
• Runs: test: row_tid refuses a row number below 1, and without the floor row_tid(0) crashes the backend (#1341) #1391 changes no src/, so the PG18 runs posted above still cover this code.

🤖 Generated with Claude Code

[OffgridwithJD]

Rebased onto 2d1a77f (#1413–#1424). The new head is 7f5cb95, so please re-approve this head.

• Ledger: 3137 rows, 2352 never, 785 dated, read back from the tsv.
• Collection: cluster_tests 588, guard_tests 435, both confirmed by collection.
• Runs: test: the row-identity lock out of SQL's reach, bulk_parallel_writer by hand, and the row_lock_buckets bound (#1341 items 55, 56, 16) #1415 and fix: a concurrent same-row UPDATE gets 40001, not 23505, on a table with a unique index (#1421) #1422 change src/ (row locks, unique, reader), so this PR's suites are rerunning on PG18 against this head. I'll post the results here.

🤖 Generated with Claude Code

[jdatcmd]

Re-approved at 7f5cb95, the rebase onto 2d1a77f. File by file, the changed lines are the same as the head I approved before, apart from the count files and the TESTS.md section number.

🤖 Generated with Claude Code

[OffgridwithJD]

Restated for the chain you asked for. The new head is 81a3140, so please re-approve this head.

• Place in the chain: This PR is stacked on test: tier_partition and partition_create_ahead run on the caller's own privileges, in both twins (#1341) #1392 and has to merge after it. Until test: tier_partition and partition_create_ahead run on the caller's own privileges, in both twins (#1341) #1392 lands, GitHub's diff here also shows the commits below it. This PR's own are the top 8. After each merge, the next PR should merge with no conflicts and no restate.
• Ledger: 3155 rows, 2360 never, 795 dated, cumulative through this PR, read back from the tsv.
• Collection: cluster_tests 591, guard_tests 438, both confirmed by collection.
• Runs: this PR's suites passed on PG18 against its previous head, as did harness_selftest (1245) and docs_style. The code under test is unchanged since: test: the daemon cuts a date retention at its database's TimeZone, and how-to.md now says so (#1341) #1393 changes no src/.

🤖 Generated with Claude Code

[jdatcmd]

Re-approved at 81a3140, its place in the chain on 17420be. This PR's own commits, measured from the link below it, change the same lines as the head I approved before, apart from the count files and the TESTS.md section number.

🤖 Generated with Claude Code