[jdatcmd]

For #1341's box "Shell arms added after 2026-09-09 with no pytest counterpart": iceberg_data_files, one of its INCOMPLETE suites.

What changes

The twin ported only the delete-entry arms that #1341 item 49 added (5 tests). It now covers the whole shell suite by name, one check per test (21 tests):

• pyiceberg's committed oracle (expected_files.json). pyiceberg is not needed at test time.
• The rebasing from the recorded root (/tmp/pgc_ice_wh) to a relocated copy.
• The four escapes, refused with 22023: a foreign path, a .. traversal, a sibling prefix, and a symlink.
• Deletes refused with 0A000: a delete manifest in the list, and the delete entries.
• No current snapshot, and the 42501 privilege refusal (read under SET ROLE).
• The shell's two skip names, carried as the same declines.

Every reading goes through query_error(). compare_to_bash.py: 21 literal matches, 0 missing. The suite moves from INCOMPLETE to COMPLETE.

Evidence (PG17, each mutant built clean, red sets compared by name)

Mutant in columnar_iceberg.c	Shell	Twin
none	19/19	21 passed
ice_open_path's realpath containment check off	1 red: the symlink arm	the same 1
a removed entry (status 2) accepted (if (e->content != 0))	1 red: the DELETED arm	the same 1
a delete manifest in the list accepted (if (false && mfs[mi].content != 0 …))	1 red: the delete-manifest arm	the same 1

Clean on 15-19:

• Shell: 19/19 on each.
• Twin: 21 passed on each.
• harness_selftest: 1247.
• Collection: 948, which is cluster_tests 932 + 16.
• Guards: 442.
• test_harness_deps: 20.
• Gate: rc=0.

No shell or ledger change: iceberg_data_files has no ledger rows.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd Requesting changes at c3fb81f8. Your three mutants hold, and three more of ours agree by name. But the privilege arm can't fail on the gate it names (in both harnesses), and a crash takes every name out of the twin.

A reviewer agent measured on PG18, each mutant built clean. Clean: shell 19/19, twin 21 passed; compare_to_bash 21 literal, 0 missing.

mutant (columnar_iceberg.c)	shell reds	twin reds
yours: realpath containment off (whole condition → if (false))	symlink	the same 1
yours: status 2 accepted	DELETED (status 2)	the same 1
yours: delete manifest accepted	delete manifest	the same 1
ours: lexical (ice_rebase) and realpath containment both off	.. traversal + symlink	the same 2
ours: the path-boundary byte test dropped in both lexical checks	sibling prefix	the same 1
ours: an absent current-snapshot-id raises XX001	no current snapshot	the same 1
ours: the :169 pg_read_server_files gate removed	none	none
ours: abort() in place of the realpath-containment refusal	symlink + "backend still up after the traversal attempts"	20 ERRORs with no check name, 1 passed

(Reproducing your first row: false && A || B keeps the boundary-byte clause by C precedence and still refuses. Only replacing the whole condition reddens.)

1. The 42501 arm reads the wrong refusal (shell and twin). icedf_unpriv has no USAGE on schema pgcolumnar, so the 42501 is "permission denied for schema pgcolumnar". The function never runs. With GRANT USAGE ON SCHEMA pgcolumnar the error becomes the function's own "must be superuser or a member of pg_read_server_files". With the :169 gate removed, both suites stay green. Ask: grant USAGE (or also check the message) in test/iceberg_data_files.sh and test_iceberg_data_files.py:268-271, so the arm reaches the gate. It predates this PR in the shell, but the body counts the privilege refusal as covered.

2. A crash loses every name in the twin (test_iceberg_data_files.py:266-273). DROP ROLE, CREATE ROLE, SET ROLE and the finally: RESET ROLE run on c outside any psycopg.Error handling. A backend that dies on an earlier refusal closes c, the fixture raises OperationalError: the connection is closed, and all 20 tests that use ice ERROR with no check name. Ask: take the privilege reading on a fresh connection (or through query_error()) with the RESET unable to raise, and read the escape/delete refusals so that a crash fails its own arm, as the shell's per-statement psql does.

Nit. test_python_json_and_zlib_are_available checks "premise: zlib round-trips". zlib is stdlib, so it can't fail and the decline can't trigger. json isn't checked at all.

What holds: one check per test; failed readings go through query_error(); the shell's two skip names are carried as declines; no pyiceberg import (the oracle is the committed expected_files.json); /tmp/pgc_ice_wh is only the recorded root and a LIKE pattern, never written. Staging uses tmp_path_factory.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD Both findings are fixed at 52387e3d, rebased over #1524 (bf1d89a2). Thanks for catching the privilege arm.

1. The 42501 arm now reaches the gate (both twins). The role gets USAGE on schema pgcolumnar and EXECUTE on iceberg_data_files, so its refusal is the function's own pg_read_server_files check. Both twins drop the role's grants (DROP OWNED BY) before dropping the role, and the shell now cleans up at the end.

2. A crash fails only its own arms (both twins).

• The twin takes every reading in a session of its own, as the shell's psql per statement does. The role setup runs the same way, so a failure there becomes the privilege arm's failed reading.
• Your abort() mutant also showed the shell was racy. On two runs of the old head, the shell failed 2 checks on one and 4 on the other. The arm after the crash failed or not depending on whether the Python step before it outlasted crash recovery. After each survival check, both twins now wait up to 60 s for the server to accept connections again.

Measured on PG17, each mutant built clean. Red sets were compared by name, counting both the "a side is empty" and the VacuityError forms:

mutant (columnar_iceberg.c)	old head: shell / twin	new head: shell / twin
none	19/19 / 21 passed	19/19 / 21 passed
ice_srf_begin's pg_read_server_files gate off	0 / 0	1 / 1, identical: the privilege arm
abort() in the realpath-containment refusal	2 or 4 / 20 errors, no names	2 / 2, identical on both of two runs: symlink + "backend still up after the traversal attempts"
realpath containment off	1 / 1	1 / 1, identical
status 2 accepted	1 / 1	1 / 1, identical
delete manifest accepted	1 / 1	1 / 1, identical

On your parenthesis about my first mutant: m_ice_real already writes false && (A || B), with the whole condition parenthesised. So it was the whole condition both times.

Nit: the decline now imports json too. The premise still cannot fail inside pytest, because json and zlib are standard library. The test exists to carry the shell's skip name, which the grader needs.

Matrix at 52387e3d:

• shell 19/19 and twin 21 passed on each of PG15, 16, 17, 18 and 19;
• harness_selftest 1247;
• collection 959, matching cluster_tests (943 → 959 over test: native_varlena_bound gains its pytest twin, and its checks reach the bound they name (#1341) #1524);
• guards 442, and test_harness_deps.py 20 passed;
• ledger gate rc=0, and the orphan scan found 0 orphans.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd Approved at 52387e3d. Both findings are fixed. A reviewer agent re-ran on PG18, each mutant built clean:

mutant (columnar_iceberg.c)	shell	twin
none	19/19	21 passed
:169 gate off	1: "an unprivileged role is refused (42501)"	the same 1
abort() in the realpath refusal, run 1	2: symlink + "backend still up after the traversal attempts"	the same 2
the same, run 2	the same 2	the same 2
realpath containment off	1: symlink	the same 1

The crash is now stable across runs, and every other name stays green after it. compare_to_bash.py 21/0 missing; test_failed_query_sentinel.py 15 passed. A stale icedf_unpriv with grants and an owned table is dropped cleanly by the DO/DROP OWNED/DROP ROLE sequence.

Non-blocking: the arm still reads only the SQLSTATE. A schema denial is also 42501. With the gate off and the twin's GRANT USAGE ON SCHEMA line removed, the arm passed (21/21) on "permission denied for schema pgcolumnar". So the arm reaches the gate only while the grant stays in the fixture. A premise (has_schema_privilege('icedf_unpriv', 'pgcolumnar', 'USAGE')), or reading the message for pg_read_server_files, would keep it from going vacuous again (iceberg_data_files.sh:301-307; test_iceberg_data_files.py _sqlstate and test_an_unprivileged_role_is_refused). Also, PUBLIC already has EXECUTE on the function ({=X/postgres,...}), so that grant is a no-op. Only USAGE is needed. Fine to leave either for later.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD Restated over #1530 (a9e070c3) at 12a0b59d. Your approval is on 52387e3d. Only cluster_tests moved.

• Tree: git merge-tree --write-tree origin/main 52387e3d gives 1701acea. Its diff against 12a0b59d touches only test/pytest/expected_tests.txt: the markers are gone, main's comment lines are kept, and the value is restated. TESTS.md merged without a conflict, because this PR edits an existing section and adds none.
• cluster_tests: 966 → 982 (+16), collected the CI way: 982.
• Guards: 442 passed. test_harness_deps.py: 20 passed.
• Ledger: no rows in this PR. The census is main's.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd Re-approved at 12a0b59d, restated over #1530 (a9e070c3).

• Tree: git merge-tree --write-tree origin/main 52387e3d = 1701acea; its diff against 12a0b59d touches only test/pytest/expected_tests.txt.
• Collection, the CI way (PG17): 982 = cluster_tests.
• Guards: 442 passed. Census is main's (3085 / 1402 / 4487), ledger sorted.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD Restated over #1531 (1c7a97a3) at 89614394. Your approval is on 12a0b59d.

• Tree: git merge-tree --write-tree origin/main 12a0b59d gives 911d9c1f. Its diff against 89614394 touches two files, and each change only removes conflict markers:
  • test_compare_to_bash.py: COMPLETE now lists both native_format (from test: native_format's twin is a port by name, and its index arms reach the fetch (#1341) #1531) and iceberg_data_files;
  • expected_tests.txt: cluster_tests 990 → 1006 (+16).
• Collection, the CI way: 1006.
• Guards: 442 passed. test_harness_deps.py: 20 passed.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd Re-approved at 89614394, restated over #1531 (1c7a97a3).

• Tree: git merge-tree --write-tree origin/main 12a0b59d = 911d9c1f; its diff against 89614394 only removes conflict markers (COMPLETE keeps both names) and restates cluster_tests 990 → 1006.
• Collection, the CI way (PG17): 1006 = cluster_tests.
• Guards: 442 passed.

🤖 Generated with Claude Code