Skip to content

test: the Iceberg REST readers' server-file gate, and no borrowed environment token (#1341 items 8, 9, 14, 52) #1409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdatcmd merged 1 commit into from
Oct 15, 2026
Merged

test: the Iceberg REST readers' server-file gate, and no borrowed environment token (#1341 items 8, 9, 14, 52) #1409

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions test/check_ledger.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1889,6 +1889,35 @@ harness_selftest 640-every-suite-job-installs-the-extensions the detector names
harness_selftest 640-every-suite-job-installs-the-extensions the detector names a workflow in which it finds no suite job 15;16;17;18;19 2026-10-14 640's _ej_missing without its no-suite-job guard
harness_selftest 640-every-suite-job-installs-the-extensions the detector names a workflow it cannot read 15;16;17;18;19 2026-10-14 640's _ej_missing without its unreadable-file guard
harness_selftest 640-every-suite-job-installs-the-extensions the suite-running jobs are exactly the five this part knows 15;16;17;18;19 2026-10-14 640's job scan no longer counts a cluster job that runs pytest
iceberg_rest_server iceberg_rest_server a catalog_uri option on the USER MAPPING is rejected (HV00D) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a half OAuth2 credential (id, no secret) is refused before any request (28000) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a link-local oauth_token_uri is refused by the allow-list (42501) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a non-superuser with no token mapping is refused (28000) despite the environment token, before any request 15;16;17;18;19 2026-10-15 the REST resolver letting every role borrow the environment token
iceberg_rest_server iceberg_rest_server a server with an OAuth2 mapping reads by a minted bearer 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a token option on the SERVER is rejected (HV00D) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a warehouse option on the SERVER is accepted (not HV00D) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a warehouse-configured server resolves a table location 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a wrong OAuth2 client secret is refused (28000) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server a wrong mapping token is refused (28000) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server an oauth option on the SERVER is rejected (HV00D) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server an off-allow-list oauth_token_uri host is refused (42501) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server and it cannot mark its own mapping credentials_required false (42501) 15;16;17;18;19 2026-10-15 the catalog validator letting a non-superuser set credentials_required false
iceberg_rest_server iceberg_rest_server backend still up 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server control: with credentials_required false set by a superuser, the same role reads by the environment token 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server iceberg_rest_table_location('cat',...) reads via the mapping token 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server loadTable then carried the minted bearer (Authorization present) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server no mapping and no env token is refused, not bypassed (28000) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server oauth options are accepted on a USER MAPPING (no error) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server premise: rest fixture up 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server premise: the PG server log holds lines to search, for the client secret 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server premise: the PG server log holds lines to search, for the mapping token 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server the OAuth2 token endpoint was called (POST /v1/oauth/tokens) 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server the catalog-URI form still works with the env token 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server the client secret never appears in the PG server log 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server the client secret never appears in the fixture request log 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server the mapping token never appears in the PG server log 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server the request carried an Authorization header 15;16;17;18;19 never -
iceberg_rest_server iceberg_rest_server the warehouse was sent on GET /v1/config (?warehouse=wh1) 15;16;17;18;19 never -
import_export_privilege import_export_privilege export_arrow refuses SELECT of a table the caller cannot read 15;16;17;18;19 never -
import_export_privilege import_export_privilege export_parquet refuses SELECT of a table the caller cannot read 15;16;17;18;19 never -
import_export_privilege import_export_privilege get_storage_id refuses a caller who cannot SELECT the relation 15;16;17;18;19 never -
Expand Down Expand Up @@ -2496,6 +2525,64 @@ rollup rollup the rollup to be dumped folds its source 15;16;17;18;19 never -
rollup rollup the second refresh of the NULL-key rollup folded the append 15;16;17;18;19 2026-10-13 refresh_rollup folding every group, ignoring the recorded snapshot;refresh_rollup folding only the first new group;row_tid mapping 290 rows a block instead of the AM's
rollup rollup the zone rollups hold a row per key of the recompute 15;16;17;18;19 2026-10-13 refresh_rollup folding only the first new group;row_tid mapping 290 rows a block instead of the AM's
rollup rollup two refreshes started together fold the new rows once between them 15;16;17;18;19 2026-10-13 refresh_rollup folding every group, ignoring the recorded snapshot;refresh_rollup folding only the first new group;refresh_rollup without the self-conflicting lock;row_tid mapping 290 rows a block instead of the AM's
server_file_privilege server_file_privilege coverage check found the file functions (not an empty scan) 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: file_split_offsets 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_current_snapshot 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_data_files 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_fdw_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_rest_namespaces 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_rest_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_rest_table_location 15;16;17;18;19 2026-10-15 iceberg_rest_table_location without its pg_read_server_files gate
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_rest_tables 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: iceberg_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: import_arrow 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: import_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: parallel_copy 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: parquet_fdw_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: parquet_schema 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: read_avro_manifest 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: read_manifest_list 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_read_server_files gate: read_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_write_server_files gate: export_arrow 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_write_server_files gate: export_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_write_server_files gate: parallel_export_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege no role is refused by the pg_write_server_files gate: tier_partition 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: export_arrow 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: export_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: file_split_offsets 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: iceberg_current_snapshot 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: iceberg_data_files 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: iceberg_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: import_arrow 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: import_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: parallel_copy 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: parallel_export_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: parquet_schema 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: read_avro_manifest 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: read_manifest_list 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: read_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege server-file function is covered by the boundary test: tier_partition 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: file_split_offsets 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_current_snapshot 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_data_files 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_fdw_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_rest_namespaces 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_rest_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_rest_table_location 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_rest_tables 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: iceberg_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: import_arrow 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: import_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: parallel_copy 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: parquet_fdw_scan 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: parquet_schema 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: read_avro_manifest 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: read_manifest_list 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_read_server_files role gets past the gate: read_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_write_server_files role gets past the gate: export_arrow 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_write_server_files role gets past the gate: export_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_write_server_files role gets past the gate: parallel_export_parquet 15;16;17;18;19 never -
server_file_privilege server_file_privilege the pg_write_server_files role gets past the gate: tier_partition 15;16;17;18;19 never -
storage_row_repair storage_row_repair REFRESH MATERIALIZED VIEW repairs the view's row 15;16;17;18;19 2026-10-12 PgColumnarRetargetStorageRelation returns at once, so no repair repoints a storage row
storage_row_repair storage_row_repair a same-type ALTER COLUMN ... USING repairs the table's row 15;16;17;18;19 2026-10-12 PgColumnarRetargetStorageRelation returns at once, so no repair repoints a storage row
storage_row_repair storage_row_repair after both repairs the notes' query finds nothing 15;16;17;18;19 2026-10-12 PgColumnarRetargetStorageRelation returns at once, so no repair repoints a storage row;the release notes' query is edited from NOT EXISTS to EXISTS
Expand Down
18 changes: 16 additions & 2 deletions test/check_ledger_budget.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
#
# Adding a check to a suite that is already covered does not move it, which is
# what makes it safe to bound.
suites_not_covered 234
suites_not_covered 232
#
# checks_never_observed_red -- A CENSUS. NOT a ceiling, and it must not become
# one.
Expand Down Expand Up @@ -2282,4 +2282,18 @@ suites_not_covered 234
# never 2095 = 2095
# dated 688 + 3 = 691
# 2095 + 691 == 2786
checks_never_observed_red 2095
#
# #1341 items 8, 9, 14 and 52, the Iceberg REST privilege gates (jdatcmd): 2095 ->
# 2179, and suites_not_covered 234 -> 232. server_file_privilege and
# iceberg_rest_server had no rows; they enter with all their checks from five-major
# runs (58/58 and 29/29 on each of 15-19). Three are dated: the REST read gate by
# iceberg_rest_table_location without it, the environment-token refusal by the
# resolver letting every role borrow the token, and the credentials_required arm by
# the validator letting a non-superuser set it false. The other 84 arrive never:
# seeding a suite moves the census up by its undated rows.
#
# ledger rows 2786 + 87 = 2873
# never 2095 + 84 = 2179
# dated 691 + 3 = 694
# 2179 + 694 == 2873
checks_never_observed_red 2179
28 changes: 28 additions & 0 deletions test/iceberg_rest_server.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,5 +227,33 @@ check "the catalog-URI form still works with the env token" \
"$(q "SELECT pgcolumnar.iceberg_rest_table_location('$CAT','db','events')")" \
"$MDLOC"

# ---- a non-superuser does not borrow the server's environment token (#1341 items 9, 14)
# The environment token is the postmaster's ambient identity: only a superuser, or a
# mapping a superuser marked credentials_required 'false', may use it. ice_r holds
# every other grant the call needs, so the missing token mapping is all that refuses
# it, and the refusal comes before any request reaches the catalog.
q "CREATE ROLE ice_r LOGIN NOSUPERUSER; GRANT pg_read_server_files TO ice_r;
GRANT USAGE ON SCHEMA pgcolumnar TO ice_r; GRANT USAGE ON FOREIGN SERVER cat TO ice_r;" >/dev/null
as_ice_r() { # as_ice_r SQL -> the single value, or the SQLSTATE it fails with, or noconnect
local out rc st
out="$(env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U ice_r -d "$PGC_DB" \
-X -qtA -v ON_ERROR_STOP=1 -v VERBOSITY=sqlstate -c "$1" 2>&1)"
rc=$?
st="$(sed -n 's/^ERROR: *\([0-9A-Z]\{5\}\).*/\1/p' <<<"$out" | head -1)"
if [ -n "$st" ]; then echo "$st"
elif [ "$rc" = 0 ]; then v="$(printf '%s\n' "$out" | tail -1)"; echo "${v:-ok}"
else echo noconnect; fi
}
before="$(wc -l < "$REST_LOG")"
check "a non-superuser with no token mapping is refused (28000) despite the environment token, before any request" \
"$(as_ice_r "SELECT pgcolumnar.iceberg_rest_table_location('cat','db','events')")|$(( $(wc -l < "$REST_LOG") - before ))" \
"28000|0"
q "CREATE USER MAPPING FOR ice_r SERVER cat" >/dev/null
check "and it cannot mark its own mapping credentials_required false (42501)" \
"$(as_ice_r "ALTER USER MAPPING FOR ice_r SERVER cat OPTIONS (ADD credentials_required 'false')")" "42501"
q "ALTER USER MAPPING FOR ice_r SERVER cat OPTIONS (ADD credentials_required 'false')" >/dev/null
check "control: with credentials_required false set by a superuser, the same role reads by the environment token" \
"$(as_ice_r "SELECT pgcolumnar.iceberg_rest_table_location('cat','db','events')")" "$MDLOC"

check "backend still up" "$(q 'SELECT 1')" "1"
pgc_summary
11 changes: 11 additions & 0 deletions test/server_file_privilege.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ psql_run "GRANT SELECT, INSERT ON imp_target TO t_priv;"
psql_run "CREATE SERVER pqsrv FOREIGN DATA WRAPPER pgcolumnar_parquet;"
psql_run "CREATE FOREIGN TABLE ft (id int, v text) SERVER pqsrv OPTIONS (path '$nope');"
psql_run "GRANT SELECT ON ft TO t_none, t_priv;"
psql_run "CREATE SERVER icesrv FOREIGN DATA WRAPPER pgcolumnar_iceberg;"
psql_run "CREATE FOREIGN TABLE ift (id bigint) SERVER icesrv OPTIONS (metadata_path '$nope');"
psql_run "GRANT SELECT ON ift TO t_none, t_priv;"

# tier_partition (#1295) writes its file through export_parquet, so its gate is
# export_parquet's. To reach it a role needs SELECT and, for the SHARE lock the
Expand Down Expand Up @@ -100,6 +103,14 @@ entry_points=(
"export_arrow|write|SELECT pgcolumnar.export_arrow('imp_target'::regclass, '$out.arrow')"
"parallel_export_parquet|write|SELECT pgcolumnar.parallel_export_parquet('imp_target'::regclass, '$outdir', 2)"
"tier_partition|write|SELECT pgcolumnar.tier_partition('sfp_p1'::regclass, 'pqsrv', '$out.parquet')"
# The REST catalog readers (#388 phase 7) and the Iceberg wrapper's scan carry the
# same read gate (#1341 items 8 and 52). A URL nothing listens on: the privileged
# role gets past the gate and fails on the connection, without naming the role.
"iceberg_rest_table_location|read|SELECT pgcolumnar.iceberg_rest_table_location('http://127.0.0.1:1', 'db', 't')"
"iceberg_rest_scan|read|SELECT * FROM pgcolumnar.iceberg_rest_scan('http://127.0.0.1:1', 'db', 't') AS t(id bigint)"
"iceberg_rest_namespaces|read|SELECT * FROM pgcolumnar.iceberg_rest_namespaces('http://127.0.0.1:1')"
"iceberg_rest_tables|read|SELECT * FROM pgcolumnar.iceberg_rest_tables('http://127.0.0.1:1', 'db')"
"iceberg_fdw_scan|read|SELECT * FROM ift"
)

declare -A listed
Expand Down
Loading