Skip to content

fix: the Iceberg wrapper reads no server file while planning for a role the scan would refuse (#1366) - #1367

Merged
jdatcmd merged 2 commits into
mainfrom
fix/fdw-plan-privilege
Oct 14, 2026
Merged

jdatcmd merged 2 commits into
mainfrom
fix/fdw-plan-privilege

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator

Fixes #1366.

Change

icefdwGetForeignRelSize now reads metadata_path only when the caller has pg_read_server_files, the same bar icefdwBeginForeignScan enforces. Everyone else gets the flat estimate (1000), as the Parquet wrapper has since #330.

Red first (PG 17, both twins, against main's code)

  • "an unprivileged planner gets the flat estimate, not the Iceberg table's row count": got [5] want [1000]
  • "and a metadata_path that does not exist plans the same, so planning probes nothing": got [58P01] want [1000]

The Parquet gate is pinned too: with its has_privs_of_role removed, "an unprivileged planner gets the flat Parquet estimate" fails with got [22582] want [1000]. That gate had no test.

Controls: a privileged planner still gets the manifest count, and the Parquet file's size-derived estimate. The unprivileged scan is still refused (42501).

A fixture trap, for the record

My first Parquet control read flat (1000) even for the superuser. The export had failed (export_parquet needs a columnar source), so there was no file to size. A premise now asserts the file was written.

Evidence

  • fdw_plan_privilege: 8/8 on each of PG 15–19, and its pytest twin passes on each.
  • Related suites on PG 17: iceberg_fdw 74/74, iceberg_fdw_estimate 4/4, iceberg_scan 7/7, server_file_privilege 48/48.
  • Ledger: 8 rows, 3 dated by the two mutations, 5 never; rows 2343 → 2351, census 1796 → 1801. Gate rc=0, ceiling 246.
  • Counts: cluster_tests 533 → 534. The database-free set gives 429 passed.
  • Registration: registered in the suite list, declared COMPLETE, documented in TESTS.md section 105, with a CHANGELOG entry.

Overlap

#1364 also adds TESTS.md section 105, and both move cluster_tests and the budget, so whichever lands second renumbers and restates.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving 5e1c07c. Thanks for taking this one.

I ran this PR's tests on main's product code, using a worktree of this head with src/ from main 1933577, then on this head:

fdw_plan_privilege.sh test_fdw_plan_privilege.py
main's code, PG17 2 failed of 8: the unprivileged planner gets "[5] want [1000]", and a missing path gets "[58P01]" 1 failed
this head, PG 15–19 8/8 on each passed on each

CI is 15/15.

The fix is the #330 gate, applied before the read, with the scan's own gate untouched. It reads metadata_path from the catalog first, which touches no file.

One nit, not blocking: the new block sits under the existing "Estimate from the manifests the scan reads anyway…" comment. That comment now describes the code two blocks down. Moving the new comment and gate above it would keep each comment next to its code.

@jdatcmd

jdatcmd commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, rebased onto df6a02f. The new head is 4f4d48c, replacing 5e1c07c, which you approved.

  • Patch: identical to 5e1c07c outside the count files, except the TESTS.md section number. Main's §105 is now import_export_privilege, so this one is §106.
  • Counts: cluster_tests 541 → 542 by collection. The ledger is restated on 2466 / 1895 / 571 to give 2474 / 1900 / 574.
  • Gate: clean, with 284 registered and the ceiling at 244. The guard set gives 429 passed.

Several open PRs claim §106 (this one, #1364, #1385, #1395), and cluster_tests 542 (#1396 as well). Whichever merges later renumbers and recounts. Please re-approve 4f4d48c once CI is green.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 4f4d48c, with full CI green (15/15): the patch is line-for-line identical to 5e1c07c outside the count files. Main has moved again (#1385 and #1369 merged), so it will need another budget restatement before it merges.

🤖 Generated with Claude Code

jdatcmd and others added 2 commits October 14, 2026 19:29
…le the scan would refuse

Reported to jdatcmd by OffgridwithJD at the owner's request (#1283). The
Iceberg FDW's GetForeignRelSize read metadata_path for any role holding SELECT
on the foreign table; the pg_read_server_files gate ran only at scan start.

Red first, PG 17, both twins, on main's code:
  an unprivileged planner gets the flat estimate, not the Iceberg table's
  row count: got [5] want [1000]
  and a metadata_path that does not exist plans the same, so planning probes
  nothing: got [58P01] want [1000]

So EXPLAIN told a role without pg_read_server_files a server-side table's row
count, and whether a server path exists. The planner now reads the metadata
only for a role the scan would admit, and gives everyone else the flat
estimate, as the Parquet wrapper has since #330.

That Parquet gate had no test either. The new suite pins it too: with its
has_privs_of_role removed, "an unprivileged planner gets the flat Parquet
estimate" reports got [22582] want [1000]. The controls show a privileged
planner still gets the manifest count and the size estimate.

Clean on PG 17: fdw_plan_privilege 8/8 and its pytest twin; iceberg_fdw 74/74,
iceberg_fdw_estimate 4/4, iceberg_scan 7/7, server_file_privilege 48/48.
New suite registered, declared COMPLETE, TESTS.md section 105, cluster_tests
533 -> 534.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and two mutations

8 rows, 3 dated, 5 never. Census 1796 -> 1801, rows 2343 -> 2351. The gate
over the five green logs agrees, and the coverage ceiling stays at 246.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd

jdatcmd commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, thanks for approving 4f4d48c. Main moved (#1385, #1369, #1396), so I rebased again. The new head is 91e2a97, on dc6b060.

Please re-approve this head. I'm holding #1397's rebase until this one is in.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd force-pushed the fix/fdw-plan-privilege branch from 4f4d48c to 91e2a97 Compare October 14, 2026 19:30

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 91e2a97, with full CI green: the patch is line-for-line identical to 5e1c07c outside the count files. TESTS.md only adds lines (now §107).

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Iceberg FDW: planning reads metadata_path for a role the scan would refuse (EXPLAIN discloses row counts and probes paths)

2 participants