Skip to content

fix: load each table from the catalog on lookup - #116

Draft
NoahKusaba wants to merge 1 commit into
apache:mainfrom
NoahKusaba:fix/fresh-table-lookup
Draft

NoahKusaba wants to merge 1 commit into
apache:mainfrom
NoahKusaba:fix/fresh-table-lookup

Conversation

@NoahKusaba

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

IcebergSchemaProvider builds each table's provider once, when the catalog is registered, and every query reuses it with the schema the table had then. After another client changes the schema, INSERTs fail (#56) and reads return wrong results or panic (#57).

What changes are included in this PR?

  • IcebergSchemaProvider keeps only table names. Each table() lookup loads the table and builds a new provider, including for $ metadata tables.
  • Registering the catalog no longer loads every table.
  • Removes the now-unused IcebergCatalogTableProvider::metadata_table.

Table names are still listed once, so tables other clients create or drop stay invisible (rest of #24).

Are there any user-facing changes?

Queries see schema changes made by other clients. Each lookup costs one load_table, on top of the one scan/insert_into already does; a follow-up can have the provider reuse the loaded table. No API changes.

Are these changes tested?

Yes. tests/table_lookup.rs has a session use a table, another client change its schema, and the session use it again. Both tests fail on main with the errors from the issues.

@NoahKusaba
NoahKusaba marked this pull request as ready for review November 18, 2026 14:53
@NoahKusaba

Copy link
Copy Markdown
Contributor Author

Line count is inflated by tests.

@NoahKusaba
NoahKusaba marked this pull request as draft November 19, 2026 18:51
IcebergSchemaProvider built every table's provider when the catalog was
registered and returned it to every query, so a session kept planning
against a table's old schema after another client changed it. Keep only
the table names and load the table on each lookup.

Closes apache#56, apache#57.
@NoahKusaba
NoahKusaba force-pushed the fix/fresh-table-lookup branch from 60c87e8 to fc85b58 Compare November 19, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant