Skip to content

Persist the Iceberg catalog between runs #6

Closed

Description

@codecapo

The catalog is in memory today (Warehouse::open_local uses MemoryCatalog), so tables are forgotten when infoctl exits. Incremental runs need the catalog to survive between runs.

Crate: crates/storage

Approach: a SQLite file in the project (for example .infostash/catalog.db). Check first whether the pinned iceberg-rust commit has a SQL catalog we can use; if not, store namespace, table name and current metadata location ourselves.

Done when

  • Warehouse::open_local reopens tables created by an earlier process
  • Commits from two runs in a row both show up in the snapshot log
  • A test creates, commits, drops the warehouse, reopens it and reads the rows back
  • Any new dependency is justified in the PR

Blocks: #9, #10, #11, #15

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions