Skip to content

fix(io): initialize manifest caches only when used - #3679

Merged
CTTY merged 1 commit into
apache:mainfrom
alexanderbianchi:codex/lazy-object-cache
Nov 16, 2026
Merged

CTTY merged 1 commit into
apache:mainfrom
alexanderbianchi:codex/lazy-object-cache

Conversation

@alexanderbianchi

@alexanderbianchi alexanderbianchi commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

What changes are included in this PR?

Loading a table constructs a Moka manifest cache even when the caller only needs table metadata. Dropping that unused cache performs Moka's epoch-reclamation work, which can also run deferred cleanup from other caches on the calling thread.

Initialize the cache on the first manifest or manifest-list read, and never construct it when caching is disabled. Clones share the same lazy initialization and cached objects. Cache keys, capacity, weighting, encryption, and runtime scheduling retain their existing behavior.

Are these changes tested?

  • cargo test -p iceberg --lib --all-features --locked: 2,154 passed, 1 existing ignored (including all 7 object-cache tests).
  • cargo clippy -p iceberg --all-targets --all-features --locked -- -D warnings: passed.
  • cargo fmt --all -- --check: passed.
  • Regenerated the Iceberg public API with make generate-public-api scoped to the core crate: no changes.

Regression coverage checks that constructing, cloning, and dropping an unused cache does not initialize Moka; first use shares one cache with the configured capacity; clones return the same cached manifest/list objects; and disabled-cache reads never initialize Moka.

Defer Moka cache construction until the first manifest or manifest-list
read, and skip it entirely when caching is disabled. Share lazy cache
state across clones and retain existing cache policy and encryption.

Adapted from DataDog#34 for upstream main, without the
fork-specific FileIO rebinding helper.

Co-authored-by: Geoffrey Claude <geoffrey.claude@datadoghq.com>
@alexanderbianchi
alexanderbianchi marked this pull request as ready for review November 12, 2026 15:46
@alexanderbianchi
alexanderbianchi marked this pull request as draft November 12, 2026 22:22
@alexanderbianchi
alexanderbianchi marked this pull request as ready for review November 16, 2026 17:03

@CTTY CTTY left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the improvement!

Just curious, was this motivated by an use case where the eager cache initialization caused noticeable performance overhead?

@CTTY
CTTY added this pull request to the merge queue Nov 16, 2026
Merged via the queue into apache:main with commit c9d5217 Nov 16, 2026
27 checks passed
@alexanderbianchi

alexanderbianchi commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Just curious, was this motivated by an use case where the eager cache initialization caused noticeable performance overhead?

Yes! Our Table instances are request-scoped; we don’t retain them across requests. Each request has its own observability, scheduling priority, and authorization context.

In particular, Iceberg-spawned tasks don’t propagate our observability context, so our integration carries it through a request-bound FileIO. The catalog’s storage factory is configured at catalog scope, without a per-load override. With a shared catalog, we currently reconstruct the returned Table to attach that FileIO and the query’s runtime.

This means the table-owned manifest cache also has a request lifetime: its construction and teardown costs recur rather than being amortized across requests. It may still help with repeated scans within one request.

We also expose a schema-only endpoint for the query service. Those requests construct Tables but never read manifests, so they received no benefit from the eagerly constructed cache. Nevertheless, dropping it performed Moka’s synchronous reclamation work, potentially executing deferred cleanup from other caches.

Lazy initialization removes that hidden cost for metadata-only requests. Separately, easier request-context propagation and per-load storage binding would simplify this integration.

Admittedly our use is maybe a bit out the ordinary. For us, DataFusion + iceberg-rust fit into a broader query stack with it's own parser, schema service, various data stores + engines, and layers of access controls + fairness mechanisms. Query-scoped execution bindings and reusable table metadata have different lifetimes in our system. Bundling metadata, FileIO, runtime, and cache ownership into one Table makes that separation awkward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants