Skip to content

fix(appkit): honor TTL in CacheManager.getOrExecute#326

Merged
MarioCadenas merged 6 commits into
databricks:mainfrom
Tim-Hoare:fix/cache-getorexecute-ttl
May 13, 2026
Merged

fix(appkit): honor TTL in CacheManager.getOrExecute#326
MarioCadenas merged 6 commits into
databricks:mainfrom
Tim-Hoare:fix/cache-getorexecute-ttl

Conversation

@Tim-Hoare
Copy link
Copy Markdown
Contributor

Fixes #325.

Problem

CacheManager.getOrExecute returns cached entries without checking expiry, so when InMemoryStorage is the backend (the default when Lakebase isn't configured) cached values are served indefinitely — until LRU eviction at maxSize or process restart. The configured ttl is silently ignored on this code path.

This affects every AppKit app using the analytics plugin's default cache (enabled: true, ttl: 3600) without Lakebase. In a deployed Databricks App we observed query results staying cached for >19 hours despite a 1-hour TTL.

Why it slipped through

InMemoryStorage.get() is intentionally a raw lookup — the existing test should return expired entry on get (expiry check is done by CacheManager) makes that contract explicit. The standalone CacheManager.get() honors it correctly. But CacheManager.getOrExecute didn't, and the existing TTL test ("should respect TTL expiry") only covered the set / get pair, not getOrExecute.

Fix

In getOrExecute, after storage.get(cacheKey) returns an entry, check Date.now() > cached.expiry. If expired: delete it and fall through to the normal cache-miss path so fn() re-executes. Otherwise return the cached value as before.

Net change is small and stays within the existing "expiry handling lives at the CacheManager layer" contract — no signature changes, no storage changes.

Test

Added a regression test in the describe("getOrExecute", …) block: caches a value with a 1ms TTL, waits, calls getOrExecute again, and asserts that fn was invoked twice and returns the new value.

Confirmed:

  • Without the fix: new test fails (expected 'result-1' to be 'result-2').
  • With the fix: new test passes; all 1287 existing appkit package tests continue to pass.

Lint (biome check) and typecheck (tsc --noEmit) clean.

Notes

  • I picked the CacheManager-level fix to preserve the existing contract documented in the storage tests. Happy to refactor (e.g. push expiry into storage implementations, or factor a getValid() helper used by both get and getOrExecute) if a different shape is preferred.
  • DCO sign-off included.

Tim-Hoare and others added 2 commits April 29, 2026 12:27
CacheManager.getOrExecute calls storage.get() and returns the cached
value without checking expiry. InMemoryStorage.get() returns entries
unconditionally by design ("expiry check is done by CacheManager"),
but the check was missing on the getOrExecute path — only the
standalone CacheManager.get() honored TTL.

With the in-memory backend (the default when Lakebase is not
configured), this caused cached entries to live until LRU eviction at
maxSize or process restart, regardless of the configured ttl. Apps
using the analytics plugin's default cache (enabled, ttl: 3600) would
serve stale query results indefinitely.

Add an explicit expiry check in getOrExecute: if storage returns an
expired entry, delete it and fall through to a normal cache miss so
fn() re-executes.

Adds a regression test in the getOrExecute describe block. The
existing TTL test only covered the set/get pair, which is why this
was not caught.

Fixes databricks#325

Signed-off-by: Tim Hoare <t.hoare@codat.io>
@MarioCadenas MarioCadenas requested a review from a team as a code owner May 6, 2026 10:59
@MarioCadenas MarioCadenas requested review from MarioCadenas and calvarjorge and removed request for a team May 6, 2026 10:59
Copy link
Copy Markdown
Collaborator

@MarioCadenas MarioCadenas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @Tim-Hoare, first of all thanks so much for reporting the bug and fixing it, sorry it took a bit to review 😅

LGTM on the fix — the bug is real and the change correctly aligns getOrExecute with the same expiry contract that get() and has() already use.

Suggestion: extract a getValid() helper

You called this out in the PR description, and I think it's worth doing — the Date.now() > entry.expiry + storage.delete + treat-as-miss pattern now lives in three places (get, has, and the new branch in getOrExecute). If a fourth caller appears, this becomes a bug magnet (exactly how this PR happened).

A small private helper collapses it:

private async getValid<T>(
  key: string,
): Promise<{ value: T; expiry: number } | null> {
  const entry = await this.storage.get<T>(key);
  if (!entry) return null;
  if (Date.now() > entry.expiry) {
    await this.storage.delete(key);
    return null;
  }
  return entry;
}

As I said, up to you if you want to refactor it and implement the getValid method 😄 , happy to merge as is

MarioCadenas and others added 3 commits May 12, 2026 19:30
Address review feedback on databricks#326: collapse the duplicated
'storage.get + expiry check + delete-on-expired' pattern in get(),
has(), and getOrExecute() into a single private getValid() helper to
avoid future drift.

Signed-off-by: TimHoare <t.hoare@codat.io>
@Tim-Hoare
Copy link
Copy Markdown
Contributor Author

Thanks @MarioCadenas! I have added the getValid helper if you would like to re-review

@MarioCadenas MarioCadenas enabled auto-merge (squash) May 13, 2026 18:46
@MarioCadenas
Copy link
Copy Markdown
Collaborator

Thanks a lot @Tim-Hoare ! approving and merging 😄

@MarioCadenas MarioCadenas disabled auto-merge May 13, 2026 18:49
@MarioCadenas MarioCadenas merged commit 0aedd1b into databricks:main May 13, 2026
3 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cache TTL not honored in getOrExecute with in-memory storage (results never expire)

2 participants