feat(api): revalidate the collection cache against disk on read - #90
Merged
Merged
Conversation
The app serves a collection from an in-memory tree it patches as the UI mutates resources. CLI commands write straight to disk, so the cache went stale and a browser refresh did not help — it re-read the same cache. Only a restart did. Detect the change on read instead of being told about it. Before serving the tree, cache status or a search, the API takes a stat-only fingerprint of the translations folder (file and folder counts, total size, newest mtime) and compares it with the one recorded at index time. A mismatch drops the cache, so the request answers 202 and the next one returns fresh data. Watching the folder was the obvious alternative and is not portable: inotify never fires for Windows-side writes on a WSL /mnt/c mount, and the same holds for several network and container mounts. A CLI-side HTTP invalidate is not portable either — localhost is not shared across the WSL boundary — and would miss every change that does not come from the CLI, such as a git checkout or a hand edit. The scan is throttled to once every 2000 ms (LINGO_TRACKER_REVALIDATE_INTERVAL_MS), and the API's own writes re-take the fingerprint on the next tick, coalesced, so bulk endpoints do not trigger a redundant re-index. Closes #86 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RifJr7xyhZkaTWn4ssV1b4
|
🎉 This PR is included in version 0.18.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #86
Problem
The app serves a collection from an in-memory tree that it patches as the UI mutates
resources. CLI commands write straight to
resource_entries.json/tracker_meta.jsonandnever tell a running app, so the cache goes stale — and a browser refresh does not fix it,
because the page re-reads the same server-side cache. Only a restart does. Users reasonably
read that as data loss.
Why not the two approaches in the issue
Both have a WSL failure mode, and both miss changes that do not come from the CLI.
POST /invalidatelocalhostis not shared. WSL2→Windows host needs the gateway IP from/etc/resolv.conf, or Win11 22H2+ mirrored networking. Silent no-op exactly when it matters.fs.watch/ chokidar in the API/mnt/c(drvfs/9p) never raise inotify events in WSL — microsoft/WSL#4739. The watcher looks healthy and fires nothing. Same on several network and container mounts.Neither sees a
git checkout, a branch switch, or a hand edit.What this does instead: revalidate on read
Before serving the tree, the cache status, or a search, the API takes a stat-only
fingerprint of the collection's translations folder and compares it with the one recorded at
index time. A mismatch drops the cache, so the request answers
202 Acceptedand the nextone returns fresh data. The existing not-ready → async reindex → UI retry protocol does the
rest, so the Tracker UI needed no change.
computeTreeFingerprint()(libs/core/src/lib/resource/tree-fingerprint.ts) walks thefolder with
readdir+statonly, returning{fileCount, folderCount, totalSize, maxMtimeMs}. Counts and size sit alongside mtime because mtime granularity is as coarseas one to two seconds on drvfs and FAT, so a same-second edit can leave mtime untouched.
CollectionCacheService.revalidate()is throttled to 2000 ms, configurable viaLINGO_TRACKER_REVALIDATE_INTERVAL_MS.per-resource cache patches cost one scan rather than one per resource — and the app never
reads its own write as somebody else's.
No new dependency, no watcher, no port resolution, no daemon. Behaves identically on macOS,
Linux, Windows, WSL, containers and network shares.
Verified end to end
Built API run against a scratch project on port 3931:
add-resourcevia the CLIPOST /resourcesvia the APIresource_entries.jsonSuites green: core 1227, api 200, tracker 493, plus cli and domain.
pnpm lintandpnpm typecheckclean.