fix: parse Claude session logs incrementally instead of re-reading whole files - #338
Open
notcadenwhitt-stack wants to merge 1 commit into
Open
notcadenwhitt-stack wants to merge 1 commit into
notcadenwhitt-stack wants to merge 1 commit into
Conversation
…ole files On an account with a large Claude Code history (~2GB of session logs in the current month, single files up to 274MB) the cold scan pegged a core for over a minute without finishing, and because the blob cache is keyed on whole-file mtime+size, the active session file was re-parsed from byte zero on every refresh. Three changes: - ISO8601Parser reuses one lock-guarded formatter pair instead of allocating an ISO8601DateFormatter per line (the dominant sampled cost). - parseClaudeFile scans lines at the byte level via Data/FileHandle rather than materializing the file as a String and splitting into Substrings. - Blob records a per-file byte offset; when a file has only grown, only the appended tail is parsed and merged. Same-size or shrunk files are re-read from the start. Older cache files without the key still load. Measured on the history above: cold start ~15s then idle (was 60s+ and never completed), steady-state refresh stays at 0% CPU. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #337.
On an account with a large Claude Code history (~2GB of session logs this month, single files up to 274MB) the cold scan pegged a core for over a minute without finishing, and because
Blobis keyed on whole-file mtime+size, the active session file was re-parsed from byte zero on every refresh.Three changes, all in the Claude local-log path:
ISO8601Parserreuses one lock-guardedISO8601DateFormatterpair instead of allocating per call. This is the per-line timestamp parser for every session log, and was the dominant cost insample. The lock is needed because the provider caches call it concurrently.LocalUsageReader.parseClaudeFilescans lines at the byte level viaData/FileHandleinstead ofString(contentsOf:)+split, so a 274MB file is never materialized as a Swift String. A newparseClaudeFile(_:fromOffset:fmt:)reads from a byte offset and reports how far it consumed (stopping at the last complete line, so a half-written record is retried next refresh).LocalUsageCache.Blobgains an optionaloffset. Incollect, when a cached file has strictly grown, only the appended tail is parsed and merged (dedup by id). Same-size rewrites and truncations still take the full-reparse path, so the existing "mtime changed → reparse" contract holds. Older cache files without the key decode fine (decodeIfPresent) and fall back to a full parse once.Only the Claude reader opts into the incremental path; every other
collectcaller is unchanged.Measured on the history above, release build:
Type of change
UI changes
None.
Checklist
swift buildandswift testpass locally (1165/1166; the one failure,SessionKeySettingsRenderingTests, fails identically on unmodifiedmainin my environment and is unrelated to this change)LocalUsageCacheTests: tail-only parse on append (proven by rewriting the prefix in place and asserting it is not re-read), full reparse on truncation, half-written trailing line retried on the next refresh, and pre-offsetcache files still decoding🤖 Generated with Claude Code