Skip to content

fix: parse Claude session logs incrementally instead of re-reading whole files - #338

Open
notcadenwhitt-stack wants to merge 1 commit into
chattymin:mainfrom
notcadenwhitt-stack:fix/incremental-claude-log-parsing
Open

notcadenwhitt-stack wants to merge 1 commit into
chattymin:mainfrom
notcadenwhitt-stack:fix/incremental-claude-log-parsing

Conversation

@notcadenwhitt-stack

Copy link
Copy Markdown

Summary

Fixes #337.

On an account with a large Claude Code history (~2GB of session logs this month, single files up to 274MB) the cold scan pegged a core for over a minute without finishing, and because Blob is keyed on whole-file mtime+size, the active session file was re-parsed from byte zero on every refresh.

Three changes, all in the Claude local-log path:

  • ISO8601Parser reuses one lock-guarded ISO8601DateFormatter pair instead of allocating per call. This is the per-line timestamp parser for every session log, and was the dominant cost in sample. The lock is needed because the provider caches call it concurrently.
  • LocalUsageReader.parseClaudeFile scans lines at the byte level via Data/FileHandle instead of String(contentsOf:) + split, so a 274MB file is never materialized as a Swift String. A new parseClaudeFile(_:fromOffset:fmt:) reads from a byte offset and reports how far it consumed (stopping at the last complete line, so a half-written record is retried next refresh).
  • LocalUsageCache.Blob gains an optional offset. In collect, when a cached file has strictly grown, only the appended tail is parsed and merged (dedup by id). Same-size rewrites and truncations still take the full-reparse path, so the existing "mtime changed → reparse" contract holds. Older cache files without the key decode fine (decodeIfPresent) and fall back to a full parse once.

Only the Claude reader opts into the incremental path; every other collect caller is unchanged.

Measured on the history above, release build:

Before After
Cold start 60s+ at ~100% CPU, never completed, Claude cache never written ~15s, then idle; cache persists
Peak memory 900MB+ and climbing ~75MB
Warm relaunch / steady-state refresh (active session growing) full re-parse every refresh 0.0% CPU

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Documentation
  • Other:

UI changes

None.

Checklist

  • swift build and swift test pass locally (1165/1166; the one failure, SessionKeySettingsRenderingTests, fails identically on unmodified main in my environment and is unrelated to this change)
  • PR title and description are written in English
  • UI changes are described above (before/after — images optional)
  • No copyrighted assets, secrets, or private tooling references are committed
  • Tests were added or updated for this change — four new cases in LocalUsageCacheTests: tail-only parse on append (proven by rewriting the prefix in place and asserting it is not re-read), full reparse on truncation, half-written trailing line retried on the next refresh, and pre-offset cache files still decoding

🤖 Generated with Claude Code

…ole files

On an account with a large Claude Code history (~2GB of session logs in
the current month, single files up to 274MB) the cold scan pegged a core
for over a minute without finishing, and because the blob cache is keyed
on whole-file mtime+size, the active session file was re-parsed from byte
zero on every refresh.

Three changes:
- ISO8601Parser reuses one lock-guarded formatter pair instead of
  allocating an ISO8601DateFormatter per line (the dominant sampled cost).
- parseClaudeFile scans lines at the byte level via Data/FileHandle rather
  than materializing the file as a String and splitting into Substrings.
- Blob records a per-file byte offset; when a file has only grown, only
  the appended tail is parsed and merged. Same-size or shrunk files are
  re-read from the start. Older cache files without the key still load.

Measured on the history above: cold start ~15s then idle (was 60s+ and
never completed), steady-state refresh stays at 0% CPU.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude log scan pegs CPU indefinitely on large histories; active session file re-parsed every refresh

1 participant