Repository navigation
feat(sync): archive Codex's compressed archived rollouts - #190
Open
HengYangDS wants to merge 1 commit into
Open
HengYangDS wants to merge 1 commit into
HengYangDS wants to merge 1 commit into
Conversation
Codex moves a finished rollout from sessions/YYYY/MM/DD/ into a flat archived_sessions/ directory and compresses it to .jsonl.zst. Sync only walked sessions/ for .jsonl files, so archived Codex conversations never reached the archive once they left sessions/. Sync now also reads archived_sessions/, maps a flat rollout back to its dated archive path, and decompresses .jsonl.zst sources into the plain transcript they hold, using node:zlib's zstd support (Node 22.15 and later; compressed files are skipped where it is unavailable). The content check streams the transcript instead of reading it whole. Codex has also rewritten older rollouts into a new record format before compressing them. Index rows address archive lines, so a compressed source replaces an existing archive only when its content extends that archive; otherwise the archive is kept and stamped with the source time so the unchanged rewrite is not decompressed again. Generated with Claude Code
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Codex moves finished rollouts into a flat
archived_sessions/directory and compresses them as.jsonl.zst. Sync only scanssessions/for.jsonl, so it misses those archives. In the reported workstation sample, 6,017 of 6,022 archived rollouts were compressed.This scans both source directories, maps a flat archive back to its dated path, and streams decompression and the message-content check. It uses Node's zstd support (22.15 or later); unsupported runtimes skip compressed inputs.
A compressed source replaces an indexed archive only when it extends that archive. A rewritten source could otherwise move the lines addressed by existing rows. Non-extending rewrites keep the existing archive and receive the source timestamp, avoiding repeated decompression. Plain sources retain their current behavior.
Validation
Head
b166ac3b, based on main7e065193. Six new tests cover paths, compressed archived/live sources, continuation and rewrite preservation. TypeScript passes. The full local run has 313 passed and 43 failed; unchanged main has 307 passed and the same 43 embedding-dependent timeouts. The full suite is not green.The emitted paths/sync artifacts are included. The unrelated MCP bundle remains unchanged because rebuilding it in that environment adds dependency churn. Current CI needs approval. Review, release and installed use remain pending; #193 owns broader rewrite reconciliation.