Repository navigation
feat(sync): prune seen txns below a per-origin local_seq floor, refuse loudly below it (T13318) - #1981
Merged
Conversation
…e loudly below it (T13318) Proof first (code reading on the stack top, tests below): an origin's transactions are FIRST delivered in local_seq order on every path. The sealer mints local_seq rising (never reused, T13033); buildSegment packs the lowest sealed txns of its own replica in order and stops at a partial group; push uploads lowest replicaSeq first, resending the same bytes; the server and every puller refuse a replicaSeq gap or replay. A raced or crashed genesis cut returns folded txns to sealed only while genesis_pending blocks push. Rebind and copy never re-send inherited txn ids (spec H3); revival replays from the local inbox and a re-emit is a new txn. A retired replica's late segment is still contiguous in its own replicaSeq. - _sync_seen_floor (new local-only journal table): per stream and origin the highest local_seq staged, pruned_upto, and the seen row/byte counts; the migration seeds it from rows already staged. - A txn at or below its origin's floor with no seen row refuses its segment (refusedKind 'below-floor', naming pruned vs out of order); the pull stops before it and never skips it as seen. - A segment carrying a txn that is not its replica's is refused. - pruneSeenTxns drops each origin's rows below its floor (keeping the floor's own row); the cloud pull prunes after every pull. pullStream's unused stream-seq pruneSeenUpTo is replaced by pruneSeen. - seenTxnReport sums the per-origin counts: O(origins), not O(rows) (review-hotfix LOW on #1972). - Vault bundles clear _sync_seen_floor with _sync_seen_txn and the cursor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjjTKbQXyCFdkLBSeuyKLt
…s the ledger meanwhile (T13318, #1981 LOW) A store whose journal records the floor folder but whose _sync_seen_floor table is gone got nothing from ensureSyncSchema (the folder reads as applied), so pull threw "no such table" and cloud status read 0. The pull now heals the table (healSyncSchema re-runs the applied folders, which re-seeds the counts from the seen rows), and seenTxnReport walks the ledger until then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjjTKbQXyCFdkLBSeuyKLt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1972 (
feat/T13317-seen-txn-doctor). It gets no CI until it is retargeted to main.Proof (AC1): an origin's transactions are first delivered in local_seq order
I checked every path by reading the code on the stack top. The tests below cover the order, revive and rebind cases.
<replica>:<local_seq>(sealer.tssealInTransaction). The counter only rises and is never reused (SEAL_COUNTER_KEY, T13033).buildSegment(segments.ts) takesstate='sealed' AND replica=<own>inlocal_seqorder underBEGIN IMMEDIATE. It stops at a partial group. Each segment takesnextReplicaSeq, which is past the persisted high-water mark.pushStreamsends the lowest unpushedreplicaSeqfirst, one at a time, and retries with the same bytes. A same-seq/different-hash segment, or a server ahead of the store, meansstore-behindorE_CONFLICT, then rebind.journal.tspullrefusesreplica-gapandreplica-replay. A replica's segments therefore arrive in exactreplicaSeqorder, and with the packing rule its txns arrive inlocal_seqorder.uncutGenesisreturns folded txns tosealedonly whilegenesis_pendingis set, andpushStreamrefuses while it is set. So no later txn is packed before them, and this holds for crash-resume too.inherit.ts). Old-replica segments are never uploaded under the new id (unpushedSegmentsfilters by replica). The reconcile emits new txns under the new id, and the floors are per origin.replicaSeq.reemitOf, §2.11 §6). Neither one re-delivers the id.Preconditions this rests on, filed or noted:
local_seqorder per (stream, origin).closeCutfolds every sealed txn regardless of stream, and a txn has a singlesegmentedstate, so both need per-stream handling.Change
_sync_seen_floor, migration20261009120000_t13318-seen-floor. For each stream and origin it holdsstaged_upto,pruned_upto,seen_rowsandseen_bytes. The migration backfills it from seen rows already staged. The backfill isINSERT OR IGNORE, so a re-run is safe.refusedKind: 'below-floor'. The message says whether its seen row was pruned (it carries thepruned_uptomark) or it arrived out of order. The pull stops before that segment and never skips the txn as seen.refusedKind: 'segment'.pruneSeenTxns(db, stream)drops each origin's rows below its floor and keeps the floor's own row. The cloud pull now passespruneSeen: true. The unused stream-seq optionpruneSeenUpTois gone.seenTxnReportnow sums the per-origin counts, so it is O(origins). This fixes the review-hotfix LOW on feat(cloud): cloud status reports the seen-txn ledger size per store (T13317) #1972._sync_seen_flooralong with_sync_seen_txnand_sync_cursor.Verified
pull.test.ts: 12/12 pass. New tests:cloud-sync-status.test.ts(8),nexus-vault.test.ts,table-classification-gate.test.ts,sync-store.test.ts,nexus-attach.test.ts: 207 pass.mininstead ofmaxin the backfill, and made the report walk the ledger again.🤖 Generated with Claude Code
https://claude.ai/code/session_01VjjTKbQXyCFdkLBSeuyKLt