Skip to content

feat(sync): prune seen txns below a per-origin local_seq floor, refuse loudly below it (T13318) - #1981

Merged
kryptobaseddev merged 5 commits into
mainfrom
feat/T13318-seen-floor
Oct 11, 2026
Merged

kryptobaseddev merged 5 commits into
mainfrom
feat/T13318-seen-floor

Conversation

@kryptobaseddev

Copy link
Copy Markdown
Owner

Stacked on #1972 (feat/T13317-seen-txn-doctor). It gets no CI until it is retargeted to main.

Proof (AC1): an origin's transactions are first delivered in local_seq order

I checked every path by reading the code on the stack top. The tests below cover the order, revive and rebind cases.

  • Mint: a txn id is <replica>:<local_seq> (sealer.ts sealInTransaction). The counter only rises and is never reused (SEAL_COUNTER_KEY, T13033).
  • Pack: buildSegment (segments.ts) takes state='sealed' AND replica=<own> in local_seq order under BEGIN IMMEDIATE. It stops at a partial group. Each segment takes nextReplicaSeq, which is past the persisted high-water mark.
  • Upload: pushStream sends the lowest unpushed replicaSeq first, one at a time, and retries with the same bytes. A same-seq/different-hash segment, or a server ahead of the store, means store-behind or E_CONFLICT, then rebind.
  • Receive: journal.ts pull refuses replica-gap and replica-replay. A replica's segments therefore arrive in exact replicaSeq order, and with the packing rule its txns arrive in local_seq order.
  • Genesis: uncutGenesis returns folded txns to sealed only while genesis_pending is set, and pushStream refuses while it is set. So no later txn is packed before them, and this holds for crash-resume too.
  • Rebind or copy: inherited captures and txns are never sealed or sent (spec §1.5 H3, inherit.ts). Old-replica segments are never uploaded under the new id (unpushedSegments filters by replica). The reconcile emits new txns under the new id, and the floors are per origin.
  • Retired replica: a late segment is still contiguous in its own replicaSeq.
  • Revive: revival replays the original op from the local inbox, and a re-emit is a new txn (reemitOf, §2.11 §6). Neither one re-delivers the id.
  • Idle no-pingpong (test(sync): two idle replicas exchange no segments after the first round (T13256) #1968), typed skip (refactor(sync): cloud sync classifies skipped legs by a typed refusal kind (T13315) #1970), checkpoint join: none of these mint or reorder txns. A restored store starts with an empty seen ledger and an empty floor, from the checkpoint cursor.

Preconditions this rests on, filed or noted:

  • Split parts: spec §2.4 gives every part of a split group the same txn id, which conflicts with seen dedup. Nothing produces split parts today, so this is latent. Filed as T13329.
  • Single stream per store: this holds today. When T13316 routes home-stream ops from a project store, packing must stay in local_seq order per (stream, origin). closeCut folds every sealed txn regardless of stream, and a txn has a single segmented state, so both need per-stream handling.
  • Rebind (T13278): the new rebind code must keep H3. It must never re-send an old-replica txn id under any replica.

Change

  • New local-only journal table _sync_seen_floor, migration 20261009120000_t13318-seen-floor. For each stream and origin it holds staged_upto, pruned_upto, seen_rows and seen_bytes. The migration backfills it from seen rows already staged. The backfill is INSERT OR IGNORE, so a re-run is safe.
  • A txn at or below its origin's floor that has no seen row refuses its segment with refusedKind: 'below-floor'. The message says whether its seen row was pruned (it carries the pruned_upto mark) or it arrived out of order. The pull stops before that segment and never skips the txn as seen.
  • A segment carrying a txn that is not its replica's is refused with refusedKind: 'segment'.
  • pruneSeenTxns(db, stream) drops each origin's rows below its floor and keeps the floor's own row. The cloud pull now passes pruneSeen: true. The unused stream-seq option pruneSeenUpTo is gone.
  • seenTxnReport now sums the per-origin counts, so it is O(origins). This fixes the review-hotfix LOW on feat(cloud): cloud status reports the seen-txn ledger size per store (T13317) #1972.
  • Vault bundles clear _sync_seen_floor along with _sync_seen_txn and _sync_cursor.

Verified

  • pull.test.ts: 12/12 pass. New tests:
    • a txn below the floor is skipped while its row exists and refused once pruned, and the refusal holds on retry;
    • a txn that was never staged and sits below the floor is refused as out of order;
    • a foreign txn id is refused;
    • the migration backfill;
    • rebind: a retired replica's late segment and the successor's segment interleave, and neither is refused;
    • revive: a void stays seen, the author's next txn is staged above the floor, and re-delivery of the void is skipped.
  • cloud-sync-status.test.ts (8), nexus-vault.test.ts, table-classification-gate.test.ts, sync-store.test.ts, nexus-attach.test.ts: 207 pass.
  • Mutation checks: every one was killed. I disabled the floor refusal, pruned the floor row too, dropped the foreign check, left the floor unraised, ignored seen rows, skipped the prune, used min instead of max in the backfill, and made the report walk the ledger again.
  • Biome on the diff set. Typecheck (tsc -b) and build pass. Gates 28, 36, 37, 21 and 38 (base: stack top) pass. Gate 39 exits 0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VjjTKbQXyCFdkLBSeuyKLt

kryptobaseddev and others added 4 commits October 9, 2026 21:04
…e loudly below it (T13318)

Proof first (code reading on the stack top, tests below): an origin's
transactions are FIRST delivered in local_seq order on every path. The
sealer mints local_seq rising (never reused, T13033); buildSegment packs
the lowest sealed txns of its own replica in order and stops at a partial
group; push uploads lowest replicaSeq first, resending the same bytes;
the server and every puller refuse a replicaSeq gap or replay. A raced or
crashed genesis cut returns folded txns to sealed only while
genesis_pending blocks push. Rebind and copy never re-send inherited txn
ids (spec H3); revival replays from the local inbox and a re-emit is a
new txn. A retired replica's late segment is still contiguous in its own
replicaSeq.

- _sync_seen_floor (new local-only journal table): per stream and origin
  the highest local_seq staged, pruned_upto, and the seen row/byte counts;
  the migration seeds it from rows already staged.
- A txn at or below its origin's floor with no seen row refuses its
  segment (refusedKind 'below-floor', naming pruned vs out of order); the
  pull stops before it and never skips it as seen.
- A segment carrying a txn that is not its replica's is refused.
- pruneSeenTxns drops each origin's rows below its floor (keeping the
  floor's own row); the cloud pull prunes after every pull. pullStream's
  unused stream-seq pruneSeenUpTo is replaced by pruneSeen.
- seenTxnReport sums the per-origin counts: O(origins), not O(rows)
  (review-hotfix LOW on #1972).
- Vault bundles clear _sync_seen_floor with _sync_seen_txn and the cursor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjjTKbQXyCFdkLBSeuyKLt
…s the ledger meanwhile (T13318, #1981 LOW)

A store whose journal records the floor folder but whose _sync_seen_floor
table is gone got nothing from ensureSyncSchema (the folder reads as
applied), so pull threw "no such table" and cloud status read 0. The pull
now heals the table (healSyncSchema re-runs the applied folders, which
re-seeds the counts from the seen rows), and seenTxnReport walks the
ledger until then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjjTKbQXyCFdkLBSeuyKLt
Base automatically changed from feat/T13317-seen-txn-doctor to main October 10, 2026 21:17
@kryptobaseddev
kryptobaseddev merged commit 182263e into main Oct 11, 2026
105 checks passed
@kryptobaseddev
kryptobaseddev deleted the feat/T13318-seen-floor branch October 11, 2026 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant