Skip to content

syncsetup: a window displaced by a newer hold on the same computer lets go at once - #2034

Merged
AbirAbbas merged 6 commits into
app/v3from
fix/drive-available-retires-displaced-hold
Oct 29, 2026
Merged

AbirAbbas merged 6 commits into
app/v3from
fix/drive-available-retires-displaced-hold

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Two windows or processes on one computer can share a codeaf home, for example the desktop and a terminal. When one of them takes a chat over at a higher fence, the other window is displaced. The displaced drive went to available() and stayed there: it kept reporting the hold in RenewalStatus and kept its send receipt (v3/sync/holds/<hash>) until a heartbeat came round. Drive.available (internal/syncsetup/confirm_work.go) neither retired the receipt nor woke renewal readers. Both same-device routes into it have this gap: becomeViewer (reached from the ownership watch's CheckLease) and ConfirmWork.

Found on a Mac (macOS 15.6, Go 1.26.5, furrow staged). There, TestWatchDisplacementRetiresTheDriveHoldImmediately/higher_local_fence fails on every run with "watch displacement left the drive held until a heartbeat" after 5.03 s. The cause is pure logic, not darwin code. The test only looks Linux-green because Linux CI stages no furrow, so it skips there.

What changed

  • availableAfter(held, reading) replaces the plain path wherever a fresh relay reading is in hand: the expiry and same-device branches of becomeViewer, CheckLease and ConfirmWork. It works in three steps:
    1. It closes the work gate first. This is the order becomeViewer's other-device path already used.
    2. It retires only the receipt the drive held before the reading. CheckLease, ConfirmWork and becomeViewer snapshot {directoryID, recordedFence} before they read. Removal goes through the existing conditional forgetSendLease, so a newer local writer's receipt survives, and so does a receipt this window reacquired meanwhile.
    3. Only then does it tell readers. renewalChanged() runs, then OnViewer, and only on a change.
  • A reading that still names this device at the snapshot's fence keeps the receipt, whether the lease is lapsed or released. clusterstate.Heartbeat renews an unchanged fence even after a release, so the window can resume. The receipt is retired when the reading names another device or a higher fence, even if that newer fence has since expired or been released.
  • A batcher branch supersession comes without a reading. It follows a refusal of this drive's fence under the flush lock, and it retires.
  • A failed heartbeat (renewIdleLease) and a failed receipt write after a reacquire still go through plain available(), which keeps the receipt.

Composition with #2031 (merged as 1c4ec8a; this branch is rebased onto it):

  • pair: merge a computer's own chats when it joins, instead of replacing them #2031 renamed lockSendLease to lockChat(id, suffix) inside recordSendLease/forgetSendLease, and added a .publication lock in driving() and PublishCopy.
  • This branch calls only forgetSendLease, whose meaning pair: merge a computer's own chats when it joins, instead of replacing them #2031 kept. It touches neither the lock helpers nor driving(), and the rebase had no conflicts.
  • lockChat(id, "") (the receipt lock) is taken only inside recordSendLease and forgetSendLease. Both are leaf file operations that call nothing back, so no door reaches forgetSendLease while already holding the lock.
  • The receipt and publication locks are separate files. The only ordering is publication → receipt (driving → reconcileSendLease), so there is no cycle.

How it was checked (this Mac)

  • New engine-free tests in internal/syncsetup/displaced_same_device_test.go (no staged furrow, no build tags, so Linux CI runs them):
    • same-device takeover through becomeViewer, ConfirmWork and CheckLease (own receipt, and a newer writer's receipt);
    • a stale refusal after a reacquire;
    • a deterministic barrier: the reacquire records its receipt during the displacing read, and that receipt survives;
    • a released same fence keeps its receipt and resumes after a heartbeat;
    • the gate refuses work while retirement is blocked on the receipt lock, with no notifications until the lock is released;
    • expired and released replacement fences on all three doors;
    • a lapse that may resume keeps its receipt.
  • Every new test fails on the code it replaces. They pass ×3, pass under -race, and pass with furrow unstaged.
  • TestWatchDisplacementRetiresTheDriveHoldImmediately: higher_local_fence FAIL (5.03 s) on app/v3, PASS (0.03 s) here; another_holder still passes.
  • go vet ./internal/syncsetup passes on darwin and GOOS=linux; gofmt is clean.
  • The whole ./internal/syncsetup package, the receipt consumers' tests in tui3 and cmd/codeaf, the Mac full-tree gate and the dispatched CI runs are reported below.

Codex adversarial review, three rounds:

  • Round 1. Fixed: a stale refusal could delete a reacquired receipt, and an expired or released replacement bypassed retirement.

  • Round 2. Fixed: retirement read recordedFence after the reading; a released same fence was retired although it can resume; admission blocked on the receipt lock before closing the gate.

  • Round 3. Rebutted: the reviewer kept one major, that the initiating ConfirmWork call can still wait without bound if another process is suspended while holding the receipt lock. The gate does close first, so waiting admissions refuse within their budget. The same blocking lock is already taken synchronously on the same admission paths:

    • ConfirmWork's success path (recordSendLease, confirm_work.go:53);
    • the tool gate's CheckLease → becomeViewer other-device path (forgetSendLease, drive.go:433 on app/v3);
    • drive start, take and close.

    Every holder covers only a read plus rename/unlink. Making this one branch asynchronous would let an "available" notice arrive after a later resume. A bounded lock belongs to the helper pair: merge a computer's own chats when it joins, instead of replacing them #2031 renames, so it should be a separate change for every caller.

  • Pre-existing, not touched here (follow-up): becomeViewer's other-device path keeps a stale s.By hint and retires recordedFence without a snapshot. A delayed refusal naming another device can therefore delete a receipt that this window reacquired. The round-3 review confirmed this branch neither adds nor widens that path.

After the rebase onto #2031

New test TestDisplacementRetiresWithoutHoldingTheReceiptLock:

  • What it does: runs becomeViewer, ConfirmWork and CheckLease through a same-computer displacement against a 5 s deadline. It probes the receipt lock non-blocking at the moment each door announces the change.
  • Results: passes ×3 under -race. If ConfirmWork is made to take lockChat(id, "") before retiring, it fails at 5 s.
  • Windows: a closed handle's lock may be released late there, so only the deadline applies on Windows.

Results on the rebased head:

  • Focused set with furrow staged (all 11 displacement tests, including TestWatchDisplacementRetiresTheDriveHoldImmediately/higher_local_fence): passes ×3.
  • go vet ./internal/syncsetup: darwin, GOOS=linux and GOOS=windows.
  • Package runs on this Mac: ./internal/syncsetup plus every package that imports it (cmd/codeaf, internal/tui3, internal/enginehost, internal/raftdirectory), on the branch and on app/v3 1c4ec8a, compared by test name. higher_local_fence is fixed and nothing new fails.
  • Learner flakes: two learner tests time out at 30 s about half the time on both trees under load (TestR9R23PersonalLearnerRefreshesColdRouterGeneration, TestR26R27C30OpenLearnerIdleAndStateWake; ×5 reruns on each tree).
  • CI (PR gate 40941512764 and Full check 40941517031, against app/v3 1c4ec8a's 40940481074 and 40940487776):
    • The full tests show the same 28 failures as the base.
    • One addition is TestSeniorDevShellResolvesBareAndCrewModelWords. It is a temp-dir cleanup race ("directory not empty") and passes on this Mac on both trees.
    • The touched-packages leg flags TestR7PersonalLeaderFallbackAndNoAutomaticRegain/unpaired. That test also fails in the base's own full check, and it passes on this Mac on both trees.

Codex review, round 4, on the rebased head:

  • Locks: no self-deadlock and no lock-order cycle. Composition with pair: merge a computer's own chats when it joins, instead of replacing them #2031's driving(), PublishCopy and pair-merge copy is sound.
  • Major (round 3's, repeated): the initiating ConfirmWork can wait on another process's receipt lock. The rebuttal stands: app/v3's ConfirmWork resume path already takes the same blocking lock synchronously, under d.mu (recordSendLease). A bounded lock helper for every caller is a separate change.
  • Minor: the probe could falsely fail on Windows. Fixed.

Manual verification, real processes on this Mac (app/v3 1c4ec8a vs this branch, both built with the shipped sidecar recipe):

  • Setup: a local relay on loopback with two homes paired through it.
  • Displaced side: codeaf app --home A, the engine process the desktop runs, driven over its wire.
  • Taker: a second process on the same home A takes the chat at a higher fence on the same device.
  • Before: the viewer line flips to "available", but the receipt v3/sync/holds/<hash> stays until the process quits. One run still had it after 130 s, through four heartbeat ticks.
  • After: the receipt is removed 15–55 ms after the takeover and stays gone. In every trial with precise timestamps it was removed just before the "available" event.
  • Taker holding the receipt lock for 3 s or 8 s: sends are refused in ~6 ms throughout and pings stay under 9 ms. The receipt is retired 1–2 ms after the lock is released. No hang, and quit takes 0.13–0.41 s.

🤖 Generated with Claude Code

AbirAbbas added a commit that referenced this pull request Oct 29, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AbirAbbas and others added 5 commits October 29, 2026 15:25
…ts go at once

When another window or process on this computer takes a chat over at a
higher fence, the displaced drive went to available() without retiring its
send receipt or waking renewal readers, so it kept reporting the hold and
kept its hold file until a heartbeat (TestWatchDisplacementRetiresTheDrive
HoldImmediately/higher_local_fence).

The two same-device displacement paths (becomeViewer and ConfirmWork) now
retire this drive's own receipt conditionally, preserving a newer local
writer's, before available() notifies. available() wakes renewal readers on
a state change before OnViewer. A lapse that may renew the same fence keeps
its receipt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…her hold

Review found two gaps in the first commit. A refusal that arrives after this
window reacquired the chat could delete the receipt of the hold it now
drives, and a replacement fence that had already expired or been released
reached available() through the expiry branches and left the displaced
receipt behind.

retireSendLease now takes the fresh relay reading and keeps the receipt only
while that reading names this device at the receipt's own unreleased fence
(a lapse this drive may renew). The expiry branches of becomeViewer,
CheckLease and ConfirmWork retire before available() too. Without a fresh
reading, becomeViewer retires only for a batcher branch, which follows a
refusal of this drive's fence under the flush lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re the reading

Review round 2 found three more gaps. A reacquire that recorded its receipt
between a displacing reading and retirement could lose it, because
retirement read recordedFence afterwards. A released lease of an unchanged
fence can still be renewed by a heartbeat (clusterstate.Heartbeat checks
device and fence only), so retiring its receipt could strand a window that
resumes. And the same-device admission branch now waited on the receipt
lock before its work gate closed.

CheckLease, ConfirmWork and becomeViewer snapshot the receipt before they
read the relay. availableAfter closes the gate, removes only that snapshot
(conditionally, so newer receipts survive) unless the reading still names
this device at the snapshot's fence whatever its expiry, and then tells
readers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…holds

The receipt lock (lockChat(id, "")) is a file lock, so a door that
held it while retiring the displaced receipt would wait on itself.
Each of becomeViewer, ConfirmWork and CheckLease must finish, and
nothing on its call chain may hold the lock when it announces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas force-pushed the fix/drive-available-retires-displaced-hold branch from ca74246 to e967bc8 Compare October 29, 2026 19:28
Windows may release a closed handle's lock late, so an immediate
non-blocking probe there could fail after correct cleanup. The
five-second deadline still proves no door waits on itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas marked this pull request as ready for review October 29, 2026 20:10
@AbirAbbas
AbirAbbas merged commit bd98788 into app/v3 Oct 29, 2026
12 of 20 checks passed
@AbirAbbas
AbirAbbas deleted the fix/drive-available-retires-displaced-hold branch October 29, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant