[v0.8 validation] #852 resume reclaim on accepted #851 - #1203
Draft
sethkarten wants to merge 38 commits into
Draft
[v0.8 validation] #852 resume reclaim on accepted #851#1203sethkarten wants to merge 38 commits into
sethkarten wants to merge 38 commits into
Conversation
… workers A worker with a durable stop intent is now reported as "stopping" and a disconnected worker is never reported as "ready". Stopping workers are excluded from live session lists and daemon-wide fan-out commands.
…y checks The list response feeds shutdownStaleDaemonIfNotBusy and probeRunningDaemonSessions, so hiding stopping workers made a tombstoned-but-running worker look idle and let a stale-daemon replace terminate it silently. Stopping workers stay listed with an honest "stopping" workerState; command fan-outs still skip them.
…ates
Revision 14 records the workerState wire-semantics change ("stopping"
state; disconnected workers no longer report "ready") so version probes
can tell old and new daemons apart. The field stays optional and
backward-tolerant, so no capability gate is needed.
… workers A worker with a durable stop intent is now reported as "stopping" and a disconnected worker is never reported as "ready". Stopping workers are excluded from live session lists and daemon-wide fan-out commands.
…y checks The list response feeds shutdownStaleDaemonIfNotBusy and probeRunningDaemonSessions, so hiding stopping workers made a tombstoned-but-running worker look idle and let a stale-daemon replace terminate it silently. Stopping workers stay listed with an honest "stopping" workerState; command fan-outs still skip them.
…ates
Revision 14 records the workerState wire-semantics change ("stopping"
state; disconnected workers no longer report "ready") so version probes
can tell old and new daemons apart. The field stays optional and
backward-tolerant, so no capability gate is needed.
…ng registrations When a worker does not exit within the stop deadline, the supervisor now keeps watching the process, escalates to SIGKILL, and completes the interrupted cleanup once the process dies. Process liveness checks also treat zombie processes as dead so cleanup is not deferred forever.
… generation The background finalizer now snapshots pid, processStartId, and stopRevision when scheduled and aborts if the stop is rescinded or the worker is relaunched, so it can never SIGKILL a retried worker or an unrelated process that reused the pid. stopWorker signalling is likewise identity-aware.
- Fail closed when a recorded processStartId cannot be observed, so a recycled pid is never signalled even if identity observation fails. - Record a schedule-time identity for workers that never had one. - Retry transient finalization cleanup failures instead of stranding the dead registration permanently. - Probe liveness with a cheap kill(0) on every poll and throttle the ps-backed zombie/identity checks so wedged workers cannot saturate the supervisor event loop.
…gone stopWorker used the identity check as a liveness predicate, so a transient getProcessStartId failure could skip signalling and delete the registration of a still-running worker. Identity verdicts are now directional: only a confirmed-current pid is signalled, only a confirmed-gone/replaced pid is cleaned up, and an unknown verdict keeps waiting.
…ched mid-await stopWorker can yield during archival while a retry rescinds the stop and relaunches the worker on the same registration. The cleanup tail now verifies the registered process is still the one it stopped before removing the registration or descriptor, so a relaunched worker is never orphaned by a stale stop invocation.
The throttled identity cache can be up to 500ms old, long enough for a pid to be recycled. Both SIGKILL sites (stopWorker force escalation and the stop finalizer) now run a fresh identity check immediately before signalling; the cache remains only for read-only wait-loop polling.
…ages A transiently unobservable identity at the escalation deadline now skips that attempt without marking the kill done, so a later pass that re-verifies the original process still escalates instead of leaving a wedged worker registered forever.
…scinded stops All stopWorker polling and signalling now use the pid and start identity captured at entry, so a retry relaunching the worker mid-stop can never be SIGKILLed through the mutable descriptor. The cleanup guard also aborts when a removeDescriptor stop lost its tombstone, catching a rescission that lands before the successor pid does.
Reopening a saved session used to fail forever when a stopped worker left a tombstoned registration behind (stop timed out, process died later, and finalization was interrupted). The supervisor now detects such stale registrations during create/resume, completes the interrupted stop, and launches a fresh worker for the same saved transcript.
…gistrations A recycled pid used to make a dead worker look alive, so its stale registration was never reclaimed and resume kept failing. Reclaim now checks processStartId, treating a recycled pid as gone; the stop path never signals a pid whose identity no longer matches.
…tion A timed-out stop already has a background finalizer completing the same cleanup, so the resume-time reclaim now awaits it instead of running a duplicate stop that could repeat archival and cron-lock cleanup. Also document the intentional finalizer/reclaim race in the end-to-end test: both paths are covered deterministically by unit tests.
…ervable Align resume-time reclaim with the directional identity verdicts: only a confirmed-gone or confirmed-replaced pid is reclaimed; a transient identity lookup failure leaves the registration untouched.
…inalizer Reclaim now checks confirmed process death first and then delegates the cleanup to scheduleWorkerStopFinalization, so concurrent resumes share one stop instead of duplicating archival, and the bounded wait keeps a resume request from blocking on a finalizer that cannot settle.
…esume When the bounded reclaim wait expires before the finalizer finishes, the resume now fails with a retry hint instead of falling through to reuseWorkerForCreate with a registration whose process is confirmed dead - the exact failure mode this PR heals.
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Validation scope
Draft prerequisite validation surface for the #852 stale-registration resume-reclaim delta replayed onto accepted #851
d254ed555d0e6ec6144295de518800a690dd6618.9de578b618fa276507b6505ec964ea42bb1d5815overda89f2d0834fb4921ffb618ccf5e8e5070f6a200.158e61269ac29bce322ff23ce93057b0bc6b7d10passed CI31462324600, including process smoke, but was rejected after valid Macroscope Highr3755486555found a root-kill ownership gap.3dced5e220762055d5a50b789d777182bf3000d0.stopWorkertuple assertions; it does not weaken lifecycle fencing.tsgo --noEmitpassed and correctly configured monitor passed 82/82.31463436712: SUCCESS, 10/10 jobs including process smoke. CodeQL31463434792: SUCCESS, 4/4 analyzers. Macroscope passed; fresh audit found zero unresolved threads andr3755486555resolved.v080/prereq-852-ci-candidateat exact158e61269…; contributor refs remain unchanged.This is not a human-readiness or merge claim. It validates exact parent-first prerequisite ancestry before final Core-common replay.
Note
Resume session reclaim on accepted stop by finalizing stale worker registrations
reclaimStaleWorkerRegistrationto the daemon supervisor so that when a session is resumed and a stale (stopped/dead) worker registration exists, the supervisor finalizes the stop and frees the slot for a fresh worker launch.processIdentityandsignalCurrentWorkerto prevent SIGTERM/SIGKILL from being sent to recycled PIDs that have been reused by unrelated processes.'stopping'state toDaemonWorkerLifecycle,SessionSummary, andWorkerEvictionSnapshot;effectiveWorkerStatenow reports'stopping'or'recovering'to clients instead of raw descriptor lifecycle.scheduleWorkerStopFinalization,finalizeTimedOutWorkerStop) escalates timed-out stops to SIGKILL and retries cleanup without leaving dead workers registered indefinitely.stopWorkeris substantially rewritten with tuple-fenced, race-resilient logic; stops may now complete asynchronously in the background after a timeout, which changes observable cleanup timing.📊 Macroscope summarized 158e612. 7 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted
🗂️ Filtered Issues
No issues evaluated.