Repository navigation
effect runtime: a corrupt locator spawns a second daemon and doctor reports stopped #6871
Description
What happens
Rewriting only the live locator's schema_version (loopx_effect_runtime_info_v0
→ …_v1; token/host/port/pid untouched) makes the next RPC succeed against a
second runtime daemon, while the original keeps running and still answers a
raw runtime.ping with its saved token:
healthy: {"daemons":1,"daemon_pid":17902,"doctor_state":"running","doctor_diagnostic":null,"rpc_ok":true}
wounded: {"daemons":2,"daemon_pid":18047,"old_pid":17902,"old_pid_alive":true,"old_pid_serving":true,"doctor_state":"stopped","doctor_diagnostic":null,"rpc_ok":true}
loopx doctor reports state: stopped / diagnostic_code: null while the old
process is alive and serving, and nothing in the log records the misread. 5/5
runs.
Why this path is taken
_read_info collapses JSON decode failure, a wrong schema_version, and
fingerprint/token/port/pid mismatches into None
(loopx/control_plane/effect_runtime.py:459-497), and callers treat None as
"no runtime" and cold-start a new daemon (:1073-1077). Doctor's view comes from
the same locator read (:1196-1200), so it reports what the locator says, not
whether the recorded pid is alive.
Trigger honesty
The locator is published atomically under a file mutation lock
(effect_runtime_server.ts, atomicWriteJson), so a same-version crash does not
produce a corrupt one. This models a corrupt/restored temp file or an external
edit — rare, but the failure mode (an orphaned daemon per corrupt locator, plus
a false "stopped") is silent and unbounded until cleanup.
Expected
Distinguish "no locator" from "locator present but unreadable": keep a
diagnostic code, and have doctor report pid liveness alongside the locator state.
Before cold-starting, reuse a live daemon that still answers on the recorded
port/token instead of orphaning it.
Reproduced
Snapshot 105c4bc1; unchanged on main at 1c7b258cb. The script runs a real
RPC through the Python boundary, corrupts the live locator, runs doctor, proves
the old daemon still serves with a raw TCP runtime.ping, then repeats the same
RPC. Cleanup kills only processes whose argv carries the run id and whose command
contains effect_runtime_server.ts; verified 0 remaining after every run.
Activity
JasonBuildAI commentedon Oct 19, 2026
I'm taking this issue. I'll verify the corrupt-locator path against the latest main, add a readable-locator diagnostic plus liveness-aware reuse before cold start, and open a PR linked here once the reproduction and tests pass.
JasonBuildAI commentedon Oct 19, 2026
Implemented in #6892: #6892
The locator reader now distinguishes
missingfrominvalid, exposeslocator_stateandrecorded_pid_aliveinloopx doctor, and before cold-start authenticates the recorded endpoint/token withruntime.ping. The live daemon is reused only when the authenticated response matches its PID, fingerprint and locator path. An unreadable locator keeps its diagnostic instead of looking absent.Evidence: the new real-daemon test corrupts only
schema_version. Before the fix, doctor reportedstoppedand a second PID started while the original stayed alive; after the fix, doctor reportsrunning/invalid/recorded_pid_alive=true, and the next RPC reuses the original PID. Focused tests (21 passed), the Effect runtime integration file (97 passed), Ruff and the semantic advisory pass. The premerge catalog has four documented environment/baseline failures (Windows symlink privilege, missing Bash, the existingloopx-update-smokeassertion, and the pre-existing 44-vs-43 semantic twin budget), all detailed in the PR.AI disclosure: this change was drafted with AI assistance at my request, under my direction and review.
JasonBuildAI commentedon Oct 19, 2026
Follow-up on #6892: pushed
4b340d9e6to address the two P2 findings from independent review.loopx doctor --deepnow re-reads the locator after the probe may have started the runtime, so a successful probe cannot retain a stalemissingstate.locator_state=invalidwith a diagnostic, not as an absent locator.Validation on the final head: focused tests
43 passed; Effect runtime/readiness suite212 passed, 7 skipped; Ruff passed; mypy has no new normalized diagnostics versus764b989dc. DCOSign-offpassed and the remaining hosted checks are pending. No merge is requested.JasonBuildAI commentedon Oct 19, 2026
Follow-up on #6892: pushed
607a787ceto address the second independent-review boundaries.2 failed: the recovery path cold-started; a live valid locator was projected asmissing) and pass after it.Validation on
607a787ce: focused tests45 passed; Effect runtime/readiness suite214 passed, 7 skipped; Ruff passed. mypy exits 1 with the same 48 errors and 4 unique diagnostic shapes as base764b989dc, with no new diagnostic added by this branch. DCO and the remaining hosted checks are still running. No merge is requested.JasonBuildAI commentedon Oct 19, 2026
Follow-up on #6892: pushed
9dadb773dto close the third independent-review boundary.10**30from corrupted locator bytes) was probed before schema validation, which could raiseOverflowErrorin readiness and in the realloopx doctor --installation-only --format jsonpath. The reader now accepts only a positive integer within the PID range and contains probe overflow; such a locator is reported asinvalidwith unknown liveness instead of crashing.Both regressions failed before the commit (
2 failed) and pass after it (2 passed in 1.71s). On the final head: focused tests47 passed in 69.99s; Effect runtime/readiness suite216 passed, 7 skipped; Ruff passed. mypy exits 1 with the same 48 errors and 4 unique diagnostic shapes as base764b989dc, with no new diagnostic added by this branch. The exact-head premerge again passed all direct checks, the module ceiling, 8/8 risk-profile smokes and the public boundary; the same four catalog failures reproduce independently on764b989dc(Windows symlink privilege, missing Bash,loopx-update-smoke, and the pre-existing semantic-twin budget).No merge is requested; the PR is left for maintainer review.