Skip to content

fix: pin localmodels 0.1.0.dev40, recover a not-ready local model launch, clean up failed installs - #236

Closed
oldwinter53 wants to merge 3 commits into
mainfrom
oldwinter/014-bc2
Closed

oldwinter53 wants to merge 3 commits into
mainfrom
oldwinter/014-bc2

Conversation

@oldwinter53

Copy link
Copy Markdown

Summary

0.1.4 group B and C2 from the fix list: pin localmodels 0.1.0.dev40, recover a launch whose local model stack is not ready, and clean up after a failed install attempt so Try Again works without killing processes by hand (P1-10, P1-8, #173).

B: localmodels 0.1.0.dev40 (45eaccab)

  • Pins 0.1.0.dev40 (source a4de3e35) in apps/desktop/distribution/{darwin-arm64,linux-x64,win32-x64}.json, and in scripts/installs/stack.sh and stack.ps1, copied byte for byte from the published launchers. Also updates the README run-without-cloning URLs and syncs the scripts/install pins with sync-install-stack-pin.mjs.
  • I downloaded and checked each archive myself against release.json: darwin-arm64 14953559 B 648e8909…, linux-x64 20790291 B a8e176fa…, win32-x64 23191788 B de073060…, stack.sh 15225 B f4bc0795…, stack.ps1 32584 B 04fff205….
  • What dev40 changes for Linger: setup binding reports pi-extensions-unavailable, and stack apply replaces a jiti-less Pi. Harness apply now exits 1 with RUNTIME_ACTION_FAILED or RUNTIME_EXTENSIONS_FAILED. A plan with an unusable SIMPLEX_PI_PATH, or a Windows runtime root longer than about 90 characters without LongPathsEnabled, is refused at plan time. Stack down also stops the library Bifrost.

C2: launch recovery (P1-10, 5b9ccf3c)

Root cause: prepareLocalModels only observed the saved binding. When the binding was not ready it returned LOCAL_MODELS_RECOVERY_REQUIRED with "Review an updated plan". The start-failure page has no way to do that, so Try Again failed the same way and Set Up Again was the only exit. On dev40, a 0.1.3 machine with a jiti-less Pi reports pi-extensions-unavailable and lands on the same dead end.

  • packages/local-installation/src/local-models.ts: when a binding saved for the approved plan is not ready, and the saved plan inputs (planId, GPU sharing, no decision model) still match the current choices, the approved plan is applied again with stack apply --approve <planId>. The result is bound only once setup binding verifies it. This covers every binding reason, pi-extensions-unavailable included. Only a plan the person approved is ever applied: changed inputs still need a new review, and a missing or damaged inputs file never authorizes an apply. The binding's reason codes go out in details.reasons and to the installer log.
  • apps/desktop/src/main/local-server.ts (#runResume): the start-failure text puts the first known binding reason code in Linger's own words, for example "The local model's Pi can't load extensions, which every Linger run needs." Then, keyed by details.code, it names an action that works. For LOCAL_MODELS_RECOVERY_REQUIRED that is Try Again, which now applies the approved plan again, or Set Up Again. For LOCAL_MODELS_APPROVAL_REQUIRED it is Set Up Again. All of this is driven by codes; nothing matches on prose.
  • packages/local-installation/src/managed-runtimes.ts and dependencies.ts: localmodels harness errors now become structured failures.
    • A RUNTIME_ACTION_FAILED from harness plan becomes RUNTIME_PLAN_REFUSED (unusable SIMPLEX_PI_PATH, or a Windows runtime root too long for Pi).
    • RUNTIME_ACTION_FAILED and RUNTIME_EXTENSIONS_FAILED from apply keep their own codes. They used to be folded into LOCALMODELS_RUNTIME_UNAVAILABLE.
    • apps/linger-installer/src/server/jobs.ts puts these codes on the failed job's errorCode instead of INSTALL_FAILED.
  • Known limit: dev40 gives the Windows long-path refusal no code of its own, only RUNTIME_ACTION_FAILED at plan time. Linger therefore reports it as RUNTIME_PLAN_REFUSED and shows the upstream cause text, which names the root length and long paths. Telling it apart from other plan refusals needs a distinct code from localmodels. Linger does not copy the 90-character budget or the registry check.

C2: failed-install cleanup (P1-8 / #173, 7270430a)

Root cause: when startNativeProcess could not prove the new supervisor's lease identity (on 0.1.3 this was the localized ps lstart), its cleanup also found nothing by lease. The supervisor and PostgreSQL kept running with no record, and every Try Again then hit postmaster.pid already exists or "identity could not be verified".

  • packages/local-installation/src/native-process.ts, failed start: when the lease cannot be proved, the start now ends the process group of the supervisor it spawned and has not reaped yet. That PID cannot belong to another program.
  • reclaimNativeOrphans on macOS and Linux: it now also ends a supervisor whose command line names a job file in this instance's process directory under a lease no record holds, once that process's environment proves the lease. Windows already did this. The services step runs it before PostgreSQL starts, so a state left by 0.1.3 heals on the next attempt.
  • The POSIX stop of one lease is split out of stopNativeProcess so both paths share it.
  • Not changed: the installer manager (installer.js) still stays up for its idle timeout after a failed job. It does not block Try Again, which reuses that manager.

Docs

apps/desktop/AGENTS.md, apps/linger-installer/AGENTS.md and packages/local-installation/AGENTS.md are updated for the resume re-apply, the reason and action copy, the runtime error codes, and the POSIX reclaim.

Critical path

Nothing here touches the user message → provider_ready path. The extra work runs only while installing or resuming: one stack apply when the stack is not ready, and one process listing of this user's processes before services start.

Tests (each written first, failing before the fix)

  • local-models.test.ts: a not-ready binding (deployment-unavailable, pi-extensions-unavailable) applies the approved plan again and binds. A stack still not ready asks for recovery and keeps the earlier binding and its reasons. A failed re-apply reports the upstream cause. Changed GPU sharing never re-applies.
  • managed-runtimes.test.ts and dependencies.test.ts: plan refusal maps to RUNTIME_PLAN_REFUSED (Windows long-root text); apply keeps RUNTIME_ACTION_FAILED and RUNTIME_EXTENSIONS_FAILED.
  • jobs.test.ts: the runtime cause becomes the failed job's errorCode.
  • local-server.test.ts: the resume failure names the action for each code and words pi-extensions-unavailable.
  • native-process-unverified.test.ts (new) uses real processes. When every identity check fails, the start still ends the supervisor and the service.
  • native-process.test.ts uses real processes. A service whose record was lost is reclaimed, and a recorded one is untouched.
  • make check BASE=origin/main: repository, docs, packages, teamhub-cli, installer, desktop, scripts-fast and web all pass. make check-repository-gate BASE=origin/main passes.
  • One flake: native-process.test.ts ("cleans the launched owned group when identity persistence fails", which this PR does not touch) timed out twice while the host load average was 47–70. It passed 3 of 3 standalone runs and the make check-packages rerun.

Real-machine checks for the 0.1.4 RC round (not run here)

  • macOS (macmini, zh-CN, 24 GB): Quick Install Pi + local model, then stop the stack (stack down, or kill the llama server), quit, and launch. It should start because the resume applies the approved plan again. Also try Unload, then upgrade 0.1.3 → 0.1.4.
  • macOS: kill the installer job during the services step (or force a PostgreSQL start failure), then Try Again. It should succeed without a manual kill, and no stray native-process-worker.mjs or postgres -D … should remain from the first attempt.
  • Linux (local-118 harness): upgrade a 0.1.3 Quick Install whose Pi has no jiti. The resume should log pi-extensions-unavailable, stack apply should replace Pi, and a Pi chat with the computer tools should work.
  • Linux: Customize > Pi with a bad SIMPLEX_PI_PATH should fail with errorCode RUNTIME_PLAN_REFUSED and show the upstream cause.
  • Windows (homewin): Customize > Pi into a custom location with a runtime root over 90 characters and long paths off. It should be refused up front (RUNTIME_PLAN_REFUSED, text naming the root length), not fail mid-install.
  • Any platform: make a resume fail with the stack still not ready (for example, a port held). The start-failure page should show the reason sentence plus "Try Again … or Set Up Again …", and Try Again should work once the cause is gone.

Overlaps

#213 and #224 edit the same apps/desktop/distribution/*.json lines, so they rebase over one URL/sha change. This PR does not touch local-server.ts sections they edit beyond #runResume.

🤖 Generated with Claude Code

oldwinter90 and others added 3 commits November 19, 2026 02:53
0.1.0.dev40 replaces a Pi that cannot load extensions when the stack is
applied again, reports it in setup binding as pi-extensions-unavailable,
fails harness apply with RUNTIME_ACTION_FAILED or RUNTIME_EXTENSIONS_FAILED
instead of exiting 0 after an incomplete install, refuses an unusable
SIMPLEX_PI_PATH and a too-long Windows runtime root at plan time, reports
GPU and free-memory shortfalls with byte figures, and stops the library
Bifrost on stack down.

- Pin 0.1.0.dev40 in the three desktop distributions and in
  scripts/installs/stack.sh and stack.ps1, copied from the published
  launchers; the three tool archives and both launchers were checked
  against the release's SHA256 and size.
- Point the run-without-cloning URLs in scripts/installs/README.md at the
  pinned version, and sync the stack pins in scripts/install with
  scripts/sync-install-stack-pin.mjs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aves behind

A start whose lease identity could not be proved left its supervisor and
service running with no record, so PostgreSQL kept postmaster.pid and every
Try Again failed until the processes were killed by hand.

- When a start fails before its record is kept and the lease cannot be
  proved, end the process group of the supervisor it spawned and has not
  reaped yet; that PID cannot belong to another program.
- On macOS and Linux, reclaimNativeOrphans now also ends a supervisor whose
  command line names a job file in the instance's process directory under a
  lease no record holds, once its environment proves the lease; the
  services step runs it before PostgreSQL starts, so a state an earlier
  release left heals on the next attempt.
- Split the POSIX stop of one lease out of stopNativeProcess to share it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…en the stack is not ready

A launch that found the local model stack not ready failed with
LOCAL_MODELS_RECOVERY_REQUIRED and asked to review an updated plan, which
the start failure page cannot do; Try Again failed the same way and Set Up
Again was the only way out. A 0.1.3 Pi installed without extension support
reports pi-extensions-unavailable from localmodels 0.1.0.dev40 the same way.

- When the saved binding for the approved plan is not ready and the saved
  plan inputs still match, apply that plan again and bind once it verifies;
  changed inputs still need a new review. The binding's reason codes go out
  in details.reasons and to the installer log.
- The desktop's start failure words the first known reason code and names
  the action that works for LOCAL_MODELS_RECOVERY_REQUIRED (Try Again) and
  LOCAL_MODELS_APPROVAL_REQUIRED (Set Up Again).
- Map localmodels harness errors to structured failures: a refused harness
  plan (unusable SIMPLEX_PI_PATH, a Windows runtime root too long for Pi
  without long paths) is RUNTIME_PLAN_REFUSED, and RUNTIME_ACTION_FAILED and
  RUNTIME_EXTENSIONS_FAILED keep their codes through dependency preparation
  and into the failed job's errorCode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@linger-review linger-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes (1 P1, 1 P3)

Scope: base main, merge base 080cbcf9. I reviewed 3 commits and 26 files at 5b9ccf3c. Loaded skills: coderbook, ponytail-review and karpathy-guidelines. The review ran in a separate session.

Findings

P1

packages/local-installation/src/native-process.ts:769: the macOS process listing in ownCommands truncates command lines, so the new POSIX reclaim matches nothing on macOS

  • Problem: ownCommands runs /bin/ps -axo pid=,uid=,command= with no ww. When stdout, stderr and stdin are not terminals, macOS (BSD) ps cuts each line to about 79 columns unless -ww is passed. Two existing call sites pass it for exactly this reason: native-process-identity.ts:221,302 (eww) and the daemon's agent_chrome.go:42 and agent_browser.go:298 (-axww).
  • Failure scenario: The installer manager and the desktop app run ps through pipes, so the command column is cut to about 79 characters. A supervisor's command line is longer than that: the node exec path, then .../native-process-worker.mjs, then ~/Library/Application Support/.../native/processes/postgres.<64 hex>.job. The indexOf(jobs) lookup and the ^name.lease.job match therefore never succeed. reclaimUnrecordedLeases returns [], and an unrecorded supervisor and PostgreSQL left by 0.1.3 keep postmaster.pid against every retry. That is the macOS case this PR names: "a state left by 0.1.3 heals on the next attempt" and the macOS RC check list. Linux reads /proc/<pid>/cmdline and is unaffected, which is why the new native-process.test.ts case passes in CI.
  • Fix: Use ["-axww", "-o", "pid=,uid=,command="]. Ideally also add a darwin unit test like native-process-darwin.test.ts that asserts the ww argument, or feeds long ps output.

P3

apps/desktop/src/main/local-server.ts:2695: RESUME_ACTIONS gives "Try Again" for every LOCAL_MODELS_RECOVERY_REQUIRED, including the case where retrying cannot help

  • prepareLocalModels returns the same code when the saved inputs no longer authorize the plan (!approvedInputs, for example a missing or damaged inputs file). The page then shows "…Review a new plan…" followed by "Try Again to apply the approved local model plan again". In that case Try Again fails the same way every time.
  • Give that branch a distinct code that maps to the Set Up Again action, or carry a structured flag in details, so the action text stays code-driven.

Verified as fine

  • prepareLocalModels:
    • The re-apply path runs only when a saved binding names the current planId, localModelsReplan is unset, and the saved inputs match (planId, shareGpu, decision === false). A missing or damaged inputs file never authorizes an apply.
    • The result is persisted only after setup binding reports ready.
    • reasons are validated (^[a-z][a-z0-9-]{0,63}$) before they go into details and the log. The upstream message is trimmed and capped at 300 characters, for display only.
  • Desktop #runResume: messages are keyed by details.code and reason codes through Object.hasOwn lookups. Nothing matches on prose.
  • endUnreapedGroup: it signals -pid only while Node has not reaped the child. After that, a live group ID cannot be reused. It is unreachable on Windows, which returns earlier.
  • stopLease extraction: record removal still happens only after a successful POSIX stop, and the NATIVE_PROCESS_REPLACED path behaves as before.
  • reclaimUnrecordedLeases: it requires a recorded-lease miss, this instance's job directory, a known service name, and a lease proved through the process environment (nativeProcessIdentity) before it signals anything. The services step and stopNativeSource call it under withInstanceLock.
  • runtimeFailureCode replaces the two duplicated LOCALMODELS_RUNTIME_UNAVAILABLE checks. RUNTIME_ACTION_FAILED becomes RUNTIME_PLAN_REFUSED only for harness plan. The installer copies these codes to errorCode only for failed jobs, and no desktop or installer UI consumer assumes a closed set.
  • Pins: the darwin, linux and win32 distribution JSON, stack.sh and stack.ps1, the README URLs and the scripts/install pins all move together to dev40.
  • Docs: apps/desktop, apps/linger-installer and packages/local-installation AGENTS.md are updated for the new behavior.
  • Tests: there are tests for the re-apply branches, the error-code mapping, job errorCode, the resume copy, the unverified-start cleanup and the reclaim of an unrecorded lease.

Process notes

  • No merge commits and no fixup commits. The branch is up to date with main. BLOCKED reflects required review only. Commits are authored by linked account oldwinter90.

CI

  • All checks pass: desktop, installer, packages, teamhub-cli, web, web-static, repository-gate, CodeQL and the three web test shards.
  • computer-go, contracts and teamhub were skipped by path; the PR doesn't touch them.
  • CI runs on Linux, so the macOS ps path in ownCommands is untested. I ran no tests myself.

Automated review by Claude Code


Automated review of 5b9ccf3 by Claude Code.

@oldwinter53

Copy link
Copy Markdown
Author

Consolidated into #244 with #235, #236 and #238 (same commits, replayed on the newest main; the body carries this PR's description, tests and real-machine checks). Please review there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants