diff --git a/apps/desktop/AGENTS.md b/apps/desktop/AGENTS.md index b82fc3cf..0fb07a66 100644 --- a/apps/desktop/AGENTS.md +++ b/apps/desktop/AGENTS.md @@ -138,8 +138,14 @@ Gateway and runs Agents. The daemon exits with the app. Quick Install installs the `recommendation` of the localmodels setup report (`installer.js --inspect`, run once as the setup window opens): `local-models` is the planned model served by Pi, shown with the model, - GPU, memory and download, plus the tool's caveat when it has one; - otherwise the named runtime, with the tool's reason for no local model. + GPU, memory and download (the card's memory is the whole stack's, so + "with its services"); otherwise the named runtime, with why there is no + local model. The page words each reason by its code and the counts it + carries (`gpuIndex`, `freeBytes`, `requiredBytes`); the tool's message + goes only to the main process log (`local_server.inspected`), never to + the page. Under Customize the Local model card stays visible once the + check ends: disabled, "Not available on this computer", with that reason. + Off macOS the page names the system tray, not the menu bar. Under Customize the local model runs only behind Pi: with another runtime it is greyed (`aria-disabled`, still focusable), does not toggle, and says "Local models are only available with the Pi runtime."; choosing another @@ -149,10 +155,11 @@ Gateway and runs Agents. The daemon exits with the app. memory it needs now; a tool before that field marks it by `basis: "capacity"` or reason `free-memory-required`, either one), the install would plan again against the memory free then and stop, so the local model - shows on hold and is not offered. With the tool's `stack.memory` the page - says "quit other programs first; it needs X of memory, Y is free" (and the - graphics memory on a discrete GPU), adding the tool's words for any other - reason code; without it, the tool's reason. Quick Install installs the + shows on hold and is not offered. `gpu-memory-required` names the GPU and + its free against needed memory and says to close programs that use the + GPU; `free-memory-required` gives the RAM counts (from the reason, or the + tool's `stack.memory` when the reason has none). The page still words + `gpu-ownership-unknown`, which localmodels 0.1.0.dev39 reports. Quick Install installs the runtime the installer recommends instead, and Check again (`local.recheck`) runs the check again. A tool before `stack.state` still recommends `local-models` there, so the installer's setup adapter @@ -174,8 +181,9 @@ Gateway and runs Agents. The daemon exits with the app. services on launch with `installer.js --resume`, which also restarts them when the instance's `user.env` changed. A source folder removed since, or one the installer no longer accepts as a Linger checkout (such as one the - earlier Simplex app unpacked), is unpacked again from the app's package - first and replaces the recorded one. The record also names the app + earlier Simplex app unpacked, or this package's own folder with files + missing since), is unpacked again from the app's package first and + replaces the recorded one. The record also names the app version that unpacked the source; when this app brings another source package (it was installed over an older one), that launch unpacks it and resumes on it, with the "Updating Linger" page instead of the older @@ -185,7 +193,12 @@ Gateway and runs Agents. The daemon exits with the app. from their folder, the record keeps naming it, and the start failure page's Retry moves them again. An app older than the recorded version (a downgrade) leaves the services on their newer source and logs - `local_server.source_newer_than_app`. A Quick Install that stopped + `local_server.source_newer_than_app`. When the local model stops a resume, + the start failure names the stack's first known binding reason code + (`details.reasons`, such as `pi-extensions-unavailable`) in its own words + and, by `details.code`, the action that works: Try Again for + `LOCAL_MODELS_RECOVERY_REQUIRED` (the resume applies the approved plan + again), Set Up Again to review a new plan or choose another runtime. A Quick Install that stopped before the computer enrolled is not resumed: setup shows again and Install finishes it, reusing a finished job only while its origin answers (a stopped server gets a new job, which starts the services again). The start failure @@ -270,7 +283,10 @@ Gateway and runs Agents. The daemon exits with the app. `desktop` section puts the daemon in Desktop mode: no self-update, it exits when its stdin closes, and it reports `{"type":"status",...}` lines on stdout, which drive the tray and the SPA status. Its log goes to - `/logs/computer-daemon.log`. Crashes restart with backoff; new + `/logs/computer-daemon.log`. Crashes restart with backoff; the + exit reason, logged with `computer_daemon.exited` and shown once starts + keep failing, is the last stderr line outside the daemon's log records (a + Go crash: its `panic:` line), or none; new folders or a new name restart it once tasks finish. On macOS it writes `{"type":"hello","features":["permissions.v1"]}` to the daemon's stdin after spawn and answers `permission.request` lines with @@ -326,12 +342,25 @@ Gateway and runs Agents. The daemon exits with the app. on `--quit`, so setup skips `--quit` for it (CloseLinger asks the person to quit a running app). Retry in that prompt runs the stop again. Nothing running: the stop finds nothing and setup goes on. The stop also asks the pinned localmodels kit, when that kit is installed in the - home, to end its foundations controller under `runtime/model-services` - (`foundations stop`) and the stack's processes (`stack down`); it downloads - nothing, and Linger does not end the tool's processes itself. + home, to unload every model its model library loaded (`library status`, + then `library unload` for each `loaded` model), to end its foundations + controller under `runtime/model-services` (`foundations stop`), and to stop + the stack's processes and the gateway the tool started, the library's + Bifrost included (`stack down`); it downloads nothing, and Linger does not + end the tool's processes itself. Every step runs even when an earlier one + fails. On Windows the tool cannot prove and end a controller from a kit that has no stop command. - The tool refuses the chat stack on Windows, so no engine runs there. + A quit the person chose (the tray, the app menu, the Dock) does the same + for the local models, except the controller, which serves the services + that keep running: `LocalServer` starts `installer.js --home --stop-local-models` under `ELECTRON_RUN_AS_NODE`, detached, with + its output in `/installer/local-models-stop.log`, and does not wait + for it. It runs only when `desktop/local-server.json` names an + installation. The Computer exits with the app, so nothing uses the models + until the next launch, whose resume applies the approved plan again. A + quit for an update or a relaunch, and the `--quit` of setup or the + uninstaller (which stop the models themselves right after), leave them. - `src/main/legacy-migration.ts`: on start, stops a bridge an older version left running and moves its credential and the Desktop 0.16 daemon identity into the daemon layout, so the computer keeps its id. diff --git a/apps/desktop/distribution/darwin-arm64.json b/apps/desktop/distribution/darwin-arm64.json index d6c637ad..391fb54b 100644 --- a/apps/desktop/distribution/darwin-arm64.json +++ b/apps/desktop/distribution/darwin-arm64.json @@ -2,11 +2,11 @@ "platform": "darwin", "arch": "arm64", "localmodelsTool": { - "url": "https://d2ikbbcl3j6jz9.cloudfront.net/cli/localmodels/0.1.0.dev39/darwin-arm64.tgz", - "sha256": "e15c1d96c4db70cc834b42693228c6b6bbe7eb575e79d19e8aff2f0bf8ba6e48", - "bytes": 14937177, - "sourceCommit": "9a72159a9c114e323de703fcbf6e030fbf7d480a", - "manifestSha256": "b9f750c55834913394ebc459395f5331915d02d98ebeecee3586fb1721a7bc41" + "url": "https://d2ikbbcl3j6jz9.cloudfront.net/cli/localmodels/0.1.0.dev40/darwin-arm64.tgz", + "sha256": "648e8909915d8da2b87ae4f4797cd22f2d46d8a7cd025287ab9e6b5436119687", + "bytes": 14953559, + "sourceCommit": "a4de3e359d2a4f9b654f9ae7bdf6887c55896b7d", + "manifestSha256": "15f7138847ef4d13f9c2df455ec04423dc35d30d759d5f952dcb0b69d81db13b" }, "nativeSources": [ { diff --git a/apps/desktop/distribution/linux-x64.json b/apps/desktop/distribution/linux-x64.json index a425f170..ad7b3ee7 100644 --- a/apps/desktop/distribution/linux-x64.json +++ b/apps/desktop/distribution/linux-x64.json @@ -2,11 +2,11 @@ "platform": "linux", "arch": "x64", "localmodelsTool": { - "url": "https://d2ikbbcl3j6jz9.cloudfront.net/cli/localmodels/0.1.0.dev39/linux-x64.tgz", - "sha256": "f58f55f94e3559ffd959af9e9f44663a41ec34aef53a1c2d1ff7279ec207af3a", - "bytes": 20776267, - "sourceCommit": "9a72159a9c114e323de703fcbf6e030fbf7d480a", - "manifestSha256": "60664ab53e44b9967bf21e54a2e2fb263862296537ebd24691ab96e7ed268336" + "url": "https://d2ikbbcl3j6jz9.cloudfront.net/cli/localmodels/0.1.0.dev40/linux-x64.tgz", + "sha256": "a8e176fa426360f489bf9c552064faf7e1dd88072b463a13185ff1193a223683", + "bytes": 20790291, + "sourceCommit": "a4de3e359d2a4f9b654f9ae7bdf6887c55896b7d", + "manifestSha256": "d36a909ddad7be0de0642766102a5478efd22d7d437c86d8b63ac7bddc24e082" }, "nativeSources": [ { diff --git a/apps/desktop/distribution/win32-x64.json b/apps/desktop/distribution/win32-x64.json index f20abd5f..26966b59 100644 --- a/apps/desktop/distribution/win32-x64.json +++ b/apps/desktop/distribution/win32-x64.json @@ -2,11 +2,11 @@ "platform": "win32", "arch": "x64", "localmodelsTool": { - "url": "https://d2ikbbcl3j6jz9.cloudfront.net/cli/localmodels/0.1.0.dev39/win32-x64.tgz", - "sha256": "492ed40845a7d732f077b575c7ef4d75f06965c991e5e2c4c0e9f356ed0da37f", - "bytes": 23174114, - "sourceCommit": "9a72159a9c114e323de703fcbf6e030fbf7d480a", - "manifestSha256": "f17122b311b71a3b37e3207a2c0cdce6f84b3a14e09ef0bceed81be74e90e161" + "url": "https://d2ikbbcl3j6jz9.cloudfront.net/cli/localmodels/0.1.0.dev40/win32-x64.tgz", + "sha256": "de07306075b5efbf2158c41a7c87fe4939bac45b9b6f953270ed00017a6540be", + "bytes": 23191788, + "sourceCommit": "a4de3e359d2a4f9b654f9ae7bdf6887c55896b7d", + "manifestSha256": "c7e2fab80d7aa2bdf52975a0b6e180611830a5d3c013bb302d16ff5c3c0d42dd" }, "nativeSources": [ { diff --git a/apps/desktop/src/main/computer-daemon.test.ts b/apps/desktop/src/main/computer-daemon.test.ts index b7c7ff27..8a7d0518 100644 --- a/apps/desktop/src/main/computer-daemon.test.ts +++ b/apps/desktop/src/main/computer-daemon.test.ts @@ -630,4 +630,57 @@ describe.skipIf(process.platform === "win32")("spawnDaemon", () => { expect(reason).toBe("read Computer config: denied"); expect(readFileSync(log, "utf8")).toContain("starting\n"); }); + + const exitOf = async (script: string) => { + const directory = mkdtempSync(path.join(os.tmpdir(), "spawn-daemon-")); + const executable = path.join(directory, "daemon"); + writeFileSync(executable, `#!/bin/sh\n${script}`); + chmodSync(executable, 0o755); + const log = path.join(directory, "logs", "computer-daemon.log"); + const child = spawnDaemon(executable, log)("config.json", {}); + return new Promise<[number | null, string?]>((resolve) => + child.onExit((...exit) => resolve(exit)), + ); + }; + + it("never takes a log record for the exit reason", async () => { + const [code, reason] = await exitOf( + [ + 'echo "2026/11/17 20:26:26 INFO local_models.job_finished action=download exit_code=0" >&2', + 'echo "2026/11/17 20:29:38 WARN runtime.diagnostic type=pi status=ready" >&2', + "kill -9 $$", + ].join("\n"), + ); + expect(code).toBeNull(); + expect(reason).toBeUndefined(); + }); + + it("keeps the error the daemon printed when log records follow it", async () => { + const [, reason] = await exitOf( + [ + 'echo "2026/11/17 20:26:26 INFO daemon.started" >&2', + 'echo "read Computer config: private path is not owned by the current user" >&2', + 'echo "2026/11/17 20:26:27 INFO runner.shutdown phase=completed" >&2', + "exit 1", + ].join("\n"), + ); + expect(reason).toBe( + "read Computer config: private path is not owned by the current user", + ); + }); + + it("reports a Go panic by its panic line, not its stack trace", async () => { + const [code, reason] = await exitOf( + [ + 'echo "panic: assignment to entry in nil map" >&2', + "echo >&2", + 'echo "goroutine 1 [running]:" >&2', + 'echo "main.main()" >&2', + 'echo "\t/src/main.go:10 +0x1d" >&2', + "exit 2", + ].join("\n"), + ); + expect(code).toBe(2); + expect(reason).toBe("panic: assignment to entry in nil map"); + }); }); diff --git a/apps/desktop/src/main/computer-daemon.ts b/apps/desktop/src/main/computer-daemon.ts index 5fd584d8..dceacb73 100644 --- a/apps/desktop/src/main/computer-daemon.ts +++ b/apps/desktop/src/main/computer-daemon.ts @@ -545,6 +545,12 @@ export class ComputerDaemonModule implements Module { // Starts the daemon executable with only the given environment. Its stderr // (the daemon log) and any stdout that is not a status line go to logFile. +// A record of the daemon's log: Go's log package prefix, then the level. +const LOG_RECORD = + /^\d{4}\/\d{2}\/\d{2} \d{2}:\d{2}:\d{2}(?:\.\d+)? (?:DEBUG|INFO|WARN|ERROR)\b/; +// The first line of a Go runtime crash. +const GO_CRASH = /^(?:panic|fatal error): /; + export const spawnDaemon = (executable: string, logFile: string): SpawnDaemon => (configFile, env) => { @@ -576,9 +582,15 @@ export const spawnDaemon = }); }; child.stderr.pipe(log, { end: false }); + // The daemon prints why it stopped as its last line outside its log; + // its log records (a timestamp, then the level) may come from any time + // before. A Go runtime crash leads with its panic line, then a trace. + let crashed = false; createInterface({ input: child.stderr }).on("line", (line) => { const text = line.trim(); - if (text) reason = text.slice(0, 500); + if (!text || crashed || LOG_RECORD.test(text)) return; + crashed = GO_CRASH.test(text); + reason = text.slice(0, 500); }); createInterface({ input: child.stdout }).on("line", (line) => { if (line.startsWith("{")) for (const listener of lines) listener(line); diff --git a/apps/desktop/src/main/index.ts b/apps/desktop/src/main/index.ts index 837acaf8..b1220d56 100644 --- a/apps/desktop/src/main/index.ts +++ b/apps/desktop/src/main/index.ts @@ -326,10 +326,14 @@ async function main(): Promise { const host = new ModuleHost(modules, { config, log, bus: new EventBus() }); const showApp = (): void => setup.showApp(); + // The uninstaller, or setup replacing this release, asked to quit. + let replacing = false; app.on("second-instance", (_event, argv) => { // Without asking: the uninstaller is removing the app. - if (quitRequested(argv)) app.quit(); - else showApp(); + if (quitRequested(argv)) { + replacing = true; + app.quit(); + } else showApp(); }); app.on("activate", showApp); // Closing the window only hides it; the tray keeps the app running. @@ -340,10 +344,14 @@ async function main(): Promise { if (quitting) return; quitting = true; event.preventDefault(); - void host.stop(relaunch || updating ? "update" : "quit").finally(() => { - if (relaunch) app.relaunch(); - app.exit(0); - }); + // Setup and the uninstaller stop the services and the local models + // themselves right after --quit; a stop from here would race theirs. + void host + .stop(relaunch || updating || replacing ? "update" : "quit") + .finally(() => { + if (relaunch) app.relaunch(); + app.exit(0); + }); }); // Tasks run in the app, so quitting takes the computer offline; ask first diff --git a/apps/desktop/src/main/local-server.test.ts b/apps/desktop/src/main/local-server.test.ts index 65b421c0..3c525fad 100644 --- a/apps/desktop/src/main/local-server.test.ts +++ b/apps/desktop/src/main/local-server.test.ts @@ -76,6 +76,12 @@ if (args.includes("--user-keys")) { process.stdout.write(JSON.stringify({ saved: names }) + "\n"); process.exit(0); } +if (args.includes("--stop-local-models")) { + // The quit's stop of the local models; its output goes to a log file. + fs.writeFileSync(path.join(home, "stop-local-models.json"), JSON.stringify({ args, runAsNode: process.env.ELECTRON_RUN_AS_NODE })); + process.stdout.write(JSON.stringify({ failed: [] }) + "\n"); + process.exit(0); +} if (args.includes("--migrate")) { // Converts every legacy instance record, except a "desktop" that // migrate-skips holds back. @@ -113,8 +119,11 @@ if (args.includes("--inspect")) { } process.stderr.write("Checked this computer in 1.5s.\n"); const lowMemory = eligible && fs.existsSync(path.join(home, "low-memory")); - const reasons = eligible ? (lowMemory ? ["free-memory-required"] : []) : ["facts-unavailable"]; - const reasonMessages = eligible ? (lowMemory ? ["Close other programs before installing."] : []) : ["nvidia-smi was not found"]; + const reasons = eligible + ? lowMemory + ? [{ code: "free-memory-required", message: "Close other programs before installing.", freeBytes: 2899102924, requiredBytes: 13689304433 }] + : [] + : [{ code: "facts-unavailable", message: "nvidia-smi was not found" }]; // With the plan priced on capacity, the installer recommends a runtime. const recommendation = eligible && !lowMemory ? "local-models" : "codex"; const local = { @@ -122,7 +131,7 @@ if (args.includes("--inspect")) { state: lowMemory ? "blocked-now" : eligible ? "ready" : "unavailable", ...(lowMemory ? { basis: "capacity", memory: { ramAvailableBytes: 2899102924, ramRequiredBytes: 13689304433 } } : {}), }; - process.stdout.write(JSON.stringify({ hardware, localModels: { ...local, reasons, reasonMessages, estimate }, recommendation }) + "\n"); + process.stdout.write(JSON.stringify({ hardware, localModels: { ...local, reasons, estimate }, recommendation }) + "\n"); process.exit(0); } if (args.includes("--resume")) { @@ -134,7 +143,10 @@ if (args.includes("--resume")) { const repo = args[args.indexOf("--source-repo") + 1]; const servingFile = path.join(home, "resume-serving"); const source = fs.existsSync(servingFile) ? fs.readFileSync(servingFile, "utf8") : fs.existsSync(repo) ? fs.realpathSync(repo) : repo; - process.stdout.write("noise\n" + JSON.stringify({ code, message: code ? "Native teamhub stopped" : "ok", details: { source }, ...(runtime ? { runtime } : {}) }) + "\n"); + // resume-fails may hold the failed outcome's message and details. + const failure = code ? fs.readFileSync(path.join(home, "resume-fails"), "utf8").trim() : ""; + const outcome = failure ? JSON.parse(failure) : { message: code ? "Native teamhub stopped" : "ok", details: { source } }; + process.stdout.write("noise\n" + JSON.stringify({ code, ...outcome, ...(runtime ? { runtime } : {}) }) + "\n"); process.exit(code); } const origin = "http://127.0.0.1:4321"; @@ -1612,14 +1624,13 @@ describe("LocalServerModule", () => { expect(f.module.status().localModels).toMatchObject({ state: "checked", eligible: false, - reasons: [ - { code: "facts-unavailable", message: "nvidia-smi was not found" }, - ], + reasons: [{ code: "facts-unavailable" }], recommendation: "codex", }), ); expect(f.module.status().localModels).not.toHaveProperty("model"); - // The log says why, as the localmodels tool did. + // The page gets the code only; the tool's words go to the log. + expect(JSON.stringify(f.module.status())).not.toContain("nvidia-smi"); expect(info).toHaveBeenCalledWith( "local_server.inspected", expect.objectContaining({ @@ -1651,7 +1662,8 @@ describe("LocalServerModule", () => { reasons: [ { code: "free-memory-required", - message: "Close other programs before installing.", + freeBytes: 2899102924, + requiredBytes: 13689304433, }, ], recommendation: "codex", @@ -2318,6 +2330,40 @@ describe("LocalServerModule after a restart", () => { await f.host.stop("quit"); }); + it("stops the local models when the person quits, without waiting for them", async () => { + const stopped = (home: string) => path.join(home, "stop-local-models.json"); + const f = fixture({}, enrolled, record); + await f.host.start(); + await vi.waitFor(() => expect(f.module.status().phase).toBe("ready")); + await f.host.stop("quit"); + // It runs on after the app is gone; its outcome goes to the log. + await vi.waitFor(() => + expect( + readFileSync( + path.join(f.home, "installer", "local-models-stop.log"), + "utf8", + ), + ).toBe('{"failed":[]}\n'), + ); + expect(JSON.parse(readFileSync(stopped(f.home), "utf8"))).toEqual({ + args: ["--home", f.home, "--stop-local-models"], + runAsNode: "1", + }); + }); + + it("leaves the local models running for an update, and stops none without an installation", async () => { + const update = fixture({}, enrolled, record); + await update.host.start(); + await vi.waitFor(() => expect(update.module.status().phase).toBe("ready")); + await update.host.stop("update"); + const none = fixture(); + await none.host.start(); + await none.host.stop("quit"); + await delay(300); + for (const home of [update.home, none.home]) + expect(existsSync(path.join(home, "stop-local-models.json"))).toBe(false); + }); + it("moves the keys of the unpacked source's .env into the instance's user.env once", async () => { const f = fixture({}, enrolled, (home) => { const source = path.join(home, "desktop", "source", "previous"); @@ -2659,6 +2705,60 @@ describe("LocalServerModule after a restart", () => { await f.host.stop("quit"); }); + it.each([ + [ + "LOCAL_MODELS_RECOVERY_REQUIRED", + "Try Again to apply the approved local model plan again, or Set Up Again to review a new plan or choose another AI runtime.", + ], + [ + "LOCAL_MODELS_APPROVAL_REQUIRED", + "Set Up Again to review the local model plan or choose another AI runtime.", + ], + ])( + "names the action that works when the local model stops a restart (%s)", + async (code, action) => { + const f = fixture({}, enrolled, (home) => { + record(home); + writeFileSync( + path.join(home, "resume-fails"), + JSON.stringify({ + message: "The local model stack is not ready.", + details: { code }, + }), + ); + }); + await f.host.start(); + await vi.waitFor(() => expect(f.module.status().phase).toBe("failed")); + expect(f.showStartFailed).toHaveBeenCalledWith( + `The local model stack is not ready. ${action}`, + ); + await f.host.stop("quit"); + }, + ); + + it("says why the local model stack is not ready, from its reason code", async () => { + const f = fixture({}, enrolled, (home) => { + record(home); + writeFileSync( + path.join(home, "resume-fails"), + JSON.stringify({ + message: + "The upstream tool could not verify the local stack binding.", + details: { + code: "LOCAL_MODELS_RECOVERY_REQUIRED", + reasons: ["pi-extensions-unavailable", "gateway-unavailable"], + }, + }), + ); + }); + await f.host.start(); + await vi.waitFor(() => expect(f.module.status().phase).toBe("failed")); + expect(f.showStartFailed).toHaveBeenCalledWith( + "The local model's Pi can't load extensions, which every Linger run needs. The upstream tool could not verify the local stack binding. Try Again to apply the approved local model plan again, or Set Up Again to review a new plan or choose another AI runtime.", + ); + await f.host.stop("quit"); + }); + it("starts the services again when the person tries again", async () => { const f = fixture({}, enrolled, (home) => { record(home); @@ -2756,6 +2856,45 @@ describe("LocalServerModule after a restart", () => { await f.host.stop("quit"); }); + it("unpacks the source package again when its folder is damaged", async () => { + const root = tempDirectory(); + const archive = sourceArchive(root); + const repo = (home: string) => + path.join(home, "desktop", "source", archive.sha256.slice(0, 16)); + const f = fixture( + { + sourceArchive: { + url: "https://cdn.example/linger.tar.gz", + sha256: archive.sha256, + }, + fetch: (async () => new Response(archive.bytes)) as typeof fetch, + }, + enrolled, + (home) => { + // This app's own folder: the marker is there, package.json is gone. + mkdirSync(path.join(repo(home), "scripts"), { recursive: true }); + writeFileSync(path.join(repo(home), "scripts/source-native.mjs"), ""); + writeFileSync( + path.join(home, "desktop", "local-server.json"), + JSON.stringify({ + home, + repo: repo(home), + origin: "http://localhost:3000", + version: APP_VERSION, + }), + ); + }, + ); + await f.host.start(); + await vi.waitFor(() => expect(f.module.status().phase).toBe("ready")); + const args = installerArgs(f.home); + expect(args[args.indexOf("--source-repo") + 1]).toBe(repo(f.home)); + expect( + JSON.parse(readFileSync(path.join(repo(f.home), "package.json"), "utf8")), + ).toEqual({ name: "linger" }); + await f.host.stop("quit"); + }); + it("moves to the source package when the earlier Simplex app unpacked the one in use", async () => { const root = tempDirectory(); const archive = sourceArchive(root); diff --git a/apps/desktop/src/main/local-server.ts b/apps/desktop/src/main/local-server.ts index d69e59d1..da6cf2b3 100644 --- a/apps/desktop/src/main/local-server.ts +++ b/apps/desktop/src/main/local-server.ts @@ -1,6 +1,12 @@ import { spawn } from "node:child_process"; import { createHash, randomBytes, randomUUID } from "node:crypto"; -import { createReadStream, existsSync, readFileSync } from "node:fs"; +import { + closeSync, + createReadStream, + existsSync, + openSync, + readFileSync, +} from "node:fs"; import { constants as fsConstants, cp, @@ -59,7 +65,7 @@ import { nodeRuntimeDownloadUrls, } from "../../../../scripts/local-computer-node-distributions.mjs"; import type { Logger } from "./log.js"; -import type { Module, ModuleContext } from "./modules.js"; +import type { Module, ModuleContext, StopReason } from "./modules.js"; import { sameOrigin } from "./navigation.js"; import { humanTenant, type EnrollmentModule } from "./enrollment.js"; import { compareVersions } from "./update-manifest.js"; @@ -69,7 +75,11 @@ import { savedProviderKeys, type ProviderKeyChanges, } from "./provider-keys.js"; -import type { SetupByokCheck, SetupByokConfig } from "../shared/setup.js"; +import type { + SetupByokCheck, + SetupByokConfig, + SetupReason, +} from "../shared/setup.js"; // The pinned linger source this build installs, written by the release build: // downloaded from `url` (or a mirror of it), or bundled in the app at `file`. @@ -123,19 +133,14 @@ export type LocalModelsCheck = readonly gpu?: string; // Present when eligible or when it needs free memory. readonly model?: LocalModel; - // The localmodels tool's code and words for why not, or, when - // eligible, for what could still stop the install. + // The localmodels tool's code for why not, or, when eligible, for + // what could still stop the install, with the counts it carries. readonly reasons?: readonly SetupReason[]; // The localmodels tool's pick for this computer, which Quick Install // installs; the page adds no rule of its own. readonly recommendation?: SetupChoice; }; -export interface SetupReason { - readonly code: string; - readonly message?: string; -} - // A download under way: its bytes so far and in all, and the rate when the // source reports one. partial: the counts cover only the file downloading // now, so nothing says what remains. @@ -502,7 +507,7 @@ export class LocalServerModule implements Module { }); } - async stop(): Promise { + async stop(reason: StopReason): Promise { this.#offRetry?.(); this.#offReset?.(); this.#offDaemon?.(); @@ -511,6 +516,64 @@ export class LocalServerModule implements Module { // The installer manager keeps running a started installation; it exits // by itself once idle. this.#abort.abort(); + if (reason === "quit") await this.#stopLocalModels(); + } + + // The Computer exits with the app, so after a quit nothing uses the local + // models: the bundled installer unloads the model library's models and + // stops the stack with the gateway the localmodels tool started, which + // would otherwise keep their memory and ports until the next reboot. The + // services and the model controller keep running, as after every quit; + // the next launch brings the approved stack up again. The stop runs on + // after the app is gone, its output in the installer's folder. An update + // or an uninstall stops the models with the services instead. + async #stopLocalModels(): Promise { + if (!(await this.#readRecord()) || !existsSync(this.#installerScript())) + return; + const directory = path.join(this.#options.home, "installer"); + let output: number; + try { + await mkdir(directory, { recursive: true, mode: 0o700 }); + output = openSync( + path.join(directory, "local-models-stop.log"), + "w", + 0o600, + ); + } catch (error) { + this.#ctx?.log.warn("local_server.local_models_stop_failed", { + error: String(error), + }); + return; + } + try { + const child = spawn( + process.execPath, + [ + this.#installerScript(), + "--home", + this.#options.home, + "--stop-local-models", + ], + { + detached: true, + // A detached Windows process otherwise opens its own console window. + windowsHide: true, + stdio: ["ignore", output, output], + env: { ...process.env, ELECTRON_RUN_AS_NODE: "1" }, + }, + ); + child.once("error", (error) => + this.#ctx?.log.warn("local_server.local_models_stop_failed", { + error: String(error), + }), + ); + child.unref(); + this.#ctx?.log.info("local_server.local_models_stopping", { + home: this.#options.home, + }); + } finally { + closeSync(output); + } } status(): LocalServerStatus { @@ -600,7 +663,7 @@ export class LocalServerModule implements Module { this.#ctx?.bus.emit("local-server.changed"); }) .then( - (check) => { + ({ check, reasonLog }) => { this.#ctx?.log.info("local_server.inspected", { ms: Date.now() - started, eligible: check.state === "checked" && check.eligible, @@ -610,13 +673,7 @@ export class LocalServerModule implements Module { ...(check.state === "checked" && check.recommendation ? { recommendation: check.recommendation } : {}), - ...(check.state === "checked" && check.reasons - ? { - reasons: check.reasons.map(({ code, message }) => - message ? `${code}: ${message}` : code, - ), - } - : {}), + ...(reasonLog.length ? { reasons: reasonLog } : {}), }); this.#check = check; }, @@ -718,7 +775,11 @@ export class LocalServerModule implements Module { async #inspect( signal: AbortSignal, onDownload: (download: Transfer) => void, - ): Promise { + ): Promise<{ + check: LocalModelsCheck; + // The tool's codes with its words, for the log only. + reasonLog: string[]; + }> { // The app's own Node runs the check, so it needs no download of its own; // installing still uses the pinned Node runtime. const output = await new Promise((resolve, reject) => { @@ -776,7 +837,6 @@ export class LocalServerModule implements Module { state?: unknown; memory?: Record; reasons?: unknown; - reasonMessages?: unknown; estimate?: { model?: unknown; downloadBytes?: unknown; @@ -825,12 +885,26 @@ export class LocalServerModule implements Module { : {}), } : undefined; - const strings = (value: unknown) => - Array.isArray(value) ? value.map(String) : []; - const messages = strings(report.localModels.reasonMessages); - const reasons = strings(report.localModels.reasons).map( - (code, index): SetupReason => - messages[index] ? { code, message: messages[index] } : { code }, + const listed = Array.isArray(report.localModels.reasons) + ? (report.localModels.reasons as Record[]).filter( + (reason) => typeof reason?.code === "string", + ) + : []; + // The page words each reason by its code and counts, never the tool's. + const reasons = listed.map((reason): SetupReason => { + const counted = (key: "gpuIndex" | "freeBytes" | "requiredBytes") => + bytes(reason[key]) !== null ? { [key]: reason[key] as number } : {}; + return { + code: reason.code as string, + ...counted("gpuIndex"), + ...counted("freeBytes"), + ...counted("requiredBytes"), + }; + }); + const reasonLog = listed.map(({ code, message }) => + typeof message === "string" && message + ? `${code}: ${message}` + : `${code}`, ); const pick = report.recommendation; // The local model is recommended only with a model the page can show. @@ -840,7 +914,7 @@ export class LocalServerModule implements Module { (pick !== "local-models" || eligible) ? (pick as SetupChoice) : undefined; - return { + const check: LocalModelsCheck = { state: "checked", eligible, memoryBytes: hardware.totalMemoryBytes, @@ -864,6 +938,7 @@ export class LocalServerModule implements Module { ...(reasons.length ? { reasons } : {}), ...(recommendation ? { recommendation } : {}), }; + return { check, reasonLog }; } // Logs how long one part of setup took, so a slow setup says which part. @@ -1526,13 +1601,27 @@ export class LocalServerModule implements Module { archive.sha256.slice(0, 16), ); if (await isFile(path.join(directory, SOURCE_MARKER))) { - await this.#timed("teamhub_binaries", () => - this.#ensureTeamhubBinaries(directory, archive.sha256, signal), - ); - await this.#timed("native_runtime", () => - this.#ensureNativeRuntime(directory, archive.sha256, signal), - ); - return directory; + if ( + await validateSourceRepository(directory).then( + () => true, + () => false, + ) + ) { + await this.#timed("teamhub_binaries", () => + this.#ensureTeamhubBinaries(directory, archive.sha256, signal), + ); + await this.#timed("native_runtime", () => + this.#ensureNativeRuntime(directory, archive.sha256, signal), + ); + return directory; + } + // This package's own folder, damaged since it was unpacked (files + // removed by hand or by a cleanup): the installer would refuse it, so + // it is unpacked again in its place. + this.#ctx?.log.warn("local_server.source_damaged", { + source: path.basename(directory), + }); + await removeRetrying(directory); } if ("file" in archive) await this.#extract(archive, directory, SOURCE_MARKER, signal); @@ -2326,7 +2415,7 @@ export class LocalServerModule implements Module { const outcome = await new Promise<{ code: number; message: string; - details?: { source?: unknown }; + details?: { source?: unknown; code?: unknown; reasons?: unknown }; runtime?: InstanceRuntime; }>((resolve, reject) => { const child = spawn( @@ -2363,7 +2452,7 @@ export class LocalServerModule implements Module { JSON.parse(last) as { code: number; message: string; - details?: { source?: unknown }; + details?: { source?: unknown; code?: unknown; reasons?: unknown }; runtime?: InstanceRuntime; }, ); @@ -2372,8 +2461,29 @@ export class LocalServerModule implements Module { } }); }); - if (outcome.code !== 0) - throw new Error(outcome.message || "The installation stopped"); + if (outcome.code !== 0) { + const { code, reasons } = outcome.details ?? {}; + const action = + typeof code === "string" && Object.hasOwn(RESUME_ACTIONS, code) + ? RESUME_ACTIONS[code] + : undefined; + const reason = Array.isArray(reasons) + ? reasons.find( + (item): item is string => + typeof item === "string" && + Object.hasOwn(LOCAL_MODEL_NOT_READY, item), + ) + : undefined; + throw new Error( + [ + reason && LOCAL_MODEL_NOT_READY[reason], + outcome.message || "The installation stopped", + action, + ] + .filter(Boolean) + .join(" "), + ); + } this.#adoptRuntime(outcome.runtime); const source = outcome.details?.source; return typeof source === "string" ? source : undefined; @@ -2667,6 +2777,31 @@ const LOCAL_MODEL_FAILURES = new Set([ "GPU_SHARING_REVOCATION_FAILED", ]); +// What works from the start-failure page when the local model stops a +// restart: a restart applies the approved plan again, but only setup can +// approve a new one or switch the runtime. +const RESUME_ACTIONS: Record = { + LOCAL_MODELS_RECOVERY_REQUIRED: + "Try Again to apply the approved local model plan again, or Set Up Again to review a new plan or choose another AI runtime.", + LOCAL_MODELS_APPROVAL_REQUIRED: + "Set Up Again to review the local model plan or choose another AI runtime.", +}; + +// Why the local model stack was not ready, by the reason code localmodels' +// setup binding gives. +const LOCAL_MODEL_NOT_READY: Record = { + "pi-extensions-unavailable": + "The local model's Pi can't load extensions, which every Linger run needs.", + "deployment-unavailable": "The local model service isn't running.", + "acceptance-unavailable": "The local model stack hasn't passed its checks.", + "gateway-ownership": "The local model gateway isn't running.", + "gateway-unavailable": "The local model gateway isn't running.", + "pi-model-unavailable": "Pi can't reach the local model.", + "pi-config-changed": "Pi's settings changed while they were checked.", + "plan-invalid": "The approved local model plan can't be read.", + "binding-unavailable": "The local model stack couldn't be checked.", +}; + export interface Reconnect { readonly windowMs: number; readonly maxBackoffMs: number; diff --git a/apps/desktop/src/shared/setup.ts b/apps/desktop/src/shared/setup.ts index ef61ef8d..3ec2ccd4 100644 --- a/apps/desktop/src/shared/setup.ts +++ b/apps/desktop/src/shared/setup.ts @@ -123,6 +123,18 @@ export interface SetupTransfer { readonly partial?: boolean; } +// The localmodels tool's reason code with the counts it carries. Its words +// stay in the main process log; the page has its own copy per code. +export interface SetupReason { + readonly code: string; + // On gpu-memory-required: the card other programs hold memory on. + readonly gpuIndex?: number; + // The memory free now and what the install needs free: that card's VRAM, + // or RAM on free-memory-required. + readonly freeBytes?: number; + readonly requiredBytes?: number; +} + export type SetupLocalModelsCheck = | { readonly state: "unchecked" | "unavailable" } // While the check first downloads the localmodels tool: its bytes. @@ -147,12 +159,10 @@ export type SetupLocalModelsCheck = readonly diskBytes: number | null; readonly memoryBytes: number | null; }; - // The localmodels tool's code and words for why not, or, when - // eligible, for what could still stop the install. - readonly reasons?: readonly { - readonly code: string; - readonly message?: string; - }[]; + // The localmodels tool's code for why not, or, when eligible, for + // what could still stop the install, with the counts it carries. The + // page words each code itself. + readonly reasons?: readonly SetupReason[]; // The localmodels tool's pick, which Quick Install installs: // "local-models" is the model above served by Pi. readonly recommendation?: SetupChoice; diff --git a/apps/desktop/src/shell/setup-page.test.ts b/apps/desktop/src/shell/setup-page.test.ts index a837c6bc..fa4b1e35 100644 --- a/apps/desktop/src/shell/setup-page.test.ts +++ b/apps/desktop/src/shell/setup-page.test.ts @@ -9,6 +9,7 @@ import type { SetupCommand, SetupLocalAccount, SetupLocalModelsCheck, + SetupReason, SetupState, } from "../shared/setup.js"; @@ -203,12 +204,13 @@ const small: SetupLocalModelsCheck = { eligible: false, memoryBytes: 8 * GB, diskFreeBytes: 100 * GB, - // A code the page never parses: it shows only the message. - reasons: [ - { code: "upstream_policy", message: "No model fits this computer." }, - ], + reasons: [{ code: "plan-unavailable" }], recommendation: "codex", }; +// The tool's own words, which the page must never show. +const TOOL_WORDS = "GPU service owners unknown; explicit sharing"; +const reason = (fields: SetupReason): SetupReason => + ({ ...fields, message: TOOL_WORDS }) as SetupReason; // The model fits the hardware, but other programs hold its memory now. const held: SetupLocalModelsCheck = { state: "checked", @@ -222,13 +224,7 @@ const held: SetupLocalModelsCheck = { diskBytes: 8 * GB, memoryBytes: 12 * GB, }, - reasons: [ - { - code: "free-memory-required", - message: - "Other programs are using memory it needs (now 2.7 GB of RAM is available).", - }, - ], + reasons: [reason({ code: "free-memory-required" })], recommendation: "codex", }; const welcome = (localModels: SetupLocalModelsCheck): SetupState => { @@ -263,7 +259,7 @@ describe("setup page Quick Install recommendation", () => { "Recommended for this computer: Codex", ); expect($("plan-why").textContent).toBe( - "No local model: No model fits this computer.", + "No local model fits this computer.", ); expect($("local-row").hidden).toBe(true); $("local-install").click(); @@ -278,7 +274,7 @@ describe("setup page Quick Install recommendation", () => { "Recommended for this computer: Codex", ); expect($("plan-why").textContent).toBe( - "Local model on hold: Other programs are using memory it needs (now 2.7 GB of RAM is available). Quick Install sets up Codex now, or choose Check again once memory is free.", + "Local model on hold. Other programs are using the memory it needs. Close other programs. Quick Install sets up Codex now, or choose Check again once memory is free.", ); expect($("local-recheck").hidden).toBe(false); $("local-customize").click(); @@ -313,25 +309,40 @@ describe("setup page Quick Install recommendation", () => { }); it("says how much memory a held local model needs and how much is free", async () => { - const counted = ( - code: string, - message: string, - vram = false, - ): SetupLocalModelsCheck => ({ + const withCounts = (reasons: SetupReason[]): SetupLocalModelsCheck => ({ ...held, freeMemory: { ramAvailableBytes: 2899102924, ramRequiredBytes: 13689304433, - ...(vram ? { vramFreeBytes: 3 * GB, vramRequiredBytes: 9 * GB } : {}), + vramFreeBytes: 3 * GB, + vramRequiredBytes: 9 * GB, }, - reasons: [{ code, message }], + reasons, }); + const customizeLocal = () => { + $("local-customize").click(); + document + .querySelector( + '[data-plan="source"] [data-value="local"]', + )! + .click(); + }; + // The GPU and the RAM each name what is free and what the install needs. await openPage( welcome( - counted( - "free-memory-required", - "Other programs are using memory it needs.", - ), + withCounts([ + reason({ + code: "gpu-memory-required", + gpuIndex: 1, + freeBytes: 3 * GB, + requiredBytes: 9 * GB, + }), + reason({ + code: "free-memory-required", + freeBytes: 2899102924, + requiredBytes: 13689304433, + }), + ]), ), "welcome", ); @@ -339,31 +350,95 @@ describe("setup page Quick Install recommendation", () => { "Recommended for this computer: Codex", ); expect($("plan-why").textContent).toBe( - "Local model on hold: quit other programs first; it needs 12.7 GB of memory, 2.7 GB is free. Quick Install sets up Codex now, or choose Check again once memory is free.", + "Local model on hold. Other programs are using GPU 1's memory (3.0 GB free, 9.0 GB needed). Close programs that use the GPU, such as games, video or other AI apps. Other programs are using the memory it needs (2.7 GB free, 12.7 GB needed). Close other programs. Quick Install sets up Codex now, or choose Check again once memory is free.", ); expect($("local-recheck").hidden).toBe(false); + customizeLocal(); + expect($("local-sub").textContent).toBe( + "On hold: GPU 1 has 3.0 GB free, needs 9.0 GB", + ); + expect(document.body.textContent).not.toContain(TOOL_WORDS); + // A tool that counts only the plan's memory: the RAM counts from it. + await openPage( + welcome(withCounts([reason({ code: "free-memory-required" })])), + "welcome", + ); + expect($("plan-why").textContent).toBe( + "Local model on hold. Other programs are using the memory it needs (2.7 GB free, 12.7 GB needed). Close other programs. Quick Install sets up Codex now, or choose Check again once memory is free.", + ); + customizeLocal(); + expect($("local-sub").textContent).toBe( + "On hold: 2.7 GB of memory free, needs 12.7 GB", + ); + }); + + it("words every localmodels reason by its code, never the tool's message", async () => { + const cases: [SetupLocalModelsCheck, string][] = [ + [ + { ...held, reasons: [reason({ code: "gpu-sharing-required" })] }, + "Local model on hold. Other work is running on the GPU. Close it.", + ], + [ + { ...held, reasons: [reason({ code: "plan-unavailable" })] }, + "Local model on hold. Another app is holding the memory it needs, such as another AI app. Quit it.", + ], + [ + { ...small, reasons: [reason({ code: "gpu-ownership-unknown" })] }, + "Linger couldn't tell which programs are using the GPU, so it can't share it. Close programs that use the GPU.", + ], + [ + { ...small, reasons: [reason({ code: "facts-unavailable" })] }, + "Linger couldn't read this computer's memory, disk or GPU.", + ], + [ + { ...small, reasons: [reason({ code: "unsupported-platform" })] }, + "Local models don't run on this system.", + ], + [ + { ...small, reasons: [reason({ code: "a-new-code" })] }, + "This computer can't run a local model now.", + ], + ]; + for (const [check, words] of cases) { + await openPage(welcome(check), "welcome"); + expect($("plan-why").textContent).toContain(words); + expect(document.body.textContent).not.toContain(TOOL_WORDS); + } + }); + + it("shows the Local model card disabled with the reason when none can run", async () => { + await openPage( + welcome({ + ...small, + reasons: [reason({ code: "gpu-ownership-unknown" })], + }), + "welcome", + ); $("local-customize").click(); document .querySelector( '[data-plan="source"] [data-value="local"]', )! .click(); - expect($("local-sub").textContent).toBe( - "On hold: 12.7 GB memory needed, 2.7 GB free", + expect($("local-row").hidden).toBe(false); + expect($("local-model").hidden).toBe(false); + expect(($("local-model") as HTMLButtonElement).disabled).toBe(true); + expect($("local-model").getAttribute("aria-checked")).toBe("false"); + expect($("local-model-name").textContent).toBe( + "Not available on this computer", ); - // Another code names what holds the memory, so its words stay. - await openPage( - welcome( - counted( - "plan-unavailable", - "Another inference app holds this Mac's memory; quit it and check again.", - true, - ), - ), - "welcome", + expect($("local-model-needs").textContent).toBe( + "Linger couldn't tell which programs are using the GPU, so it can't share it. Close programs that use the GPU.", ); - expect($("plan-why").textContent).toBe( - "Local model on hold: quit other programs first; it needs 12.7 GB of memory, 2.7 GB is free; graphics memory needs 9.0 GB, 3.0 GB is free. Another inference app holds this Mac's memory; quit it and check again. Quick Install sets up Codex now, or choose Check again once memory is free.", + expect(($("local-install") as HTMLButtonElement).disabled).toBe(true); + expect(document.body.textContent).not.toContain(TOOL_WORDS); + }); + + it("says the stack's memory covers the model and its services", async () => { + await openPage(welcome(capable), "welcome"); + $("local-customize").click(); + expect($("local-model-needs").textContent).toBe( + "6 GB download · 8 GB disk · 12 GB memory with its services", ); }); @@ -605,6 +680,24 @@ describe("setup page Quick Install recommendation", () => { }); }); +describe("setup page permissions", () => { + it("names the menu bar on macOS and the system tray elsewhere", async () => { + const permissions = () => + document.querySelector('[data-step="permissions"]')!.textContent!; + for (const [platform, place] of [ + ["darwin", "menu bar"], + ["win32", "system tray"], + ["linux", "system tray"], + ] as const) { + await openPage({ ...welcome(capable), platform }, "welcome"); + const words = permissions().replace(/\s+/g, " "); + expect(words).toContain(`Change these later from the ${place} or`); + expect(words).toContain(`Start in the ${place} when you log in.`); + if (platform !== "darwin") expect(words).not.toContain("menu bar"); + } + }); +}); + describe("setup page download progress", () => { const installing = (transfer?: SetupState["localServer"]["transfer"]) => { const base = state("installing"); diff --git a/apps/desktop/src/shell/setup.html b/apps/desktop/src/shell/setup.html index dd13c3f1..109de925 100644 --- a/apps/desktop/src/shell/setup.html +++ b/apps/desktop/src/shell/setup.html @@ -2057,7 +2057,8 @@

Connect this computer

A few permissions

- Change these later from the menu bar or in your system settings. + Change these later from the menu bar or in your + system settings.

@@ -2162,7 +2163,10 @@

A few permissions

>
Open at loginStart in the menu bar when you log in. + >Start in the menu bar when you log + in.