Skip to content

feat(desktop): reach the update feed from mainland China through npm - #224

Closed
oldwinter53 wants to merge 1 commit into
mainfrom
ws/whi-753-updater
Closed

oldwinter53 wants to merge 1 commit into
mainfrom
ws/whi-753-updater

Conversation

@oldwinter53

Copy link
Copy Markdown

Outcome

WHI-753 desktop updater reachable from mainland China

Scope

Desktop updater gets a China source: publish the signed manifest (manifest.json + .sig) as npm package @simplexai/linger-desktop-update and each platform installer split into <=150 MB parts as npm packages; updater in cn reads the manifest and parts from cdn.npmmirror.com tarballs (fallback npmjs, then GitHub Releases), reassembles, verifies the existing Ed25519 manifest signature and per-package sha256. Release script gains the npm publish step (not run). global path unchanged. Out of scope: changing signing keys or channel semantics.

Acceptance criteria

  • In cn the updater fetches the manifest from npmmirror and verifies the signature; installer parts download from npmmirror and the reassembled file matches the manifest sha256
  • In global the updater behaves exactly as today (GitHub first)
  • Unit tests cover part reassembly, tampered part rejection, fallback to GitHub, and the release script's split/pack step
  • make check BASE=origin/main passes apart from pre-existing failures

Validation

  • none recorded

No checker run.

Task whi-753-updater, opened by the workspace control plane.

Stacked on ws/whi-752-region: merge that first; until then this MR also shows its commits.

Refs https://linear.app/linger1024/issue/WHI-753

@linger-review linger-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes (1 P1, 2 P2)

Scope: base main, merge-base adb58a07. The PR has 2 commits and 45 files. It is stacked on #213, so c2a9407f (region resolution) is reviewed there. This review covers 32c6541e: 15 files, the updater npm feed and release packing. Helper skills loaded: coderbook, ponytail-review, karpathy-guidelines. The review ran in a separate session.

Findings

P1

1. The npmmirror CDN tarball rule is copied instead of reused (apps/desktop/src/main/update-npm.ts:38)

  • packages/local-installation/src/npm-registry.ts:5 already has npmTarballUrl(registry, name, version). It holds the same rule: the mirror maps to https://cdn.npmmirror.com/packages/<name>/<version>/<name>-<version>.tgz, and any other registry maps to <registry><name>/-/<name>-<version>.tgz.
  • The new copy differs only in using the unscoped basename. Scoped packages such as @simplexai/... need that.
  • This leaves two copies of one rule about a third-party URL layout. When npmmirror changes its CDN path, or someone fixes one copy, the other copy breaks silently. If the updater's copy is the stale one, China installs stop finding updates with no visible error.
  • Fix: make the existing helper use name.slice(name.lastIndexOf("/") + 1) for the file name. That gives the same result for unscoped names. Then import it in update-npm.ts and delete the copy. If the updater wants a URL, wrap the result in new URL(...) at the call site.

P2

2. A hand-written tar parser duplicates a dependency already in the bundle (apps/desktop/src/main/update-npm.ts:56-172)

  • ByteReader, the header checksum, octal parsing and the entry walk add about 110 lines of security-sensitive parsing that must be maintained.
  • tar@7.5.22 is already a workspace dependency. localmodels-tool.ts and native-dependencies.ts use it, and the desktop main process already imports both, so it is already in the desktop bundle.
  • tar's Parser with onReadEntry can stream each entry into the hash and file write. It can also reject any entry whose type !== "File", whose path is not in wanted, whose size is over the limit, or that repeats. That keeps the same strictness, and the tests in update-npm.test.ts can stay as they are.
  • Fix: replace the parser with tar, adding it to apps/desktop/package.json, or explain in a comment why the library can't be used.

3. The npm-pack tests use a shared /tmp/.npm cache (scripts/ci/desktop-release.test.ts:483, :529, :537)

  • run() sets HOME: tmpdir(), so npm pack in desktop-npm-pack.mjs writes its cache to /tmp/.npm.
  • On a shared host, that directory belongs to whoever ran the test first. Every other user then gets npm error code EACCES … path /tmp/.npm/_cacache/tmp/….
  • I reproduced this here: both packs the manifest and every part the way installed apps read them and refuses a package that changed after the manifest was signed fail, because /tmp/.npm is owned by another user. The Makefile requires scripts/ci tests to "use private temp dirs".
  • Fix: pass HOME: folder(), or npm_config_cache: join(folder(), "cache"), to these run([...npm-pack...]) calls.

Verified as fine

  • Trust chain: every npm manifest goes through verifyManifest with the built-in key. #npmManifest rejects a signed manifest whose version differs from its dist-tag, so an old manifest under a newer tag is refused. Each part is checked for size and sha256 against the signed manifest, and the joined file is checked against the whole-package sha256 through the existing #verify. The size of each tar member is capped by the manifest.
  • Global path: a configured feed skips npm. In global, the release feed is still asked first, and npm is used only when the release throws a non-ManifestError error. A release with no manifest still returns quietly without trying npm.
  • cn path: mirror lag falls through to npmjs and then to the GitHub release. A release manifest that npm lacks is downloaded from the release, never from parts (fromNpm). A 404 (UpdateNotFound) does not count as a regional failure, so it does not flip the home's region.
  • Publish order: desktop-npm-pack.mjs writes the parts before the manifest package, so a manifest on npm means its parts were published. A retried job skips versions npm already has. A prerelease is tagged next, and the updater ignores prerelease latest tags.
  • Disk use: each piece is deleted as it is joined, and pieces are removed on failure.
  • Manifest schema: parts is optional and the schema is non-strict, so older apps ignore it.
  • Docs: docs/desktop-release.md documents the feed, NPM_TOKEN and DESKTOP_NPM_PART_BYTES.

Process notes

  • mergeStateStatus is DIRTY: the branch conflicts with main and needs a rebase.
  • This PR is stacked on #213, which must merge first.
  • The PR body says "make check … passes", but its Validation section says no checker was run. Please run make check BASE=origin/main after the rebase.
  • No merge commits or fixup commits.

CI

  • Only Analyze (actions) and CodeQL ran, and both passed. Because of the conflict, the main CI did not run on this head, so there is no CI evidence for the change.
  • What I ran locally:
    • apps/desktop typecheck: passed.
    • vitest on update-npm.test.ts, updater.test.ts and update-manifest.test.ts: 49/49 passed.
    • scripts/ci/desktop-release.test.ts -t npm: 6 passed, 2 failed with EACCES on the shared /tmp/.npm (finding 3).

Automated review by Claude Code


Automated review of 32c6541 by Claude Code.

@linger-review linger-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes (1 P1 and 2 P2 carried over from the previous review; no new findings)

Scope: follow-up on 32c6541. Reviewed head a02cfbb against main (merge base 679bb2a): 2 commits and 45 files. This push is a pure rebase onto newer main. I checked git range-diff adb58a0..32c6541 679bb2a..HEAD, and the only changes are in surrounding context that came from main: CHECK_INTERVAL_MS, the install-scripts subcommand, and docs table width. The PR's own code is unchanged, so there was nothing new to review. Skills: the review followed the linger-pr-review procedure; I did not load coderbook, ponytail-review or karpathy-guidelines separately. Ran in a separate session.

Previous findings

  • P1: npmmirror CDN URL rule duplicated. Not resolved. apps/desktop/src/main/update-npm.ts is byte-identical to 32c6541. npmTarballUrl at update-npm.ts:38 still hand-builds https://cdn.npmmirror.com/packages/..., a second copy of the rule in packages/local-installation/src/npm-registry.ts:5. Fix: make the shared helper handle scoped names (basename after the scope) and use it from the desktop updater.
  • P2: hand-written tar parser instead of the existing tar dependency. Not resolved. extractNpmTarball at update-npm.ts:126 is unchanged.
  • P2: npm-pack tests share /tmp/.npm. Not resolved. scripts/ci/desktop-release.test.ts:31 still runs with HOME: tmpdir(), so npm pack writes its cache to the shared /tmp/.npm. On a shared host where another user owns that directory, the test fails with EACCES. Fix: pass a per-test HOME or npm_config_cache under the test's mkdtemp dir.

Findings

None new. The code is unchanged since the previous review.

Verified as fine

  • The rebase onto main carried no conflict fallout into the PR's own hunks. range-diff shows only context lines that came from main.
  • No merge commits and no fixup commits.

Process notes

  • The PR is stacked on ws/whi-752-region. Commit 7ce523e3 belongs to that branch's PR, which should merge first.
  • GitHub reports the merge state as BLOCKED.

CI

All 16 checks pass, including check (desktop), check (packages), check (installer) and repository-gate. I ran no tests myself; with no code changes there was no new suspicion to test.

Automated review by Claude Code


Automated review of a02cfbb by Claude Code.

@oldwinter53

Copy link
Copy Markdown
Author

Closing this for the 0.1.4 release; it can be reopened later.

  • It rewrites about 400 lines of the updater and changes how the release manifest is built. None of it has been run on a real machine yet, and 0.1.4 would be its first real run. A broken updater cannot be fixed by shipping another release.
  • It brings no benefit in 0.1.4. There is no NPM_TOKEN and no npm package yet, so the publish step would be skipped, and mainland clients would make two requests that return 404 before falling back to GitHub.

Reopen once npm publishing is set up and the updater path has been checked on real machines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants