Repository navigation
ci: cut setup, cache and test waiting from CI - #241
Conversation
97b171e to
eb517df
Compare
There was a problem hiding this comment.
Approve with nits (1 P2, 1 P3)
Scope: base main, merge base ddc03f14, 3 commits, 13 files, reviewed at eb517dfe. The coderbook skill was loaded. ponytail-review and karpathy-guidelines were not loaded; I applied the linger review checklist's equivalent items directly. This review ran in a separate session.
Findings
P2
apps/computer/local-daemon/internal/provider/protocol_probe.go:28-42copies an existing helper.probeTunable(newProbeTunable,get,set) matchescodexTunableinapps/computer/local-daemon/internal/provider/native/codex_process.go:33-46line for line; only the name differs. The repo's rule is to share a helper rather than copy it, and the two copies can now drift. Two simpler options:- If no probe goroutine outlives the test that lowered the budget, use plain package variables. The scaled tests run sequentially and restore the values in
t.Cleanup. This matches howclaudeTerminationGraceis lowered innative/claude_resident_unix_test.go:76. Example:var authProbeCap = 15 * time.Second, plus adiscoveryStageBudgetvariable set fromdefaultDiscoveryStageBudget(runtime.GOOS). - If the atomic is needed, move one tunable type into a small internal package and use it from both
providerandprovider/native.
- If no probe goroutine outlives the test that lowered the budget, use plain package variables. The scaled tests run sequentially and restore the values in
P3
- Commit
28b47929bundles four independent changes. It contains the node_modules cache, the shallow checkouts, the Go build-cache pruning and the 3→2 web shard change. Each can regress on its own; the shard count is a cost trade-off and the cache pruning is a separate cache-policy change. Becausemaintakes rebase merges, none of them can be reverted alone. Splitting the commit by feature would follow the repo's one-problem-per-commit convention.
Verified as fine
- Shallow checkouts.
check-packages,check-teamhub-cli,check-installer,check-desktop,check-contracts,check-computer-goandcheck-web-testsread neither BASE nor git history (Makefile:109-180,contracts-verify,build-daemon.mjs).repository-gate,web-static(React Doctor--base),changesandcheck (teamhub)(doc-comment lint) keepfetch-depth: 0. The setup action's early exit on shallow clones only skips the merge-base fallback. A job that starts using BASE would then fail on the unknown ref rather than compare HEAD with itself, as the comment says. - node_modules cache.
- The restore and save paths are identical, which matters because the cache version hashes them. They match
pnpm-workspace.yaml'sapps/*,packages/*andcontracts/generated/typescriptglobs. - The key covers the lockfile, workspace manifests, root
package.json(and so pnpm'spackageManager), patches,.npmrc, Node version, OS and arch. No restore-keys means a partial tree is never restored. - Only
repository-gatesaves, and only onpush. The workflow runs on pushes tomainonly, so pull requests cannot write an entry other jobs restore. - The save runs before
make check-repository-gate, so.viteand other tool caches stay out of the entry. setup-node's pnpm store is still used on a miss.pnpm install --frozen-lockfilestill runs on a hit.
- The restore and save paths are identical, which matters because the cache version hashes them. They match
- Go build cache pruning.
- The method holds: backdate everything by 2 h, run the check, then delete entries older than 60 min. Go refreshes an entry's mtime on
Get/OutputFileonly when it is older thanmtimeInterval(1 h), so the entries this run used are the only recent ones. - A pruned
-aentry or-doutput is a cache miss in Go, not a wrong result. Go recreates a deletedtrim.txtandREADME. - None of the touched targets overrides GOCACHE (checked
apps/computer/run-test.shandapps/teamhub/Makefile), so~/.cache/go-buildis the right directory. - Pruning runs only after
success(). A failed run saves the unpruned cache, and the next successful push prunes it.
- The method holds: backdate everything by 2 h, run the check, then delete entries older than 60 min. Go refreshes an entry's mtime on
- Production behavior unchanged.
boundedAuthTimeoutstill caps at 15 s.- The discovery stage budget is still 30 s on Linux and Windows and 15 s elsewhere.
TestProbeBudgetDefaultspins both values, including theboundedAuthTimeoutedge cases.- The scaled tests do not call
t.Parallel, so lowering the package-level values cannot leak into parallel tests.
- Test timing margins. I ran the touched tests with
-raceon a host at load average about 169 (208 cores), and they passed:go test -race -count=3 ./internal/provider -run 'TestProbeBudgetDefaults|TestCodexAuthenticationColdStartWithinLinuxBudget|TestDiscoverCodexColdInitializeWithinLinuxBudget|TestCodexAuthenticationFailureDiagnostics'passed in 18 s.go test -race -count=2 ./internal/provider/native -run 'TestClaude|TestCodexSteer'passed in 150 s.
desktop-release.test.ts. curl does not retry a missingfile://path, so the run still reaches thecannot downloadpath (scripts/ci/desktop-release.sh:151-152). The assertion still proves the folder name stays out of the output.- Docs. The shard count is updated in the
ci.ymlheader comment, theMakefilecomment andapps/web/AGENTS.md. Required check names are unchanged. - Unrelated changes. None found; every changed line traces to the stated goal of cutting CI time.
Process notes
- The branch is 5 commits behind
main.mainrequires up-to-date branches, so rebase before merging. - No merge commits and no fixup commits. The commit author is linked to the GitHub account
NightRain233.
CI
All checks pass on eb517dfe, including repository-gate, web-static, web tests (1/2), web tests (2/2), check (web), every check (<app>) job and CodeQL. CI runs the computer daemon's Go tests without -race; I ran the touched provider tests with -race myself (results above). I did not run scripts/ci/desktop-release.test.ts locally; CI covers it.
Automated review by Claude Code
Automated review of eb517df by Claude Code.
eb517df to
b415b4a
Compare
|
Addressed review 5968739656:
Also added since the review: shorter waits in the |
There was a problem hiding this comment.
Approve with nits (0 P0, 0 P1, 0 P2, 1 P3)
Scope: follow-up on eb517df, reviewing b415b4a against main (merge base 71b4243). 8 commits and 23 files in the PR; review focused on what changed since eb517df: the commit split, the probe budget rewrite, and the new commits 25c894fb (repository-gate ESLint dedupe) and b415b4aa (shorter native test graces). Loaded coderbook, ponytail-review and karpathy-guidelines. Ran in a separate session.
Previous findings
- P2
probeTunableduplicatedcodexTunable: Resolved. The type is gone.authProbeCap(apps/computer/local-daemon/internal/provider/protocol_probe.go:26) anddiscoveryStageBudget(discovery.go:110) are now plain package variables, following theclaudeTerminationGracepattern.scaleProbeBudgetsswaps them and restores them int.Cleanup. - P3 first commit mixed four independent changes: Resolved. It is now split into
ab8551a6(node_modules cache),a6f02938(shallow checkout),169bccb3(Go build cache pruning) andef813683(two web shards).
Findings
P3
apps/computer/local-daemon/internal/provider/native/pi_resident_test.go:423and:462:TestPiReleaseDuringParkProbeandTestPiSteerRacingParkIsRejectedstill sleep to reach the park probe. They do not wait on a signal. The margin pastpiSettleLaterDelaydrops from 300 ms to 200 ms, and the delay itself drops to 150 ms, so the total wait is now 350 ms. On a loaded runner where the fake's settle reaches the run more than 350 ms after turn start, two things can happen.TestPiSteerRacingParkIsRejectedunlockswriteMubefore the turn settles, the steer is applied, and the test fails withsteer = <nil>. The release test still passes but no longer exercises the park-probe race. The change saves about 0.4 s across these tests. Either keep the old margin or wait on an observable state, such as the turn's settled event, before taking the next step. This is optional: I ran both tests 3 times under-raceand they passed.
Verified as fine
-
The plain-variable probe budgets are safe as used. No test in
internal/providercallst.Parallel.go test -race -count=1 ./internal/provider/passes (21 s). -
ESLint dedupe:
check-changed-files.shskips ESLint only in gate mode, when a third argument is passed.--worktree(localmake check) still lints every changed file, and a new test pins this. All of the following lint with the single rooteslint.config.mjs, so skipping the gate's per-file pass loses no rule coverage:apps/web:srcande2eapps/desktop:srcandscriptsapps/teamhub-cliapps/linger-installer:srcpackages/document-files,packages/document-model,packages/knowledge-format,packages/service-kitpackages/local-installation:src
Each listed prefix's owning
lintscript starts witheslint --max-warnings=0, covers the directory, and runs from that app'smake check-<app>target (Makefile:117–180). The new tests pin both halves: check-plan selects a CI job for every prefix, and the owning lint script covers it. Files outside the prefixes, such asapps/web/vite.config.tsandscripts/*.mjs, are still linted by the gate, and the test asserts that too. -
Native test graces:
TestCodexInitializeCancelUnconfirmednow lowerscodexStopGracethrough the existing setter, whichnewCodexFake's cleanup restores. The other changes only shorten constants. The touched tests below pass under-race -count=3(42 s):TestPiReleaseDuringParkProbe,TestPiSteerRacingParkIsRejected,TestCodexInitializeCancelUnconfirmed,TestClaudeCallerCancelDuringTurn,TestCodexStartupStallKillsTreeandTestCodexNamesNewThreadAfterTurnStart. -
The PR description matches the diff. Production probe values are unchanged (15 s cap, 30 s Linux/Windows stage budget), and
TestProbeBudgetDefaultspins them per platform. The commit messages follow the conventions, and there are no merge or fixup commits.
Process notes
mainis 1 commit ahead of the merge base, so the branch needs a rebase before it can merge. GitHub reports the merge state as BLOCKED.
CI
- All checks pass, including
repository-gate,check (computer-go),web tests (1/2),web tests (2/2),web-staticand every app check. - CI does not run the daemon Go tests with
-race. I ran the-racetests listed above locally on the touched packages, and they passed.
Automated review by Claude Code
Automated review of b415b4a by Claude Code.
Every Node job restored the 443 MB pnpm store and spent about 10 s linking it. The installed tree (node_modules and each workspace package's) is cached on an exact key: lockfile, workspace manifests and settings, patches, Node version, OS and architecture. A hit skips the store, and the install only verifies the tree; a miss installs from the store as before. Only repository-gate saves the entry, on pushes to main, before any check writes into node_modules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
changes, repository-gate, web-static and check (teamhub) read BASE or git history; the other jobs fetch only the commit under test. The setup step that checks BASE's merge base skips shallow clones, so a job there that starts using BASE fails on the unknown ref instead of comparing HEAD with itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each Go job's cache kept five days of main pushes (teamhub over 1 GB, restored in about 25 s). On pushes the restored build cache is backdated by two hours; Go refreshes an entry's mtime on use once it is over an hour old, so after a successful run the entries still older than an hour went unused and are deleted before the save. The cache is content-addressed, so pruning can only cause misses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the web suite's CPU time dropped, three shards each spent more time in setup than in tests. Two shards cut the runner time per web change from about 216 s to 148 s; the slower shard's job (67-87 s) stays under the slowest other job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iting out curl retries The test pointed at an unresolvable host, so curl --retry 3 backed off for about 7 s before the script failed. A missing file:// folder fails the same download immediately and still proves the address stays out of the log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… out production budgets Three Linux-only provider tests slept through the real 15 s authentication cap and 30 s discovery stage budgets, about 46 s of the package's 55 s on Linux. The cap and the per-stage budget are package-level values that these tests lower tenfold, and a unit test pins the production defaults per platform. Native Runtime fakes wait less where the assertions are proportional. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…epository gate The gate linted every changed file, and for most of them the owning app's check lints the same directory in full in the same CI run, with --max-warnings=0. The gate now skips ESLint under the prefixes in scripts/ci/app-linted-paths.txt; prettier and the English-only check still cover every changed file. Tests pin that check-plan selects a CI job for each prefix and that the owning lint script covers it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…time tests Several native Runtime tests waited out 300 ms to 1 s stop graces and fake process delays that only have to outlast a signal or an in-process write. They now use 150-200 ms, which still leaves more than 100 ms wherever a process must react, and the codex initialize-cancel test lowers its 1 s fake grace through the existing setter. The Pi park-probe tests keep their sleeps: they reach the probe by time rather than by a signal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
b415b4a to
1cf211f
Compare
|
Addressed review 5968982010 (P3): |
There was a problem hiding this comment.
Approve (0 findings)
Follow-up on b415b4a. Scope: base main, merge base 63a918c4, 8 commits / 21 files in the PR; the incremental review covered the rebase plus the one changed commit (1cf211f4). Helper skills loaded: coderbook, ponytail-review, karpathy-guidelines. Ran in a separate session.
Previous findings
- P3: sleep-based Pi park-probe tests had their margin cut: Resolved.
git range-diffshows commits 1–7 unchanged after the rebase. Commit 8 no longer touches the Pi files:pi_resident_fake_test.go:129keepspiSettleLaterDelay = 200 * time.Millisecond, andpi_resident_test.go:423and:462keeppiSettleLaterDelay + 300*time.Millisecond. The commit message and PR body now say the Pi park-probe tests keep their original sleeps because they reach the probe by time rather than by a signal.
Findings
None.
Verified as fine
- Since the previous review, the only changes are the rebase onto current
main(git range-diff: commits 1–7 have identical patches) and the removal of the Pi sleep reductions from commit 8. Its commit message was updated to match. - The PR body's claim about the native tests ("The Pi park-probe tests keep their original sleeps") matches the diff.
Process notes
- No merge commits and no fixup commits. The branch is up to date with
main(0 commits behind).BLOCKEDmerge state reflects the pending required review, not a stale branch.
CI
All checks pass at 1cf211f4: repository-gate, web-static, web tests (1/2, 2/2), check (web, computer-go, contracts, desktop, installer, packages, teamhub, teamhub-cli), changes, CodeQL. Nothing was re-run locally. The only new change restores the original test timings, so no new concurrency risk needed a -race run.
Automated review by Claude Code
Automated review of 1cf211f by Claude Code.
Cuts CI time by removing work: dependency setup, oversized Go caches and real waits in tests. Parallelism and checks are unchanged, except that the web tests run as 2 shards instead of 3.
Changes
.github/actions/setup). Every Node job restored the 443 MB pnpm store (~10 s) and then spent ~10 s linking it. The installed tree (node_modulesplus each workspace package's) is now cached under an exact key: lockfile,pnpm-workspace.yaml,.npmrc, patches, every workspacepackage.json, Node version, OS and arch. There are no restore-keys. On a hit no store is restored.pnpm install --frozen-lockfilestill runs and prints "Already up to date"; pnpm's supply-chain policy check (~5 s) still runs too. On a miss the job installs from the store as before. Onlyrepository-gatesaves the entry, on pushes to main, right after install and before any check writes intonode_modules.changes,repository-gate,web-staticandcheck (teamhub)keepfetch-depth: 0. The other jobs fetch only the commit under test. None of their make targets reads BASE or git history. The setup step that checks BASE's merge base skips shallow clones. Anything there that starts using BASE then fails on the unknown ref instead of comparing HEAD with itself..github/actions/check). Each Go job's cache accumulated five days of main pushes: teamhub 1075 MB, desktop 661 MB, computer-go 444 MB. Their copies added up to more than the repository's 10 GB cache limit. On pushes the job backdates the restored build cache by 2 h. Go refreshes an entry's mtime on use only once it is over an hour old. After a successful run, entries this run did not touch are deleted. The cache is content-addressed, so pruning can only cause misses. teamhub drops to 632 MB (~25 s → ~6 s restore) and computer-go to 219 MB.internal/providerGo tests: 55 s → 15 s on Linux. Three Linux-only tests slept through the real 15 s authentication cap and 30 s per-stage discovery budget. Both are now plain package variables, likeclaudeTerminationGracein native, that these tests lower tenfold and restore int.Cleanup. They never run in parallel. A unit test pins the production defaults per platform. Production values are unchanged.provider/nativeGo tests: ~30 s → ~21 s. Stop graces and fake process delays that only have to outlast a signal or an in-process write now run at 150–200 ms, still more than 100 ms wherever a process must react. The Pi park-probe tests keep their original sleeps because they reach the probe by time rather than by a signal. Stress runs in a Linux container passed:-count=20at 2 CPUs and-count=10at 1 CPU.scripts/ci/app-linted-paths.txt. The owning app's CI check lints each of those directories in full with--max-warnings=0whenever a path under it changes. Prettier and the English-only check still cover every changed file. Tests pin that check-plan selects a CI job for each prefix, and that the owninglintscript lists the directory with--max-warnings=0.desktop-releasetest: 7 s → 0.1 s. It waited outcurl --retry 3backoff on an unresolvable host. A missingfile://folder fails immediately and still proves the address stays out of the log.Measurements
Baseline: main push run 44441997483, after #227. "After": this PR's run 44447958702, every cache hit. During measurement a temporary commit, since dropped, let pull requests save the caches; normally only main pushes save. Job wall time in seconds; runner noise between runs is about ±15 s.
Per Node job, setup drops from ~27 s to ~18 s. That is ~6 s for the node_modules restore plus ~5 s for the install, which is almost all pnpm's supply-chain policy check.
Not done / follow-ups
pnpm installon a cache hit would save another ~5 s per job, but pnpm'sverifyDepsBeforeRunthen fails with "Patches were modified", because checkout gives the patch files new mtimes. The check stays on.web-staticis bound by ESLint (~50 s, typed rules;no-misused-promisesalone is ~60% of rule time). ESLint's cache is unsafe with type-aware rules, so it stays uncached.TestRuntimeMonitorPublishesCatalogFailureImmediatelyininternal/providerfails under-raceon 1 CPU onmaintoo, unchanged by this PR. CI does not run-race.Required checks
No required check is renamed:
repository-gate,check (web)andcheck (<app>)are unchanged. The shard jobs becomeweb tests (1/2)andweb tests (2/2). They feedcheck (web)and are not required checks themselves.User wait path
Does not touch the user message →
provider_readypath.🤖 Generated with Claude Code