Skip to content

ci: stop recompiling the TypeScript compiler in every CI job - #93

Merged
alecthomas merged 4 commits into
mainfrom
ci-speed-investigation
Nov 2, 2026
Merged

alecthomas merged 4 commits into
mainfrom
ci-speed-investigation

Conversation

@alecthomas

@alecthomas alecthomas commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

The CI test and lint jobs spent most of their time compiling the TypeScript compiler, often more than once per job, and always from scratch.

The repeated build came from the TypeScript shim, a separate Go module that required older versions of x/sys, x/sync, x/text and cpuid than the root module selects. Different dependency versions give the compiler's packages different build cache keys, so each module compiled them separately. Locally, building the shim after the root module recompiled 35 TypeScript compiler packages before this change and none after it. On CI, the shim's build step dropped from about 3 minutes to under a second, and its test step from about 2 minutes to 10 seconds.

The root module is the source of truth, because it already selects versions at least as new as the shim's. Three things keep the shim in line with it:

  • bit lint-go-modules, now part of bit lint, fails when the shim's required dependencies differ from the root module's versions.
  • A pre-commit hook runs bit sync-go-modules whenever a go.mod or go.sum is staged, and stages the shim's updated files.
  • Renovate now updates the shim's indirect dependencies, and gomodTidyAll tidies the root module in the same PR.

Renovate still misses one case. A root dependency upgrade that raises a shared dependency as a side effect leaves the shim behind, and the lint fails on that PR until someone runs bit sync-go-modules on the branch.

The test and lint jobs now also cache Go's build and module caches and tsk's lint results. Only pushes to main save the cache, and every other branch restores main's latest entry, so branches cannot evict it with entries nothing else reads. The cache only helps runs after this is merged and main has saved it once.

Whole-job times vary by up to about 2x between runners, so compare job times over several runs rather than one.

A temporary go.work file in CI was also considered. Workspace mode also upgraded the root module's own dependencies (google/pprof), so CI would have tested different versions than the released binaries.

🤖 Generated with Claude Code

alecthomas and others added 2 commits November 2, 2026 15:16
…y versions

The shim module required older versions of dependencies it shares with the
root module, so CI compiled the TypeScript compiler once per module. That
added two to three minutes to the test and lint jobs.

A lint check now fails when the versions differ, a pre-commit hook brings
the shim in line, and Renovate updates the shim's indirect dependencies and
tidies the root module with them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sample server compiled the TypeScript compiler at the same time as
the typescript-compiler target, so the test job compiled it twice on the
same CPUs and took about twice as long to get past that step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alecthomas alecthomas changed the title build: keep the TypeScript shim module on the root module's dependency versions build: stop compiling the TypeScript compiler more than once in CI Nov 2, 2026
@alecthomas
alecthomas marked this pull request as ready for review November 2, 2026 04:24
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T04:26:14.739012Z ff14a40 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff14a4004a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/go-module-sync Resolved
Every CI run compiled everything from scratch, including the TypeScript
compiler, which took minutes of each test and lint run. Only main saves the
cache, so branches reuse main's latest build without evicting it.

This also reverts building the sample server after the TypeScript compiler.
CI showed no gain from it: the sample server's other packages had been
compiling in parallel and moved onto the critical path instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alecthomas alecthomas changed the title build: stop compiling the TypeScript compiler more than once in CI ci: stop recompiling the TypeScript compiler in every CI job Nov 2, 2026
A failed go command produced an empty version list, which the script read
as "no differences". The lint check then passed and the pre-commit sync
reported success without comparing or changing anything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alecthomas
alecthomas merged commit 1f7f4c5 into main Nov 2, 2026
6 checks passed
@alecthomas
alecthomas deleted the ci-speed-investigation branch November 2, 2026 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant