Skip to content

fix(build): pin GOTOOLCHAIN to go.mod so local and CI share a toolchain #580

Description

@EricAndrechek

Summary

go.mod says go 1.26.6, but nothing pins GOTOOLCHAIN. Go's default is GOTOOLCHAIN=auto, which does not mean "use go.mod's version" — it means use the local toolchain unless go.mod requires a newer one, i.e. effectively max(local, go.mod). So a contributor on a newer Go silently builds, tests and lints on a different toolchain than CI.

That is the case right now:

Go used
CI (Verify Go resolves go.mod's toolchain, run 34353998581) go1.26.6 linux/amd64
Local dev machine go1.27.1 darwin/arm64

make ci passing locally therefore does not attest to what CI runs.

Current symptom

make lint-go fails on any machine whose Go is newer than go.mod's:

✗ golangci-lint
    panic: file requires newer Go version go1.27 (application built with go1.26) [recovered, repanicked]
    go/types.(*Checker).handleBailout(...)
    github.com/golangci/golangci-lint/v2/pkg/goanalysis.(*loadingPackage).convertError(...)

golangci-lint v2.11.4 (Makefile:163) embeds a go/types built with go1.26; when GOTOOLCHAIN=auto hands it go1.27-processed packages, its type checker bails out on every package. GOTOOLCHAIN=go1.26.6 make lint-go passes clean, which confirms the toolchain is the whole story — there is no lint finding underneath it.

Today this is worked around by hand (GOTOOLCHAIN=go1.26.6 make ci, and the same prefix on git commit so the pre-commit hook passes). That is undocumented tribal knowledge, and it is easy to forget precisely when it matters.

The part that makes this more than an annoyance

.github/actions/setup-env/action.yml:181-190 deliberately drops actions/setup-go and relies on the runner image's Go, with this rationale:

The image's preinstalled go + GOTOOLCHAIN=auto (the Go ≥1.21 default) resolve go.mod's pinned version exactly

That premise is only true while the runner image's Go is older than or equal to go.mod's directive. It holds today (CI resolves to exactly 1.26.6, so the image ships ≤ 1.26.6). The moment GitHub's ubuntu-latest image ships Go 1.27, auto will prefer the image's newer toolchain, and every Go job in CI hits the panic above at once — Lint, Unit, Integration, E2E, Coverage — with no repo change to point at.

So this is a latent CI outage on an external image-update schedule we do not control, not just laptop friction.

Proposed fix

Derive the toolchain from go.mod and export it, so there is one source of truth and it self-maintains across future go directive bumps:

# Pin the toolchain to go.mod's directive. GOTOOLCHAIN=auto means
# max(local, go.mod), so a dev (or a future runner image) on a newer Go
# silently diverges from CI — and golangci-lint, built against go.mod's
# Go, panics on packages processed by a newer one.
GO_VERSION         := $(shell awk '/^go /{print $$2; exit}' go.mod)
export GOTOOLCHAIN := go$(GO_VERSION)

Exporting from the Makefile covers everything that matters, because every Go entry point goes through make:

  • all Makefile recipes (lint-go, test-*, build, …);
  • scripts/build.sh (go build), scripts/dep-cut.sh and scripts/size.sh (go tool …) — all invoked from make targets, so they inherit it;
  • .githooks/pre-commit and .githooks/pre-push, which shell out to make verify / make ci.

This matches the existing export VERSION_LDFLAGS LDFLAGS TAGS / export COV_DEFER precedent at Makefile:214-215,745.

The setup-env comment at .github/actions/setup-env/action.yml:181-190 should be corrected in the same change — it currently documents auto as doing something it does not.

Trade-offs worth deciding explicitly

  • It pins downward as well as upward. A dev on Go 1.27 stops exercising 1.27. That is the intent (match CI), but it means we lose incidental early warning of new-Go incompatibilities. The deliberate way to get that signal is a go.mod bump, which is reviewable.
  • One-time toolchain download (~80 MB) for anyone whose local Go differs, cached in the module cache. CI already caches this (gomod-v1-* key includes go.mod precisely for this reason — .github/workflows/README.md:77).
  • A bare go test ./... typed straight into a terminal is still unpinned. The Makefile export cannot reach that. Note toolchain go1.26.6 in go.mod does not fix it either — the toolchain directive is a floor under auto, not a ceiling. Fully closing that gap would need something like a committed .envrc, which is a bigger call; the make-level pin is the pragmatic 90%.
  • New coupling to record: bumping go.mod's go directive now also requires a golangci-lint release built with at least that Go, or lint-go panics in CI instead of locally. Worth a line in development.md.

Related

Found while merging the September dependency PRs, where the pre-commit hook failed on lint-go for purely environmental reasons.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions