Summary
go.mod says go 1.26.6, but nothing pins GOTOOLCHAIN. Go's default is GOTOOLCHAIN=auto, which does not mean "use go.mod's version" — it means use the local toolchain unless go.mod requires a newer one, i.e. effectively max(local, go.mod). So a contributor on a newer Go silently builds, tests and lints on a different toolchain than CI.
That is the case right now:
|
Go used |
CI (Verify Go resolves go.mod's toolchain, run 34353998581) |
go1.26.6 linux/amd64 |
| Local dev machine |
go1.27.1 darwin/arm64 |
make ci passing locally therefore does not attest to what CI runs.
Current symptom
make lint-go fails on any machine whose Go is newer than go.mod's:
✗ golangci-lint
panic: file requires newer Go version go1.27 (application built with go1.26) [recovered, repanicked]
go/types.(*Checker).handleBailout(...)
github.com/golangci/golangci-lint/v2/pkg/goanalysis.(*loadingPackage).convertError(...)
golangci-lint v2.11.4 (Makefile:163) embeds a go/types built with go1.26; when GOTOOLCHAIN=auto hands it go1.27-processed packages, its type checker bails out on every package. GOTOOLCHAIN=go1.26.6 make lint-go passes clean, which confirms the toolchain is the whole story — there is no lint finding underneath it.
Today this is worked around by hand (GOTOOLCHAIN=go1.26.6 make ci, and the same prefix on git commit so the pre-commit hook passes). That is undocumented tribal knowledge, and it is easy to forget precisely when it matters.
The part that makes this more than an annoyance
.github/actions/setup-env/action.yml:181-190 deliberately drops actions/setup-go and relies on the runner image's Go, with this rationale:
The image's preinstalled go + GOTOOLCHAIN=auto (the Go ≥1.21 default) resolve go.mod's pinned version exactly
That premise is only true while the runner image's Go is older than or equal to go.mod's directive. It holds today (CI resolves to exactly 1.26.6, so the image ships ≤ 1.26.6). The moment GitHub's ubuntu-latest image ships Go 1.27, auto will prefer the image's newer toolchain, and every Go job in CI hits the panic above at once — Lint, Unit, Integration, E2E, Coverage — with no repo change to point at.
So this is a latent CI outage on an external image-update schedule we do not control, not just laptop friction.
Proposed fix
Derive the toolchain from go.mod and export it, so there is one source of truth and it self-maintains across future go directive bumps:
# Pin the toolchain to go.mod's directive. GOTOOLCHAIN=auto means
# max(local, go.mod), so a dev (or a future runner image) on a newer Go
# silently diverges from CI — and golangci-lint, built against go.mod's
# Go, panics on packages processed by a newer one.
GO_VERSION := $(shell awk '/^go /{print $$2; exit}' go.mod)
export GOTOOLCHAIN := go$(GO_VERSION)
Exporting from the Makefile covers everything that matters, because every Go entry point goes through make:
- all Makefile recipes (
lint-go, test-*, build, …);
scripts/build.sh (go build), scripts/dep-cut.sh and scripts/size.sh (go tool …) — all invoked from make targets, so they inherit it;
.githooks/pre-commit and .githooks/pre-push, which shell out to make verify / make ci.
This matches the existing export VERSION_LDFLAGS LDFLAGS TAGS / export COV_DEFER precedent at Makefile:214-215,745.
The setup-env comment at .github/actions/setup-env/action.yml:181-190 should be corrected in the same change — it currently documents auto as doing something it does not.
Trade-offs worth deciding explicitly
- It pins downward as well as upward. A dev on Go 1.27 stops exercising 1.27. That is the intent (match CI), but it means we lose incidental early warning of new-Go incompatibilities. The deliberate way to get that signal is a
go.mod bump, which is reviewable.
- One-time toolchain download (~80 MB) for anyone whose local Go differs, cached in the module cache. CI already caches this (
gomod-v1-* key includes go.mod precisely for this reason — .github/workflows/README.md:77).
- A bare
go test ./... typed straight into a terminal is still unpinned. The Makefile export cannot reach that. Note toolchain go1.26.6 in go.mod does not fix it either — the toolchain directive is a floor under auto, not a ceiling. Fully closing that gap would need something like a committed .envrc, which is a bigger call; the make-level pin is the pragmatic 90%.
- New coupling to record: bumping
go.mod's go directive now also requires a golangci-lint release built with at least that Go, or lint-go panics in CI instead of locally. Worth a line in development.md.
Related
Found while merging the September dependency PRs, where the pre-commit hook failed on lint-go for purely environmental reasons.
Summary
go.modsaysgo 1.26.6, but nothing pinsGOTOOLCHAIN. Go's default isGOTOOLCHAIN=auto, which does not mean "use go.mod's version" — it means use the local toolchain unless go.mod requires a newer one, i.e. effectivelymax(local, go.mod). So a contributor on a newer Go silently builds, tests and lints on a different toolchain than CI.That is the case right now:
Verify Go resolves go.mod's toolchain, run 34353998581)go1.26.6 linux/amd64go1.27.1 darwin/arm64make cipassing locally therefore does not attest to what CI runs.Current symptom
make lint-gofails on any machine whose Go is newer thango.mod's:golangci-lint
v2.11.4(Makefile:163) embeds ago/typesbuilt with go1.26; whenGOTOOLCHAIN=autohands it go1.27-processed packages, its type checker bails out on every package.GOTOOLCHAIN=go1.26.6 make lint-gopasses clean, which confirms the toolchain is the whole story — there is no lint finding underneath it.Today this is worked around by hand (
GOTOOLCHAIN=go1.26.6 make ci, and the same prefix ongit commitso the pre-commit hook passes). That is undocumented tribal knowledge, and it is easy to forget precisely when it matters.The part that makes this more than an annoyance
.github/actions/setup-env/action.yml:181-190deliberately dropsactions/setup-goand relies on the runner image's Go, with this rationale:That premise is only true while the runner image's Go is older than or equal to
go.mod's directive. It holds today (CI resolves to exactly 1.26.6, so the image ships ≤ 1.26.6). The moment GitHub'subuntu-latestimage ships Go 1.27,autowill prefer the image's newer toolchain, and every Go job in CI hits the panic above at once — Lint, Unit, Integration, E2E, Coverage — with no repo change to point at.So this is a latent CI outage on an external image-update schedule we do not control, not just laptop friction.
Proposed fix
Derive the toolchain from
go.modand export it, so there is one source of truth and it self-maintains across futuregodirective bumps:Exporting from the Makefile covers everything that matters, because every Go entry point goes through
make:lint-go,test-*,build, …);scripts/build.sh(go build),scripts/dep-cut.shandscripts/size.sh(go tool …) — all invoked from make targets, so they inherit it;.githooks/pre-commitand.githooks/pre-push, which shell out tomake verify/make ci.This matches the existing
export VERSION_LDFLAGS LDFLAGS TAGS/export COV_DEFERprecedent atMakefile:214-215,745.The
setup-envcomment at.github/actions/setup-env/action.yml:181-190should be corrected in the same change — it currently documentsautoas doing something it does not.Trade-offs worth deciding explicitly
go.modbump, which is reviewable.gomod-v1-*key includesgo.modprecisely for this reason —.github/workflows/README.md:77).go test ./...typed straight into a terminal is still unpinned. The Makefile export cannot reach that. Notetoolchain go1.26.6ingo.moddoes not fix it either — thetoolchaindirective is a floor underauto, not a ceiling. Fully closing that gap would need something like a committed.envrc, which is a bigger call; the make-level pin is the pragmatic 90%.go.mod'sgodirective now also requires a golangci-lint release built with at least that Go, orlint-gopanics in CI instead of locally. Worth a line indevelopment.md.Related
GOLANGCI_LINT_VERSION := v2.11.4(Makefile:163).makestep failed without saying why; same theme of local/CI diagnosability.Found while merging the September dependency PRs, where the pre-commit hook failed on
lint-gofor purely environmental reasons.