fix(ci): isolate cel-go celmatcher patch from the shared module cache

[Wikid82]

Problem

development's toolchain image build (toolchain-image.yml → caddy-inline stage) fails on linux/arm64 with:

celmatcher.go:506:19: undefined: interpreter.InterpretableV2
celmatcher.go:529:19: undefined: interpreter.InterpretableV2
ERROR: command failed after 3 attempts: env GOOS=linux GOARCH=arm64 xcaddy build v2.11.4 ...

Evidence: run 34352152425, job 102467808367. main's equivalent build passes.

Root cause

The caddy-inline stage patched Caddy v2.11.4's modules/caddyhttp/celmatcher.go directly in the shared BuildKit module cache (${GOMODCACHE}), swapping []interpreter.Interpretable{reqAttr} → []interpreter.InterpretableV2{reqAttr} for the pinned cel-go v0.29.2 (GHSA-gcjh-h69q-9w9g).

BuildKit builds linux/amd64 + linux/arm64 concurrently over a shared --mount=type=cache,target=/go/pkg/mod. amd64 reaches Stage 3, go get cel-go@v0.29.2, and seds the shared celmatcher.go to the InterpretableV2 form while arm64 is still in xcaddy Stage 1 compiling Caddy against its native cel-go v0.28.1 → undefined: interpreter.InterpretableV2.

Latent on both main and development. It only fires on development because f4a56896 chore(deps): bump coraza-caddy to 2.6.1 enlarges arm64's Stage 1 module graph enough for amd64 to race ahead of it. main still carries coraza-caddy 2.6.0 (smaller graph, arches stay ~lockstep).

Fix — approach 1 (local copy + go mod edit -replace)

Chosen because it removes the shared-cache mutation entirely rather than papering over the timing, and it mirrors a pattern already proven in this same stage:

• The celmatcher.go patch is now applied to a local copy of the Caddy module (cp -r ${GOMODCACHE}/github.com/caddyserver/caddy/v2@vX/. /tmp/caddy-patched), followed by go mod edit -replace github.com/caddyserver/caddy/v2@vX=/tmp/caddy-patched in $BUILDDIR — the identical mechanism the stage already uses for caddy-crowdsec-bouncer and go-cs-bouncer.
• The shared module cache is never written, so the patch is order-independent and arch-isolated (/tmp is per-arch; the cache is read-only).
• Verified go build . honours a filesystem replace of the xcaddy build-target module: it is a direct import of the generated main.go, at the same version, with a byte-identical go.mod (only a .go file is edited) — structurally the same as the bouncer replaces that already pass on main. No go mod tidy after the replace (same as the bouncer block).
• The now-obsolete Stage 1 _CELM_RESTORE reverse-patch is removed (nothing forward-patches the cache any more).
• The Stage 1 bouncer IPEquals cache-restore is guarded on the stale form actually being present, so steady-state builds don't write the shared cache there either (defensive cleanup for a pre-2026-09 cache only).

Determinism preserved: SOURCE_DATE_EPOCH + rewrite-timestamp stabilise the shipped toolchain-runtime image (binaries only); the final go build is -trimpath, so /tmp/caddy-patched never leaks into the binary or its build info.

Scope check

• caddy-inline — fixed (only stage that mutated celmatcher.go / the shared cache).
• crowdsec-inline — shares /go/pkg/mod but only reads/downloads; it builds the CrowdSec binaries (no Caddy, no celmatcher.go, no in-place cache sed). Its crowdsecurity/coraza/v3 graph has no cel-go/celmatcher exposure. No change needed.

Validation

• Local: docker build --check clean; hadolint clean (no new findings); tools/dockerfile_check.sh pass; scripts/toolchain-key.sh runs and yields a stable key. Pre-commit (dockerfile-check, semgrep) green.
• CI gate: toolchain-image.yml's pull_request run on this PR must build both linux/amd64 and linux/arm64 against development's recipe, and the shipped /usr/bin/caddy must still assert cel-go v0.29.x + grpc-go v${GRPC_VERSION}.

Toolchain key / pin

This edits the caddy-inline stage body, so the toolchain key moves (expected). No CHARON_TOOLCHAIN_TAG / DIGEST bump in this PR — once merged, the (now-fixed) daily bot / a workflow_dispatch publishes the image for the new key and opens the pin-bump PR against development.

Also needs to land on main

main carries the same latent race (it just hasn't fired because its module graph keeps the two arches in lockstep). This fix is generic and will flow to main on the next development → main promotion — no separate main PR required, but the promotion should not be skipped.

Unblocks PR #1322 (GHSA-3gc6-295r-xm5m propagation into development).

https://claude.ai/code/session_01Jz4LgwfkxaF8E7TdAgk94y

[github-advanced-security]

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

• The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
• Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
• You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

✅ Supply Chain Verification Results

✅ PASSED

📦 SBOM Summary

• Components: 1769

🔍 Vulnerability Scan

Severity	Count
🔴 Critical	0
🟠 High	0
🟡 Medium	0
🟢 Low	0
Total	0

📎 Artifacts

• SBOM (CycloneDX JSON) and Grype results available in workflow artifacts

---

_{Generated by Supply Chain Verification workflow • View Details}