chore: propagate changes from main into development - #1307
Merged
Merged
Conversation
Extract the caddy-builder and crowdsec-builder Dockerfile stages into a digest-pinned, multi-arch ghcr.io/wikid82/charon-toolchain image so the ~14-min xcaddy compile runs once per security-relevant change instead of on every CI image build (currently forced via --no-cache-filter in 7 places, the root cause of the PR #1298 build-job timeout cancellations). Rev 2 incorporates the supervisor review: alternatives decision record, corrected security-refresh baseline, withdrawn overstated recurrence claim, pinned xcaddy plugins, guard-never-inert commit slicing, fork-reachable timeout preservation, and a failure-closed freshness guard. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Extract the custom Caddy + CrowdSec build recipe into a separately-versioned
prebuilt image so ordinary app builds stop recompiling xcaddy / CrowdSec on the
hot path (spec docs/plans/current_spec.md §12 Commit 1).
- scripts/lib/dockerfile-stage.sh — shared `extract_stage` (N9), sourced by both
key scripts.
- scripts/toolchain-key.sh — deterministic content-addressed tag
(caddy-crowdsec-<hex>) over the two inline stage bodies + consumed version ARGs
(incl. the two now-pinned xcaddy plugins) + the xx / digest-pinned golang base
lines + .trivyignore + a SCHEMA_VERSION.
- scripts/verify-toolchain-pin.sh — failure-closed freshness guard (B7): on a
trusted same-repo run a missing regctl / missing GHCR_READ_TOKEN /
unresolvable :$KEY / digest mismatch each exit 1; only a fork degrades to
tag-only with a ::warning::.
- scripts/tests/*.bats — determinism + sensitivity + failure-closed matrix (§7);
wired into quality-checks.yml as a non-blocking job for now.
- .github/workflows/toolchain-image.yml — daily schedule + workflow_dispatch +
pull_request(paths) + workflow_call; multi-arch $BUILDPLATFORM cross-compile
(no QEMU) of --target toolchain-runtime, GHCR publish on same-repo, Trivy
CRITICAL/HIGH gate (blocking off-PR, report-only on PR) + SARIF. The
sync-pin-on-pr / open-bump-pr jobs land in Commit 3.
- Dockerfile:
* rename caddy-builder -> caddy-inline, crowdsec-builder -> crowdsec-inline;
* digest-pin the golang:${GO_VERSION}-alpine base of both (N4) and feed the
digest line to the key;
* pin the two loose xcaddy plugins — CADDY_GEOIP2_VERSION (full pseudo-version,
caddy-geoip2 has no semver tags) and CADDY_RATELIMIT_VERSION=0.1.0 (B4) —
and feed both to the key;
* delete the dead crowdsec-fallback stage and the now-unused
CROWDSEC_RELEASE_SHA256 ARG (N1);
* add the toolchain-runtime assembly stage (binaries at identical paths) with
an io.charon.toolchain.key provenance LABEL;
* add temporary aliases `FROM caddy-inline AS caddy-builder` /
`FROM crowdsec-inline AS crowdsec-builder` so the app build is byte-identical
this commit (selector + prebuilt-image consumption land in Commit 2).
- Retarget every --no-cache-filter / no-cache-filters value from
caddy-builder,crowdsec-builder to caddy-inline,crowdsec-inline across
docker-build.yml, security-pr.yml, supply-chain-pr.yml, e2e-tests-split.yml,
nightly-build.yml, and the build-charon-image composite doc comment — so the
CVE-recurrence guard keeps invalidating the real RUN layers through the rename
(B5). Full removal is Commit 4, after the freshness guard is live (Commit 3).
Plugin/base versions are best-effort here; the exact baked CADDY_GEOIP2_VERSION
is confirmed from the first toolchain-image CI run's xcaddy log and re-pinned.
Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
The default app-build path now COPY --from's the digest-pinned prebuilt toolchain
image instead of compiling xcaddy / CrowdSec; the inline stages become the
selectable fork / bootstrap / offline fallback (spec §12 Commit 2).
- Dockerfile:
* CHARON_TOOLCHAIN_DIGEST set to the first published manifest-list (OCI index)
digest sha256:56ea3568… for ghcr.io/wikid82/charon-toolchain
:caddy-crowdsec-1efe7f19fa52a512 (multi-arch: linux/amd64 + linux/arm64
confirmed). Baked plugin versions confirmed from the toolchain build's xcaddy
log: caddy-geoip2 v0.0.0-20260623062220-3675c6e7e63d, caddy-ratelimit v0.1.0
— unchanged from Commit 1's pins.
* add CADDY_BUILDER_SRC / CROWDSEC_BUILDER_SRC selector ARGs (default
toolchain-prebuilt).
* add the toolchain-prebuilt stage
(FROM ${CHARON_TOOLCHAIN_IMAGE}@${CHARON_TOOLCHAIN_DIGEST}) and replace the
temp aliases with FROM ${CADDY_BUILDER_SRC} AS caddy-builder /
FROM ${CROWDSEC_BUILDER_SRC} AS crowdsec-builder. The unreferenced
alternative is pruned — a fork build never pulls the image.
* add org.opencontainers.image.source to the toolchain-runtime LABEL so the
GHCR package links to the repo.
- .github/actions/build-charon-image: new `builder-src` (prebuilt|inline) and
`ghcr-token` inputs; a srcsel resolver step maps them to the two build-args;
a GHCR login step (skipped for inline) so BuildKit can pull the private image.
- Every app-image build path passes the selector + a fork-detection expression
(`head.repo.full_name != '' && != github.repository` -> inline):
* docker-build.yml build-amd64 / build-arm64 (raw buildx, via job env);
* nightly-build.yml (hardcoded prebuilt — same-repo only);
* e2e-tests-split.yml build job (+ GHCR login + packages: read);
* security-pr.yml / supply-chain-pr.yml / the 4 *-integration.yml workflows
(via the composite `builder-src` + `ghcr-token`, + packages: read).
The retargeted `--no-cache-filter caddy-inline,crowdsec-inline` stays for now:
a no-op on the prebuilt path (stages pruned), live on the inline fork path
(B5). Full removal is Commit 4, after the freshness guard is required (Commit 3).
- Makefile: `build-offline` target (--build-arg *_BUILDER_SRC=*-inline).
Local gate: `docker buildx build --target caddy-builder` on the default path
pulls the toolchain image (40 MB) with NO xcaddy / go build step; the pulled
caddy lists http.handlers.{rate_limit,crowdsec,geoip2} + geoip2 and reports
v2.11.4. `docker build --check` clean; toolchain-key unchanged.
Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Spec §12 Commit 3 (+ Rev 2.1 determinism amendment, see docs/plans/current_spec.md
§3.4.3 — flagged for supervisor re-review).
- quality-checks.yml:
* `toolchain-key-tests` (bats) is now BLOCKING (dropped continue-on-error).
* new REQUIRED `verify-toolchain-pin` job — installs regctl
(iarekylew00t/regctl-installer, SHA-pinned) and runs
scripts/verify-toolchain-pin.sh with GHCR_READ_TOKEN=secrets.GITHUB_TOKEN
and head.repo.full_name mapped into env (B7). Failure-closed on same-repo
runs.
- toolchain-image.yml:
* DETERMINISTIC build (fixes a self-perpetuating sync-pin bot-commit loop):
`--provenance=false --sbom=false`, a fixed `SOURCE_DATE_EPOCH=1700000000`,
and `--output type=image,"name=…",push=true,rewrite-timestamp=true`. Two
independent builds now produce an identical manifest-list digest. The
toolchain image is an internal build input; the app image's provenance/SBOM
(docker-build.yml) is separate and unaffected.
* SKIP-IF-ALREADY-PUBLISHED: on any non-forced event, if `:$KEY` resolves in
GHCR the job reuses that digest and skips the build/push entirely. Only
schedule / workflow_dispatch force_rebuild / workflow_call rebuild+repush.
* `sync-pin-on-pr` — idempotent + self-trigger-safe: guarded
`github.actor != 'github-actions[bot]'`, and no-ops unless
`git diff --quiet Dockerfile` shows a real change. Pushes the two-line
TAG/DIGEST bump onto the PR head branch (::notice:: to re-run the check).
* `open-bump-pr` — off the PR path only (schedule / workflow_dispatch /
workflow_call); opens `bot/bump-toolchain-image` against base `development`
(peter-evans/create-pull-request, SHA-pinned) titled `feat(security): refresh
bundled proxy toolchain image`; on failure opens a tracked issue.
- Dockerfile N5 — after the final-stage COPY --from=caddy-builder / crowdsec-builder:
* assert the toolchain caddy binary exposes http.handlers.{rate_limit,crowdsec,
geoip2,waf} (the 4 recipe plugins; the WAF module is `http.handlers.waf`);
* assert cscli runs and emits its recognisable version block (CrowdSec 1.8.x
prints an empty `version:` field regardless of the -X ldflag).
A wrong-arch / rolled-back digest fails the app build here, not only at
toolchain-build time.
- docker-build.yml merge-and-publish: new "Verify pinned toolchain image matches
the recipe (N5)" step — pulls the digest-pinned toolchain image and asserts
its io.charon.toolchain.key LABEL == scripts/toolchain-key.sh.
- renovate.json: packageRules entry — the charon-toolchain digest is bot-owned,
not Renovate-tracked (N7), disabled there.
- docs/plans/current_spec.md §3.4.3: Rev 2.1 amendment documenting the three
determinism / loop-prevention fixes above.
Local: full `docker build .` (default prebuilt path) passes both N5 assertions;
two independent deterministic multi-arch builds produce identical manifest-list
digest sha256:6575f4c6a9f76074870c64df9dd4c9ebee812342f37f52ae5ef8f511ba9f8f00
(pinned in Commit 2); `docker build --check` clean; toolchain key unchanged
(caddy-crowdsec-1efe7f19fa52a512).
Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Spec §12 Commit 4. Now safe: (a) the default app-build path never compiles the inline stages, (b) verify-toolchain-pin is a live required check enforcing pin<->digest freshness per PR (Commit 3), (c) the daily toolchain rebuild + blocking Trivy gate covers base-image drift, (d) the N5 assertions catch a wrong digest in the app build. - docker-build.yml: remove the four `--no-cache-filter caddy-inline` / `crowdsec-inline` lines from build-amd64 / build-arm64; reword the retry-block comment. - security-pr.yml / supply-chain-pr.yml: drop the `no-cache-filters:` override (and its justification comment) from the build-charon-image `with:` block. - e2e-tests-split.yml / nightly-build.yml: drop the `no-cache-filters:` input. - build-charon-image composite: delete the `no-cache-filters` input declaration and its `docker/build-push-action` passthrough; rewrite `description:` to state the binaries come from the digest-pinned toolchain image and every stage is layer-cached. `grep -rn no-cache-filter .github/` is clean. actionlint + lefthook green. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Spec §12 Commit 5 / §3.4.4. security-weekly-rebuild.yml no longer builds a throwaway `charon:security-scan-YYYYMMDD` app image that nothing consumes. Instead: - `toolchain-rebuild` job — `uses: ./.github/workflows/toolchain-image.yml` (workflow_call, force_rebuild: true). This drives the same deterministic `--no-cache --pull` rebuild + publish + BLOCKING Trivy CRITICAL/HIGH gate + `:trivy-toolchain` SARIF + (if the digest moved) `bot/bump-toolchain-image` PR as the daily schedule. N6: the job grants `contents: write`, `pull-requests: write`, `issues: write` (plus packages / security-events) so the reusable workflow's open-bump-pr job has the perms it needs — a workflow_call'ed workflow cannot request perms the caller did not grant. - `extended-report` job — pulls the rebuilt toolchain digest and produces the fuller weekly artefacts: a MEDIUM/LOW Trivy JSON artifact (90-day retention) and an installed-package table in the step summary. No second SARIF upload (the reusable workflow already owns the `:trivy-toolchain` category). - toolchain-image.yml gains `workflow_call` `outputs.key` / `outputs.digest` so the caller can reference the rebuilt digest. actionlint + lefthook green. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
…ale comments Spec §12 Commit 6 / §3.9 / §9. Timeouts (§3.9): - docker-build.yml build-amd64: 15 -> 20 (job + nested nick-fields/retry). No compile on the hot path; 20 gives headroom for a cold GHA cache miss on the fast stages + cache export + push. build-arm64 stays 25; merge-and-publish stays 10; e2e build stays 60. - security-pr / supply-chain-pr / the 4 *-integration jobs KEEP timeout-minutes: 20 (B6 — fork-reachable, they run the ~14-min inline compile). Comments only. Stale-comment sweep: - docker-build.yml: reworded the build-split comment; dropped the dangling `docs/plans/current_spec.md §1.1` cross-ref (pointed at the retired uptime spec) and the QEMU-emulated-compile framing (both builder stages were always $BUILDPLATFORM cross-compiled). - e2e-tests-split.yml: fixed the malformed `# v4uses: docker/setup-buildx-action@…` comment mash on the Set up Docker Buildx step. - Integration + CVE-gate timeout comments reworded to describe the prebuilt vs fork-inline paths. Docs (§9): - ARCHITECTURE.md: Infrastructure table row; new "Prebuilt toolchain image" subsection under Deployment Architecture / Multi-Stage Dockerfile; Directory Structure note (new scripts/workflow, crowdsec-fallback removed, stage renames); Security Architecture / Layer 2 supply-chain-hardening note with the precise §3.8.3 scope; Local Development Setup build note + make build-offline. - SECURITY.md: new "Build Integrity — Bundled Caddy / CrowdSec Toolchain" section (daily deterministic rebuild + blocking Trivy + failure-closed verify-toolchain-pin; unpinned-transitive-MVS gap stated verbatim, no overclaim). - CONTRIBUTING.md: fork PRs compile the toolchain from source; make build-offline. - new docs/ci/toolchain-image.md: operator/maintainer runbook (how the key works, determinism, triggers, force-rebuild, respond to the failure issue / verify-toolchain-pin failure, rollback, one-time bootstrap notes). Ignore-file check (CLAUDE.md): `Dockerfile: COPY scripts/` does copy the new shell scripts into the runtime image (a few KB), consistent with the dozens of scripts/*.sh already shipped — no .dockerignore change. .gitignore: source files that must be committed, no glob match. .codecov.yml: shell/bats/YAML carry no Go/TS coverage — no change. Recorded in docs/ci/toolchain-image.md and the PR. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
…ge context `Dockerfile` `COPY scripts/ /app/scripts/` copies the whole scripts directory into the runtime image. The toolchain helpers added by this PR are used only by .github/workflows/toolchain-image.yml and the quality-checks.yml verify-toolchain-pin / bats jobs from a plain checkout — never from inside a built container — so `.dockerignore` now excludes: scripts/tests/ scripts/toolchain-key.sh scripts/verify-toolchain-pin.sh scripts/lib/dockerfile-stage.sh Verified via a scratch `COPY scripts/` probe: those paths are absent from the build context while runtime scripts (e.g. db-recovery.sh) remain. `.dockerignore` uses blacklist semantics with no `!scripts/…` re-includes, so the plain excludes take effect. Also corrects the stale premise in docs/plans/current_spec.md §2.5 that claimed `scripts/` is not copied into the image. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
…ntext exclusions - docs/ci/toolchain-image.md: the "one-time bootstrap notes" paragraph predated the `.dockerignore` follow-up and still said the new build-only scripts ship in the runtime image. Corrected: `scripts/toolchain-key.sh`, `scripts/verify-toolchain-pin.sh`, `scripts/lib/dockerfile-stage.sh` and `scripts/tests/` are excluded from the image build context via `.dockerignore`. - docs/security.md: added a one-line pointer from the novice-facing security doc to the new "Build Integrity" section in SECURITY.md. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Adds .github/workflows/integration-tests.yml, which builds the Charon image a single time via the shared build-charon-image composite action, uploads it as a 1-day run-scoped artifact, then fans out to parallel per-suite test jobs (cerberus / waf / rate-limit / crowdsec) that `docker load` that image instead of rebuilding it. Replaces four near-identical standalone workflows that each paid the image-build cost independently. Each suite job keeps its own test steps, debug-dump-on-failure and cleanup verbatim (each inspects different containers/ports/volumes). Job/check names are preserved so the diff is reviewable, but they now live under one workflow. Also guards the unconditional `docker build` in scripts/crowdsec_integration.sh behind a `docker image inspect charon:local` check, matching the sibling scripts (crowdsec_startup_test.sh, cerberus/waf/rate_limit_integration.sh) so the crowdsec job actually consumes the pre-built artifact rather than rebuilding the image a second time. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Points docs and instructions that named the four deleted workflow files (cerberus-integration.yml / waf-integration.yml / rate-limit-integration.yml / crowdsec-integration.yml) at .github/workflows/integration-tests.yml and its per-suite jobs instead. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Contributor
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
Contributor
Author
✅ Supply Chain Verification Results✅ PASSED 📦 SBOM Summary
🔍 Vulnerability Scan
📎 Artifacts
Generated by Supply Chain Verification workflow • View Details |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Reverts 6f38be5, a stray `sync-pin-on-pr` push from the bot-authored main -> development propagation PR #1307. That job checked out the PR head branch — which for a propagation PR IS `main` — recomputed the toolchain key against the merge ref (which pulls in development's gRPC 1.83.2 bump, a toolchain-key input) and committed the resulting `caddy-crowdsec-9eb9862f44b9e769` pin directly onto `main`. `main`'s actual recipe still has `GRPC_VERSION=1.83.1`, whose key is `caddy-crowdsec-1efe7f19fa52a512` (@ sha256:6575f4c6…), the digest #1300's docker-build actually pulled. The mismatch fails `verify-toolchain-pin` on every PR against `main`. Verified on this branch: - scripts/toolchain-key.sh -> caddy-crowdsec-1efe7f19fa52a512 - imagetools inspect <that tag> -> sha256:6575f4c6a9f76074870c64df9dd4c9ebee812342f37f52ae5ef8f511ba9f8f00 Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
The `sync-pin-on-pr` job in toolchain-image.yml pushes a recomputed TAG/DIGEST pin onto `github.event.pull_request.head.ref`. Its only guard against unwanted runs was `github.actor != 'github-actions[bot]'`, which does not fire for the main -> development auto-propagation PR (#1307): that PR's `pull_request` event runs under a non-bot actor even though the PR itself is bot-authored and its head ref is `main`. Result: the job recomputed the key against the propagation merge ref (which drags in development's gRPC bump) and committed `caddy-crowdsec-9eb9862f44b9e769` straight to `main`, breaking `verify-toolchain-pin` repo-wide (see the companion revert in this PR). Adds three guards to the job `if:`: - refuse any protected head ref (main / development / nightly / feature/beta-release) outright — this job has no business rewriting a long-lived branch in place; - also skip when the PR *author* is github-actions[bot], not just when the triggering actor is; - (existing actor guard retained). Legitimate toolchain-key moves on `development` are still handled by the `open-bump-pr` job, which opens a reviewed bot PR rather than pushing in place. No workflow-`if:` test harness exists in-tree (bats covers the scripts, not YAML guards); actionlint passes. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
… protected branches (#1309)
1-day retention breaks "Re-run failed jobs" on a run older than 24h now that the suite jobs depend on the upstream build job's artifact instead of building the image themselves. 3 days covers realistic re-run windows at negligible cost. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
The `open-bump-pr` job in toolchain-image.yml opened its daily digest-refresh PR with a `feat(security):` title and commit message. `feat:` makes release-please cut a minor release on every merge to `main`, so a routine toolchain-digest refresh was bumping the version each time. Switch the bot PR title and commit-message to `chore(docker): refresh bundled proxy toolchain image`, matching the sibling `sync-pin-on-pr` commit style (`chore(docker): sync toolchain image pin to …`). The `security` label is kept; only the conventional-commit prefix changes. Also updates docs/plans/current_spec.md so its bot-PR example and commit-convention note no longer contradict the workflow. Affects future bot PRs only. Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
Wikid82
marked this pull request as ready for review
September 8, 2026 15:36
This was referenced Sep 8, 2026
Wikid82
added a commit
that referenced
this pull request
Sep 9, 2026
`development` carries c32e611 (gRPC 1.83.2), a toolchain-key input that `main` does not have yet, so the toolchain pin propagated into `development` by #1307 (caddy-crowdsec-1efe7f19fa52a512 @ sha256:6575f4c6…, computed for main's gRPC 1.83.1 recipe) is stale here: the recipe on this branch recomputes to caddy-crowdsec-9eb9862f44b9e769, failing verify-toolchain-pin. Bumps the two ARG lines to the key/digest that match development's recipe. This is the manual equivalent of the daily open-bump-pr bot (now using the chore(docker): convention). Verified on this branch: - scripts/toolchain-key.sh -> caddy-crowdsec-9eb9862f44b9e769 - imagetools inspect <that tag> -> sha256:b41e571d5951bbfc3daa3dccdca033ad9dee535a8e720ac7e3b0bce338f223b2 Claude-Session: https://claude.ai/code/session_01KXA4x9LrA2AsnLrvdHMZbS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docs/plans/current_spec.mdAutomated PR to propagate changes from main into development.
Triggered by push to main.