Repository navigation
feat: serve plugins with dependencies at the commit pinned by the Go module proxy - #83
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Mutable-tag resolution still permits unpinned downloads, and moved annotated tags can produce archives incompatible with recorded hashes.
2 open issues
What changed in this PR
This PR uses Go module proxy metadata to select GitHub archives for plugins with dependencies, aiming to preserve published code when tags move or disappear.
Changes:
- Adds proxy-pinned commit lookup and tag verification, retaining tag downloads when unchanged.
- Introduces
--pin-proxy-url, defaulting tohttps://proxy.golang.org. - Adds reference-selection tests and updates handler construction.
| File | Description |
|---|---|
pkg/handlers/pinned.go |
Resolves archive references using proxy metadata and GitHub tag checks. |
pkg/handlers/pinned_test.go |
Tests stable, moved, deleted, unknown, and error cases. |
pkg/handlers/module.go |
Uses the resolved reference for archive downloads. |
pkg/handlers/handlers.go |
Adds pinning configuration and an HTTP client with a timeout. |
pkg/handlers/handlers_test.go |
Updates constructor calls. |
pkg/handlers/blacklist_test.go |
Updates constructor calls. |
go.mod |
Promotes golang.org/x/mod to a direct dependency. |
cmd/serve/serve.go |
Passes the pinning proxy URL to handlers. |
cmd/serve/config.go |
Adds the pinning proxy configuration field. |
cmd/serve/command.go |
Registers and reads the new proxy flag. |
Review effort: Balanced
| // It is the tag while the tag still points at the commit pinned by the Go module proxy at publication, | ||
| // and the pinned commit once the tag moved or was deleted. | ||
| // Requesting the tag keeps the archive bytes: the zipball of an annotated tag is named after the tag object, not the commit. | ||
| // The pinned commit of a moved annotated tag cannot give those bytes back, its hash is realigned instead. |
There was a problem hiding this comment.
Where does this realign happen? Not in this PR. If it's piceus realign --only, please say so here. Otherwise moved annotated tags fail validate as soon as this is deployed.
There was a problem hiding this comment.
It's piceus realign --only <module>@<version> (traefik/piceus#163), run on the -next catalog and reviewed one by one before the switch. The comment now says so.
| return version, nil | ||
| } | ||
|
|
||
| if err != nil { |
There was a problem hiding this comment.
The description says a proxy error fails the download, but here we serve the tag. Please update the description.
There was a problem hiding this comment.
Right, it changed after the first review. Description updated.
| Str("tag_commit", sha).Str("pinned_commit", pinned). | ||
| Msg("Tag moved after publication, serving the pinned commit") | ||
|
|
||
| return pinned, nil |
There was a problem hiding this comment.
If a version was first downloaded after its tag moved, its recorded hash is the moved one, and this now breaks it. How many versions are in that case?
There was a problem hiding this comment.
10 of the 2,159 public versions. They will be realigned with piceus realign --only on -next before the switch, which changes the code served for them. We'll test each one there and tell the authors.
| return version, nil | ||
| } | ||
|
|
||
| pinned, err := h.pinnedCommit(ctx, moduleName, version) |
There was a problem hiding this comment.
nit: the pinned commit never changes, but we look it up on every download. Could we store it? It will also reduce number of proxy calls.
There was a problem hiding this comment.
Done, cached in memory per module@version. Only successful lookups are cached, as the proxy may learn a version later.
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func TestArchiveRef(t *testing.T) { |
There was a problem hiding this comment.
nit: missing cases for a GitHub 404 and a proxy 410.
| Required: true, | ||
| }, | ||
| &cli.StringFlag{ | ||
| Name: flagPinProxyURL, |
There was a problem hiding this comment.
Please add --pin-proxy-url to the readme.
…module proxy When a tag moved after publication, download the GitHub zipball at the commit proxy.golang.org pinned for that version instead of the tag. Part of traefik/hub-issues#3124, closes traefik/infra#12142.
Co-authored-by: aikido-pr-checks[bot] <169896070+aikido-pr-checks[bot]@users.noreply.github.com>
7ce8bdc to
801d327
Compare

Plugins with dependencies are served from the GitHub zipball of their tag. If a tag moves after publication, the same version serves different code and no longer matches its recorded hash.
The download now reads the commit pinned by proxy.golang.org (
/@v/<version>.info,Origin.Hash):Origin(version unknown to the proxy, or published before Origin existed): download the tag.The pinned commit is cached in memory per version. An empty
--pin-proxy-urlturns pinning off.The tag check uses a conditional request on the pinned commit. A 304 does not count against the GitHub rate limit, so stable tags cost no extra quota.
Hashes that no longer match the bytes served (moved annotated tags, hashes recorded after a tag moved) are realigned with
piceus realign --onlyon the -next catalog.New flag
--pin-proxy-url(defaulthttps://proxy.golang.org).Tested locally against GitHub and proxy.golang.org with
maxlerebourg/crowdsec-bouncer-traefik-plugin:a2fe60cand matches its recorded hash againPart of traefik/hub-issues#3124