Skip to content

Mirror container images with proxy mirror - #438

Merged
andrew merged 8 commits into
git-pkgs:mainfrom
pinguinfuss:issue-411-mirror-images
Oct 14, 2026
Merged

andrew merged 8 commits into
git-pkgs:mainfrom
pinguinfuss:issue-411-mirror-images

Conversation

@pinguinfuss

Copy link
Copy Markdown
Contributor

Closes #411.

proxy mirror now pre-fetches pkg:docker and pkg:oci images: index, platform manifests (with their attestations) and blobs, stored under the keys a pull reads. A later pull is served from cache even when the upstream is down.

  • Manifest cache keys no longer include the client's Accept header, so Docker, containerd and Podman share one entry. The old keys are still written and read as fallback.
  • proxy mirror uses the same upstream clients as serve. Before, Docker Hub answered every request with 401.
  • --platform (or platforms in the API) limits multi-arch images. repository_url only resolves to configured registries.

Tested offline against Docker Hub and ghcr.io with containerd and Docker.

Open question: Docker 29 also requests sha256-<digest> tags on every pull. Offline the proxy answers those with 502 and the pull fails, on main too. Returning 404 there fixes it. Should that go in here or in its own PR?

… header: a digest names one document, and a tag is shared by all clients that accept an index type, so Docker, containerd and Podman read the same entry and a pre-fetched image is found offline. Clients without index support keep their per-Accept entry, and the old keys are still written and read as fallback.
…o it follows OCI Bearer challenges, uses configured upstream credentials and the SSRF guard, and artifact downloads have no overall time limit. Before this the CLI used a plain client and Docker Hub answered every request with 401.
…kg:docker and pkg:oci PURLs: the mirror walks index, platform manifests (with their attestations) and blobs through the container handler, so a later pull is served from cache even when the upstream is down. A --platform filter (platforms in the API) limits multi-arch images, repository_url only resolves to configured registries, and names, tags and digests are validated before anything is fetched.
@pinguinfuss
pinguinfuss marked this pull request as draft October 11, 2026 22:39
@pinguinfuss

Copy link
Copy Markdown
Contributor Author

@andrew For #412 I'd change MirrorImage from #438 so it always revalidates tags with a HEAD and writes the tag only after all child manifests and blobs are cached. A failed or interrupted mirror then never leaves a tag pointing at an incomplete image, and the scheduled refresh can just reuse the normal mirror run. Would you rather have that change in #438, or in a separate PR for #412 on top of it?

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two reproduced failures need fixing before merge:

  • internal/handler/container_mirror.go:216: Mirroring updates the cached tag before fetching its layers. Starting with a fully mirrored tag, then moving that tag to an image with an unavailable layer, makes the mirror fail but leaves the tag pointing at the incomplete image. Publish the tag only after all selected manifests and blobs are cached, preserving the previous tag on failure. Please include that part of the proposed change in this PR and add a regression test.
  • internal/handler/container_manifest.go:194-198: An incompatible shared entry prevents lookup of compatible per-Accept entries. Caching a Docker manifest list, then an OCI index for the same tag, then taking the upstream offline makes a Docker-only Accept request return 503 despite its cached representation still being available. Continue to the fallback keys when the shared entry is incompatible, and add a regression test.

…accept, the lookup now goes on to the client's own per-Accept entry instead of giving up, so a Docker-only client still gets its cached manifest list after an OCI index was cached for the same tag.
…t points to, and stores the tag only once that image is complete, so a failed or interrupted mirror keeps the previously cached tag instead of pointing it at an image with missing layers.
… manifest when the digest is unchanged, falls back to the cached tag when the upstream body is cut off, and keeps a tag that a client pull moved while the walk ran. The platform filter is checked across the whole image, so a nested index without the wanted platform no longer fails it, and a 304 for a per-Accept fallback entry only refreshes that client's own keys instead of overwriting the shared entry.
@pinguinfuss
pinguinfuss marked this pull request as ready for review October 13, 2026 15:17
@pinguinfuss

Copy link
Copy Markdown
Contributor Author

@andrew Both are fixed, each with a regression test that fails without the fix.

  • The mirror now resolves a tag without caching it, walks the image, and only then stores the tag. If anything fails, the previous tag stays (67fd1ad).
  • When the shared entry doesn't match the client's Accept, the lookup goes on to the client's own per-Accept entry (5b9a5fc).

A second review round turned up a few more things, fixed in b0570bd:

  • The tag is checked with a HEAD first, so an unchanged tag costs no GET.
  • A cut-off manifest body falls back to the cached tag.
  • A tag that a client pull moved while the mirror was running is kept.
  • The platform filter is checked across the whole image, so a nested index without the wanted platform no longer fails it.
  • A 304 for a fallback entry no longer overwrites the shared entry.

That also covers the #412 part I asked about, so #412 can reuse the normal mirror run.

@pinguinfuss
pinguinfuss requested a review from andrew October 13, 2026 15:17
# Conflicts:
#	cmd/proxy/main.go
#	internal/server/server.go
@andrew
andrew merged commit ba81c1d into git-pkgs:main Oct 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

proxy mirror can't pre-fetch container images

2 participants