Skip to content

fix(image): export an image from another node when its copy is damaged - #8315

Draft
devantler wants to merge 3 commits into
mainfrom
claude/image-export-node-fallback-8251
Draft

devantler wants to merge 3 commits into
mainfrom
claude/image-export-node-fallback-8251

Conversation

@devantler

Copy link
Copy Markdown
Contributor

Why

Exporting images from a cluster with several nodes gave up as soon as the node picked for an image could not export it, for example because its copy was damaged, even when another node held a complete copy.

What

When a node cannot export its share, the export now takes those images from another node that holds them, and only fails, keeping the previous archive, when no node can supply an image. A new test also pins that an image a repair re-pulled under its tag is still published. The nested multi-node system test that this issue also carries stays open there.

Part of #8251

🤖 Generated with Claude Code

devantler and others added 2 commits November 10, 2026 22:02
A multi-node export failed as soon as the node chosen for an image could
not export it, even when another node held a complete copy. A failed
share now moves to the other nodes that hold its images, reading any
node the first plan skipped. An image no node can supply still fails
with the original diagnostics and keeps the existing archive.

Part of #8251

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A content repair can re-pull a tag@digest image by its tag, so a node
archive can name the tag instead of the planned reference. Pin that a
multi-node export still publishes it instead of refusing a complete
archive.

Part of #8251

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅⚠️MegaLinter analysis: Success with warnings

⚠️ BASH / bash-exec - 1 error
Results of bash-exec linter (version 5.3.9)
See documentation on https://megalinter.io/10.2.0/descriptors/bash_bash_exec/
-----------------------------------------------

✅ [SUCCESS] .github/actions/cleanup-hetzner/cleanup-hetzner.sh
✅ [SUCCESS] .github/actions/cleanup-hetzner/cleanup-hetzner.test.sh
✅ [SUCCESS] .github/actions/free-disk-space/free-disk-space.sh
✅ [SUCCESS] .github/actions/ksail-cluster/hetzner-inventory-config.sh
✅ [SUCCESS] .github/actions/ksail-system-test/argocd-adoption.sh
✅ [SUCCESS] .github/actions/ksail-system-test/autoscaler-audit-trial.sh
✅ [SUCCESS] .github/actions/ksail-system-test/calico-migration.sh
✅ [SUCCESS] .github/actions/ksail-system-test/calico-migration.test.sh
✅ [SUCCESS] .github/actions/ksail-system-test/chart-values-drift.sh
✅ [SUCCESS] .github/actions/ksail-system-test/helm-cli.sh
✅ [SUCCESS] .github/actions/ksail-system-test/helm-values-precedence.sh
✅ [SUCCESS] .github/actions/ksail-system-test/offline-ecr-upstream.sh
✅ [SUCCESS] .github/actions/ksail-system-test/offline-ecr-upstream.test.sh
✅ [SUCCESS] .github/actions/ksail-system-test/talos-kubernetes-upgrade.sh
✅ [SUCCESS] .github/actions/ksail-system-test/verify-offline-ecr-pull.sh
✅ [SUCCESS] .github/actions/ksail-system-test/verify-offline-ecr-pull.test.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/install-with-recovery.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/install-with-recovery.test.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/reject-remote-kustomize-resources.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/remote-resource-guard.test.sh
✅ [SUCCESS] .github/actions/pull-registry-image/pull-registry-image.sh
✅ [SUCCESS] .github/actions/rate-limit-gate/wait-for-rate-limit.sh
✅ [SUCCESS] .github/actions/rate-limit-gate/wait-for-rate-limit.test.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/cache-validation.test.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/mirror-artifact.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/pull-helper-image.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/pull-helper-image.test.sh
✅ [SUCCESS] .github/scripts/assert-gitops-deployed.sh
✅ [SUCCESS] .github/scripts/assert-gitops-deployed.test.sh
✅ [SUCCESS] .github/scripts/assert-k3k-server-creator.sh
✅ [SUCCESS] .github/scripts/assert-k3k-server-creator.test.sh
✅ [SUCCESS] .github/scripts/build-calico-migration-baseline.sh
✅ [SUCCESS] .github/scripts/cask-pr-handoff-workflow.test.sh
✅ [SUCCESS] .github/scripts/cask-read-api.sh
✅ [SUCCESS] .github/scripts/cask-read-api.test.sh
✅ [SUCCESS] .github/scripts/cask-rendering.test.sh
✅ [SUCCESS] .github/scripts/check-eks-capacity.sh
✅ [SUCCESS] .github/scripts/check-eks-capacity.test.sh
✅ [SUCCESS] .github/scripts/codeql-autobuild-prepare.sh
✅ [SUCCESS] .github/scripts/codeql-autobuild-prepare.test.sh
✅ [SUCCESS] .github/scripts/codeql-source.sh
✅ [SUCCESS] .github/scripts/collect-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/collect-cask-pr-handoff.test.sh
✅ [SUCCESS] .github/scripts/collect-release-build-experiment.sh
✅ [SUCCESS] .github/scripts/delete-eks-smoke-cluster.sh
✅ [SUCCESS] .github/scripts/delete-eks-smoke-cluster.test.sh
✅ [SUCCESS] .github/scripts/delete-old-workflow-runs.sh
✅ [SUCCESS] .github/scripts/delete-old-workflow-runs.test.sh
✅ [SUCCESS] .github/scripts/diagnostics-artifact-fixture.sh
✅ [SUCCESS] .github/scripts/diagnostics-artifact-fixture.test.sh
✅ [SUCCESS] .github/scripts/diagnostics-artifact-ksail-fixture.sh
✅ [SUCCESS] .github/scripts/download-go-modules.sh
✅ [SUCCESS] .github/scripts/eks-report-test-lib.sh
✅ [SUCCESS] .github/scripts/find-merged-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/find-merged-cask-pr-handoff.test.sh
✅ [SUCCESS] .github/scripts/finish-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/finish-cask-pr-handoff.test.sh
❌ [ERROR] .github/scripts/generate-completions.sh
    Error: File:[.github/scripts/generate-completions.sh] is not executable

✅ [SUCCESS] .github/scripts/go-notices-release.test.sh
✅ [SUCCESS] .github/scripts/install-release.test.sh
✅ [SUCCESS] .github/scripts/list-link-check-files.sh
✅ [SUCCESS] .github/scripts/prepare-nested-kind-host.sh
✅ [SUCCESS] .github/scripts/prepare-nested-kind-host.test.sh
✅ [SUCCESS] .github/scripts/profile-codeql-cohort.sh
✅ [SUCCESS] .github/scripts/profile-codeql-compare.sh
✅ [SUCCESS] .github/scripts/profile-codeql-compare.test.sh
✅ [SUCCESS] .github/scripts/profile-codeql-files.sh
✅ [SUCCESS] .github/scripts/profile-codeql-files.test.sh
✅ [SUCCESS] .github/scripts/profile-codeql-go.sh
✅ [SUCCESS] .github/scripts/profile-codeql-go.test.sh
✅ [SUCCESS] .github/scripts/profile-codeql-swap.sh
✅ [SUCCESS] .github/scripts/profile-codeql-swap.test.sh
✅ [SUCCESS] .github/scripts/publish-mcp-registry.sh
✅ [SUCCESS] .github/scripts/publish-mcp-registry.test.sh
✅ [SUCCESS] .github/scripts/publish-operator-chart.sh
✅ [SUCCESS] .github/scripts/redraft-evergreen-cask-prs.sh
✅ [SUCCESS] .github/scripts/redraft-evergreen-cask-prs.test.sh
✅ [SUCCESS] .github/scripts/release-build-collector.test.sh
✅ [SUCCESS] .github/scripts/release-build-experiment.test.sh
✅ [SUCCESS] .github/scripts/release-build-samples.lib.sh
✅ [SUCCESS] .github/scripts/report-eks-account-capacity.sh
✅ [SUCCESS] .github/scripts/report-eks-account-capacity.test.sh
✅ [SUCCESS] .github/scripts/report-eks-create-failure.sh
✅ [SUCCESS] .github/scripts/report-eks-create-failure.test.sh
✅ [SUCCESS] .github/scripts/resolve-cask-publication.sh
✅ [SUCCESS] .github/scripts/resolve-cask-publication.test.sh
✅ [SUCCESS] .github/scripts/resolve-release-publish-state.sh
✅ [SUCCESS] .github/scripts/resolve-release-publish-state.test.sh
✅ [SUCCESS] .github/scripts/resolve-release-replay.sh
✅ [SUCCESS] .github/scripts/strict-link-transport.sh
✅ [SUCCESS] .github/scripts/strict-link-transport.test.sh
✅ [SUCCESS] .github/scripts/style-clean-cask-branch.sh
✅ [SUCCESS] .github/scripts/style-clean-cask-branch.test.sh
✅ [SUCCESS] .github/scripts/summarize-mutation-report.sh
✅ [SUCCESS] .github/scripts/summarize-mutation-report.test.sh
✅ [SUCCESS] .github/scripts/summarize-nested-kubelet.sh
✅ [SUCCESS] .github/scripts/upload-release-assets.sh
✅ [SUCCESS] .github/scripts/upload-release-assets.test.sh
✅ [SUCCESS] .github/scripts/validate-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/validate-cask-pr-handoff.test.sh
✅ [SUCCESS] .github/scripts/validate-release-ref.sh
✅ [SUCCESS] .github/scripts/validate-release-ref.test.sh
✅ [SUCCESS] .github/scripts/verify-desktop-codeql.sh
✅ [SUCCESS] .github/scripts/verify-desktop-codeql.test.sh
✅ [SUCCESS] .github/scripts/verify-diagnostics-artifact.sh
✅ [SUCCESS] .github/scripts/verify-go-archive-parity.sh
✅ [SUCCESS] .github/scripts/verify-go-archive-parity.test.sh
✅ [SUCCESS] .github/scripts/verify-otelzap-parity.sh
✅ [SUCCESS] .github/scripts/verify-otelzap-parity.test.sh
✅ [SUCCESS] .github/scripts/verify-vendored-comment-scan.sh
✅ [SUCCESS] .github/scripts/wait-for-github-api-budget.sh
✅ [SUCCESS] .github/scripts/wait-for-github-api-budget.test.sh
✅ [SUCCESS] desktop/scripts/make-icns.sh
✅ [SUCCESS] desktop/scripts/make-info-plist.sh
✅ [SUCCESS] desktop/scripts/make-macos-app.sh
✅ [SUCCESS] desktop/scripts/verify-macos-archive.sh
✅ [SUCCESS] docs/demos/render.sh
✅ [SUCCESS] images/ksail-analysis-runner/smoke.sh
✅ [SUCCESS] install.sh
✅ [SUCCESS] internal/ciharness/testdata/schematic_fake_ksail.sh
✅ [SUCCESS] pkg/svc/provisioner/cluster/vcluster/containerd_registry_hosts.sh
✅ [SUCCESS] scripts/stage-webui.sh

✅ Linters with no issues

actionlint, git_diff, hadolint, jscpd, jsonlint, lychee, markdown-table-formatter, markdownlint, prettier, prettier, shellcheck, shfmt, stylelint, syft, trivy-sbom, trufflehog, v8r, v8r, yamllint

See detailed reports in MegaLinter artifacts

MegaLinter is provided by OX Security
Show us your support by starring ⭐ the repository

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

The head remains 1b5d81e434d6e635cacf879bc24e68ed62ce06fd. Both earlier CI runs were cancelled; the required aggregate failed after its dependencies were cancelled. No Go validation job reported a failure in those runs.

With no live worktree holder and no newer branch activity, I restarted Go validation and KSail CI. The Go run is on attempt 2; KSail CI accepted the rerun and is queued. Keep this draft until the required checks settle, then obtain a current-head review and verify the image-export behavior before promotion. These rerun receipts do not establish readiness.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer (interactive Claude Code session)

User trial at 1b5d81e434d6e635cacf879bc24e68ed62ce06fd

  • No container runtime on this host, so the export is exercised through its Docker boundary with real image archives and the real merge and validation code. Two nodes both list the same image; the first one's archive is damaged (in one case a layer is missing, in the other the image is absent from its archive). The export now takes the image from the second node and publishes a readable archive holding exactly that image. In the selected-image case the second node was never inventoried by the first plan, so this also exercises reading it later.
  • When no other node holds the image, the export still fails with the first node's own error plus no other node holds: <image>, and the existing archive is left untouched.
  • An image that a repair re-pulled under its tag is published under that tag instead of being refused.
  • Removing the fallback makes both fallback tests fail; restricting the archive lookup to the exact planned name makes the repaired-tag test fail. The image package tests pass locally.
  • Not covered here, so this PR is Part of #8251: the nested multi-node system test and its five review findings that the issue also carries. The CI system tests export from single-node clusters only, so the new fallback has no live run yet.

🤖 Generated with Claude Code

@github-code-quality

github-code-quality Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: Go

Go / code-coverage/go

main 01ebcd6 claude/image-export-... 82e4f8f +/-
72% 72% 0%
View coverage of the 7 most impacted files.
File main 01ebcd6 claude/image-export-... 82e4f8f +/-
pkg/svc/image/export_plan.go 96% 92% -4%
pkg/svc/provisi...pgrader_wait.go 94% 92% -2%
pkg/svc/mirror/pump.go 97% 98% +1%
pkg/svc/image/multi_export.go 84% 86% +2%
pkg/svc/install...site_removal.go 90% 92% +2%
pkg/svc/provisi...etzner/token.go 71% 75% +4%
pkg/svc/mirror/keepalive.go 88% 93% +5%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated November 11, 2026 04:04 UTC

Keep the export plan alongside main's platform notice after an export.

Part of #8251

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🫴 Ready

Development

Successfully merging this pull request may close these issues.

1 participant