Skip to content

feat(k8s): always re-pull remote game server images (v1.0.2)#4

Merged
Exonical merged 4 commits into
mainfrom
release/v1.0.2
Jun 20, 2026
Merged

feat(k8s): always re-pull remote game server images (v1.0.2)#4
Exonical merged 4 commits into
mainfrom
release/v1.0.2

Conversation

@devin-ai-integration

Copy link
Copy Markdown

Summary

Game server Pods and installation Jobs hard-coded ImagePullPolicy: IfNotPresent, so once an image was cached on a node, restarting or redeploying a server reused the stale local copy and never picked up an updated tag — with no config knob to change it. This aligns the Kubernetes backend with the Docker backend, which always attempts a pull before (re)creating a container.

New resolveImagePullPolicy centralizes the decision and is used by both the server Pod (container.go) and the install Job (installer.go):

func resolveImagePullPolicy(image string) (string, corev1.PullPolicy) {
    cleaned := strings.TrimPrefix(image, "~")
    if override := config.Get().Kubernetes.ImagePullPolicy; override != "" {
        return cleaned, corev1.PullPolicy(override)   // explicit operator override
    }
    if strings.HasPrefix(image, "~") {
        return cleaned, corev1.PullIfNotPresent        // ~-prefixed = local image, don't pull
    }
    return cleaned, corev1.PullAlways                  // remote image, re-pull updated tags
}
  • New optional kubernetes.image_pull_policy config (Always / IfNotPresent / Never) for air-gapped clusters; empty = the smart default above.
  • Helm chart surfaces it: wings.kubernetes.imagePullPolicy (values) → rendered into the ConfigMap as image_pull_policy (omitted when empty). This is independent of image.pullPolicy, which is for the Wings daemon image itself.
  • Docs added to the chart README and kubernetes/README.md.
  • Chart version/appVersion bumped to 1.0.2.

Note: the power-lock restart-while-off fix (#3) is already merged to main, so this is the only remaining change since v1.0.1. Tagging v1.0.2 after merge will trigger the release + helm/container publish workflows.

Test plan

  • go build ./..., go vet ./environment/kubernetes/... — clean
  • go test ./environment/kubernetes/... — new TestResolveImagePullPolicy covers remote→Always, ~-local→IfNotPresent, and override→configured value
  • helm lint chart/pelican-wings — passes
  • gofmt — clean

Link to Devin session: https://app.devin.ai/sessions/c6e2874ef8924e68a929e491c2018270
Requested by: @Exonical

Game server Pods and installation Jobs hard-coded ImagePullPolicy:
IfNotPresent, so once an image was cached on a node, restarting or
redeploying a server reused the stale local copy and never picked up
updated tags. This mirrors the Docker backend instead, which always
attempts a pull before (re)creating a container.

- Add resolveImagePullPolicy: remote images default to Always so updated
  tags are re-pulled; ~-prefixed local images stay IfNotPresent.
- Add optional kubernetes.image_pull_policy override (Always /
  IfNotPresent / Never) for air-gapped clusters.
- Surface image_pull_policy in the Helm chart (values + ConfigMap) and
  document the behavior in the chart and RBAC READMEs.
- Bump chart version/appVersion to 1.0.2.
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

if strings.HasPrefix(image, "~") {
return cleaned, corev1.PullIfNotPresent
}
return cleaned, corev1.PullAlways

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 Behavioral change: default pull policy shifted from IfNotPresent to Always for remote images

Before this PR, both game server Pods (container.go:151 old) and installer Jobs (installer.go:124 old) used corev1.PullIfNotPresent unconditionally. After this PR, remote (non-~) images default to corev1.PullAlways. This is an intentional change to match Docker backend behavior (which always attempts a pull for remote images, see environment/docker/container.go:381-382), but it's a meaningful behavioral difference for existing deployments: Kubernetes will now attempt to pull on every Pod/Job creation, adding latency and requiring registry access. Operators relying on pre-cached images without registry access will see failures after upgrading unless they set image_pull_policy: IfNotPresent.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Address three robustness issues in the Kubernetes backend:

- api.go: Client() stored the init error in a function-local variable, so
  after a failed first call the sync.Once never re-ran and subsequent calls
  returned (nil, nil), masking the failure. Persist the error in a package
  variable and return it on every call.
- container.go: SendCommand() checked IsAttached() before taking the lock
  that guards e.stream, so the stream could be cleared in between and cause
  a nil-pointer write. Nil-check e.stream under the read lock instead.
- quota.go: buildResourceQuota/buildLimitRange used resource.MustParse on
  user-configurable quantity strings, panicking the process on invalid
  values. Parse with resource.ParseQuantity and propagate errors through
  EnsureResourceQuota/EnsureLimitRange.
Exonical added 2 commits June 20, 2026 05:17
Harden the Kubernetes environment and Helm chart per the full CodeRabbit
review:

environment/kubernetes:
- container.go: surface Exists() errors in Create(), order Destroy() offline
  state after the delete check, use apierrors.IsNotFound, and deduplicate
  container ports shared across allocation IPs.
- power.go: handle delete/wait errors in OnBeforeStart, give signal-based
  stops a graceful SIGTERM grace period (immediate for SIGKILL), and treat a
  terminal (non-running) Pod as stopped so restart-while-off does not hang.
- installer.go: propagate caller context into WriteInstallScript, use
  foreground propagation + wait when recreating the installer Job, and reject
  SubPath values that escape the root directory.
- quota.go: fail fast on non-NotFound Get errors instead of falling through
  to Create.
- storage.go: ResizePVC is a no-op when a shared DataPVC is configured.
- network.go: reconcile Service type and annotations on a networkMode switch.
- stats.go: bound each metrics/stats request with a timeout and retry the
  metrics API every tick instead of sticking to the kubelet fallback.

server/router/system:
- update.go: sync image and stop configuration to the Kubernetes environment.
- router_system.go: return an error when no assignable IPs are discovered.
- system.go: treat a missing os-release as best-effort in Kubernetes mode.

chart/pelican-wings:
- Render credentials into a Secret instead of a ConfigMap.
- Guard wings.kubernetes.namespace against gameNamespace drift.
- Fail rendering when ResourceQuota/LimitRange are enabled but empty.
- Require an explicit serviceAccount.name when not creating one.
- Gate the broad nodes/proxy kubelet fallback behind rbac.kubeletMetricsFallback.

docs/manifests:
- Split the optional nodes/proxy ClusterRole into clusterrole-metrics-kubelet.yaml.
- Expand the chart and kubernetes RBAC READMEs (credentials, networkMode,
  multi-instance guidance, RBAC tables).

Add regression tests for port deduplication, Service type reconciliation,
ResizePVC with a shared DataPVC, and quota Get-failure fail-fast.
… test

- server/manager.go: initialize Kubernetes Metadata with the server stop
  configuration so command/signal stop behavior is correct from first boot.
- kubernetes/README.md: sync the namespace-scoped RBAC verb table with role.yaml
  (pods watch, services list, configmaps update).
- chart README: use a local values file for credentials in the quick-start
  instead of leaking them via --set.
- installer_test.go: assert the Run cancellation path returns context.Canceled.
@Exonical
Exonical merged commit ce34d8e into main Jun 20, 2026
8 checks passed
@Exonical
Exonical deleted the release/v1.0.2 branch June 20, 2026 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant