Published package's function images fail to unpack on kind

Installing the published Configuration on a kind cluster leaves every composition function unhealthy. Each function pod fails to start:

Error: failed to create containerd container: mount callback failed on
/var/lib/containerd/tmpmounts/containerd-mount1570895734:
openat etc/passwd: path escapes from parent

All 18 functions report INSTALLED=True HEALTHY=False, so no XR composes at all — an InferenceCluster sits with no status conditions and no managed resources.

Why CI and the e2e don't see it

crossplane project run side-loads the locally built images into the cluster, so nothing is ever pulled. The e2e and nix run .#run both take that path and both work. The failure appears only when a cluster pulls the package from the registry, which is what a user does.

Confirmed side by side on the same function: compose-gke-cluster runs 1/1 Running from the locally built digest sha256:1f53e937... and fails to unpack from the published digest sha256:7647de02....

Reproducing

kind create cluster --name pkgtest --image kindest/node:v1.34.2
helm install crossplane crossplane-stable/crossplane -n crossplane-system \
  --create-namespace --version 2.4.0 --wait
kubectl apply -f docs/manifests/install/prerequisites.yaml
kubectl apply -f - <<'YAML'
apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: modelplane
spec:
  package: xpkg.upbound.io/modelplane/modelplane:v0.1.0-dev.1790801594.ga4021b4
YAML
kubectl get functions          # INSTALLED=True HEALTHY=False
kubectl -n crossplane-system get pods   # CreateContainerError

What I haven't established

Whether this is specific to the containerd kind ships, or would also hit a managed cluster. And whether the cause is the Appending Marketplace extensions step that nix run .#push performs after crossplane project push — the published image goes through it and the locally built one doesn't, which makes it the obvious suspect, but I didn't isolate it by pushing a pre-append image.

Found while setting up a GKE test for #470; worked around there by using a side-loaded control plane instead.

just a side note with crossplane project run we also using a registry + imageConfig to rewrite to the local registry

please use the node image we have configured in our e2e tests:
https://github.com/modelplaneai/modelplane/blob/main/e2e/run.sh#L20

kind create cluster --name pkgtest \
  --image kindest/node:v1.34.8@sha256:02722c2dedddcfc00febf5d27fbeb9b7b2c14294c82109ff4a85d89ac9ba3256

helm install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system --create-namespace \
  --set "args={--enable-dependency-version-upgrades}" \
  --set-json 'provider.defaultActivations=[]' \
  --wait

kubectl apply -f - <<'YAML'
apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: modelplane
spec:
  package: xpkg.upbound.io/modelplane/modelplane:v0.1.0-dev.1790801594.ga4021b4
YAML

kubectl -n crossplane-system get pods  
NAME                                                              READY   STATUS              RESTARTS   AGE
crossplane-7795d555d5-4vttc                                       1/1     Running             0          5m52s
crossplane-rbac-manager-6fd84767fb-p4bqb                          1/1     Running             0          5m52s
modelplane-modelplanecompose-aks-cluster-d64444d5f488-5c86t6kgg   1/1     Running             0          2m52s
modelplane-modelplanecompose-eks-cluster-d0a8b4a810b3-69d6gdksd   1/1     Running             0          2m47s
modelplane-modelplanecompose-gke-cluster-5893c7a6d98b-5885h4kk4   1/1     Running             0          2m42s
modelplane-modelplanecompose-inference-class-947813bc93e4-vdd4s   1/1     Running             0          2m37s
modelplane-modelplanecompose-inference-cluster-aef9b8dd37b7x8bp   1/1     Running             0          2m32s
modelplane-modelplanecompose-inference-gateway-6a95c6cfd1e6d4m6   1/1     Running             0          2m21s
modelplane-modelplanecompose-metric-mapping-6bdbd1244cd4-5hmf5j   1/1     Running             0          97s
modelplane-modelplanecompose-model-cache-8fdeb94660ee-6675cq62m   1/1     Running             0          2m10s
modelplane-modelplanecompose-model-deployment-2259619159a3m772l   1/1     Running             0          2m5s
modelplane-modelplanecompose-model-endpoint-ad0ec7637ca0-5c2tlk   1/1     Running             0          118s
modelplane-modelplanecompose-model-replica-e05a3aaae5e2-85s4qjb   1/1     Running             0          113s
modelplane-modelplanecompose-model-route-372f249ad959-6f96k8hdr   1/1     Running             0          107s
modelplane-modelplanecompose-model-service-8bb33869042a-79zccn4   1/1     Running             0          103s
modelplane-modelplanecompose-nebius-cluster-73e3cc298fa3-77jjcd   1/1     Running             0          2m26s
modelplane-modelplanecompose-serving-stack-32729810ec9b-7dnztnz   1/1     Running             0          2m15s
modelplane-modelplanecompose-telemetry-destination-72f65278fqdt   1/1     Running             0          92s
modelplane-modelplanecompose-usages-c9044a934696-7f57565876kgr4   1/1     Running             0          86s
modelplane-modelplanecompose-vultr-cluster-4b9e27439fcc-68gn9p2   1/1     Running             0          82s

perhaps we need to make sure that folks using a more up2date kind version ?

Write the image's passwd and group as files, not symlinks

Navigation Menu

Repository navigation

Published package's function images fail to unpack on kind #479

Why CI and the e2e don't see it

Reproducing

What I haven't established

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Published package's function images fail to unpack on kind #479

Description

Why CI and the e2e don't see it

Reproducing

What I haven't established

Activity

haarchri commented on Oct 1, 2026

haarchri commented on Oct 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions