kubectl-syncpod

High-Speed File Transfer to and from Kubernetes PVCs

Table of Contents

• About
  • Features
  • Typical Use Cases
• Example CLI Usage
  • Upload local directory to PVC
  • Download directory from PVC to local machine
• Installation
  • Using krew
  • Homebrew installation
  • Manual Installation
    • Example installation script
  • Package-Based installation
    • Debian
    • Alpine Linux
• Execution Flow
• Comparison Table
• License

---

About

• While kubectl cp and kubectl exec can both be used to copy files, performance degrades significantly when the target size is large (e.g., ~100Gi). In such cases, execution becomes drastically slow.

• Additionally, both approaches have limitations: they either require extra tools (such as tar) to be installed in the container, or they cannot be used at all with minimal base images like distroless or scratch.

• Most importantly, these methods do not support concurrent read/write operations - a critical limitation when performance and throughput matter.

Features

• Upload local files or directories to a pod's mounted volume
• Download pod volume files back to local machine
• Safe overwrite protection
• Auto-rename existing remote directories (pgdata-new-original-YYYY-MM-DD-HHMMSS)
• Concurrent file transfer with worker pool
• Preserves directory structure
• Optional automatic chown of uploaded files inside the pod (via Kubernetes exec API)
• Fully based on SFTP + Kubernetes Exec API - no side effects on other pod processes

Typical Use Cases

You’re running PostgreSQL as a StatefulSet, and you need to restore a database from a basebackup and a WAL archive. If the volume is hostPath-based, this is relatively straightforward - you simply copy the required files onto the target node. But when using CSI-backed volumes (e.g., via a cloud provider), where the PVC is mounted as a block device, the situation becomes more complex. In such cases, conventional tools fall short.

Also - you may want to scale your StatefulSet to zero and back up the PVC contents safely and efficiently - for local testing, migration, or recovery.

Basic Scenarios:

• Download backup from PVC for verification / restore
• Sync files between PVCs and local environment
• Testing PVC mount behavior
• CI/CD pipelines to prepare volume data

---

Example CLI Usage

Upload local directory to PVC:

kubectl-syncpod upload \
  --namespace pgrwl-test \
  --pvc postgres-data \
  --mount-path=/var/lib/postgresql/data \
  --src=backups \
  --dst=pgdata-new \
  --allow-overwrite \
  --owner="999:999"

Behavior:

• If /var/lib/postgresql/data/pgdata-new exists -> it is renamed (safe overwrite)
• Contents of backups/ are uploaded into /var/lib/postgresql/data/pgdata-new/
• File ownership is set to 999:999 inside the helper pod

Download directory from PVC to local machine:

kubectl-syncpod download \
  --namespace pgrwl-test \
  --pvc postgres-data \
  --mount-path=/var/lib/postgresql/data \
  --src=pgdata-new \
  --dst=backups-copy

Behavior:

• Contents of /var/lib/postgresql/data/pgdata-new/ are downloaded
• Files are written to ./backups-copy/
• Directory structure is preserved

Installation

Using krew

1. Install the Krew plugin manager if you haven’t already.
2. Run the following command:

kubectl krew install syncpod

Homebrew installation

brew tap hashmap-kz/homebrew-tap
brew install kubectl-syncpod

Manual Installation

1. Download the latest binary for your platform from the Releases page.
2. Place the binary in your system's PATH (e.g., /usr/local/bin).

Example installation script for Unix-Based OS (requirements: tar, curl, jq)

(
set -euo pipefail

OS="$(uname | tr '[:upper:]' '[:lower:]')"
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"
TAG="$(curl -s https://api.github.com/repos/hashmap-kz/kubectl-syncpod/releases/latest | jq -r .tag_name)"

curl -L "https://github.com/hashmap-kz/kubectl-syncpod/releases/download/${TAG}/kubectl-syncpod_${TAG}_${OS}_${ARCH}.tar.gz" |
tar -xzf - -C /usr/local/bin && \
chmod +x /usr/local/bin/kubectl-syncpod
)

Package-Based installation (suitable in CI/CD)

Debian

sudo apt update -y && sudo apt install -y curl
curl -LO https://github.com/hashmap-kz/kubectl-syncpod/releases/latest/download/kubectl-syncpod_linux_amd64.deb
sudo dpkg -i kubectl-syncpod_linux_amd64.deb

Alpine Linux

apk update && apk add --no-cache bash curl
curl -LO https://github.com/hashmap-kz/kubectl-syncpod/releases/latest/download/kubectl-syncpod_linux_amd64.apk
apk add kubectl-syncpod_linux_amd64.apk --allow-untrusted

---

Execution Flow

kubectl-syncpod spins up a temporary helper pod that:

• Mounts your target PVC
• Runs an sshd server with an in-memory public key
• Listens on a randomized NodePort
• Accepts connections only via a secure, ephemeral SSH private key (never written to disk)

The CLI then:

• Uses an in-memory SFTP client to recursively transfer files
• Skips files that are already present and match by SHA-256
• Cleans up the helper pod and service automatically

---

Comparison Table

Feature	kubectl cp	kubectl exec	kubectl-syncpod (SFTP mode)
Uses sidecar or helper pod	No	No	Yes
Works with PVCs	Partial - Only if mounted in container	Partial - Manual path required	Yes - Helper pod mounts PVC
Requires tools in container (tar, sh)	Yes	Yes	No - uses sshd in helper pod
Supports readOnlyRootFilesystem pods	No	No	Yes
Works on distroless/scratch images	No	No	Yes
Affects main application container	Yes	Yes	No
Requires container to run as root	Often yes	Often yes	No or configurable via helper pod spec
Safe for production workloads	Risky	Risky	Yes - safe for read
Auto-cleans after sync	No	No	Yes
Supports concurrent transfers	No	No	Yes - parallel SFTP workers
Performance on large file trees	Slow	Slow	Fast - streaming + concurrency

---

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.