feat: add daytona sandbox provider (RFC, narrow scope)

[zozo123]

Why now

Crabbox already supports Hetzner, AWS EC2, blacksmith-testbox, and static SSH. Daytona Sandboxes give Crabbox a fifth direct backend for fast, label-tracked Linux sandboxes via a documented REST API and short-lived SSH access tokens.

I'm submitting this as a draft RFC. Two prior third-party-provider PRs (#16, #24) were closed because they were "broad" and you'd preferred a "fresh, smaller design" if revisited. This PR is intentionally narrow, and I'd rather get a thumbs up or a clean close on scope before un-drafting.

Scope (deliberately small)

In:

• provider: daytona direct local provider (no broker, no coordinator).
• Linux only.
• Reuses existing rsync/run path — no new sync mode.
• Class -> Daytona resource mapping (standard->small, fast->medium, large/beast->large) with optional cpu/memory/disk overrides.
• Daytona REST endpoints: GET /sandbox, POST /sandbox, GET/DELETE /sandbox/{id}, POST /sandbox/{id}/{start,stop}, PUT /sandbox/{id}/labels, POST /sandbox/{id}/last-activity, POST /sandbox/{id}/ssh-access?expiresInMinutes=N.
• Auth strictly via env (DAYTONA_API_KEY / DAYTONA_JWT_TOKEN, optional DAYTONA_ORGANIZATION_ID, optional DAYTONA_API_URL); per AGENTS.md, never in repo config.
• Token-bearing SSH (ssh <token>@ssh.app.daytona.io); SSHTarget.Key becomes optional and -i is omitted when empty.
• Status/list/timing JSON redact the SSH user to <token> so credentials don't leak.

Out (explicit, can revisit later):

• Worker/coordinator integration.
• VNC, WebVNC, desktop, screenshots.
• Windows/macOS targets, Tailscale.
• Daytona devcontainers/exec/code_run paths.

Architecture

Modeled on internal/cli/hcloud.go (HTTP client + Server abstraction + labels), not on internal/cli/blacksmith.go (subprocess wrapping), because Daytona exposes a REST API and labels natively. Reuses directLeaseLabels, touchDirectLeaseLabels, summarizeJSON, claimLeaseForRepoProvider, resolveLeaseClaim, leaseProviderName, allocateDirectLeaseSlug, serverSlug, findServerByAlias-style resolution.

Dispatch sites added alongside existing branches in:

• run.go (warmup, run, acquire, findLease, stop, deleteServer, touchDirectLeaseBestEffort)
• pool.go (list)
• status.go (status JSON, with token redaction)
• doctor.go (env+API ping)

ssh.go's sshBaseArgsWithOptions now omits both -i and IdentitiesOnly=yes when SSHTarget.Key == \"\". The Key field gets a doc comment noting empty is valid for bearer-token auth.

Verification

• `go vet ./...` clean.
• `go test -race -count=1 ./internal/cli/...` passes (`5.5s`), including 23 new daytona tests covering: list filter JSON encoding, auth/org headers, create body shape, SSH-access token mint, error wrapping via summarizeJSON, class mapping, cpu/memory/disk defaults+overrides, target validation, label round-trip, list output redaction, and the new SSH `-i`-when-empty behavior.
• `go build -trimpath ./cmd/crabbox` succeeds.
• `crabbox doctor --provider daytona` correctly reports missing creds and exits non-zero, then validates auth+reachability when creds are set.
• `crabbox warmup --help` surfaces every `--daytona-*` flag.

Pre-merge prerequisites for users

Daytona only — no Crabbox-side changes:

1. A Daytona API key with sandbox lifecycle scope (`DAYTONA_API_KEY`).
2. A `crabbox-ready` snapshot in the user's Daytona workspace containing `git`, `rsync`, `tar`, an OpenSSH server, and a non-root `daytona` user with sudo. Snapshot name is configurable via `daytona.snapshot` / `CRABBOX_DAYTONA_SNAPSHOT`.

Both are documented in `docs/features/daytona.md`.

Test plan

• CI (Go vet + race tests) green
• Manual smoke after rotating credentials: `crabbox doctor --provider daytona`, `warmup`, `run --id -- echo ok`, `ssh --id `, `stop`
• Confirm scope is acceptable before un-drafting (closing this RFC at any point is fine — branch stays on my fork)

[zozo123]

Rebased onto current main (24d4c7a) — no longer conflicting. Ready for a scope review whenever you have a moment, @steipete / @vincentkoc.

Local verification on this commit:

go vet ./...                         clean
go test -race -count=1 ./internal/cli/...   ok (4.7s, +23 daytona tests)
go build -trimpath ./cmd/crabbox     ok
gofmt -w (git ls-files '*.go')       no changes

CLI wiring smoke (no creds set):

$ crabbox doctor --provider daytona
ok      git      ...
ok      rsync    ...
failed  daytona  DAYTONA_API_KEY or DAYTONA_JWT_TOKEN is required

crabbox warmup --help surfaces every --daytona-* flag and the --provider help string lists the new value.

Live API smoke (warmup → run → ssh → stop) requires a DAYTONA_API_KEY plus a crabbox-ready snapshot in the user's Daytona workspace; both are user-side prerequisites documented in docs/features/daytona.md. I'm holding off on running it from a maintainer key for obvious reasons.

Happy to keep this in draft for a scope discussion, or convert to ready-for-review on your signal — your call.

[zozo123]

Live API verification complete

Ran the full lifecycle against the real Daytona API (https://app.daytona.io/api) on commit 7b08d93. Three real bugs surfaced and were fixed; the rebased PR now reflects all of them.

Bugs caught and fixed by the live smoke

1. POST /sandbox rejected cpu/memory/disk together with snapshot with "Cannot specify Sandbox resources when using a snapshot". The create body now omits cpu/memory/disk whenever daytona.snapshot is set; resources only ride along when no snapshot is provided. Documented in the helper comment and covered by TestAcquireDaytonaCreateBody_SnapshotOmitsResources / _NoSnapshotSendsResources.
2. crabbox warmup printed the live SSH access token to stdout (ready ssh=<token>@ssh.app.daytona.io:22). Added a single redactedSSHUser(cfg, server, target) helper, used at run.go:119 and in status.go, with TestRedactedSSHUser covering both cfg.Provider==daytona and server.Provider==daytona paths.
3. workroot was reported as /work/crabbox (the global default), not the daytona-specific path, because cfg mutations inside acquireDaytona did not propagate back to the warmup caller. applyResolvedServerConfig now picks up cfg.WorkRoot from the work_root server label (covered by TestApplyResolvedServerConfig_PicksUpWorkRootLabel).
4. Network classification claimed tailscale because the existing heuristic assumed any target.Host != server.PublicNet.IPv4.IP meant tailnet. Daytona uses a public relay, so the heuristic now short-circuits for daytona and reports network=public.

Verified output (key + tokens scrubbed)

$ crabbox doctor --provider daytona
ok      daytona  crabbox_sandboxes=0 auth=DAYTONA_API_KEY org=unset base=https://app.daytona.io/api snapshot=daytonaio/sandbox:0.4.3 target=- default_class=large

$ crabbox warmup --provider daytona --class standard --ttl 8m --idle-timeout 3m
provisioning provider=daytona lease=cbx_07dcd7b5c426 slug=silver-barnacle class=small snapshot=daytonaio/sandbox:0.4.3 target=- from-snapshot keep=true
provisioned lease=cbx_07dcd7b5c426 sandbox=b367...82b9 state=started class=small
leased cbx_07dcd7b5c426 slug=silver-barnacle provider=daytona server=b367...82b9 type=small ip= idle_timeout=3m0s expires=2026-05-05T09:21:57Z
ready ssh=<token>@ssh.app.daytona.io:22 network=public workroot=/home/daytona/crabbox
warmup complete total=4.49s

$ crabbox status --provider daytona --id silver-barnacle --json
"provider": "daytona", "state": "leased", "network": "public",
"sshHost": "ssh.app.daytona.io", "sshUser": "<token>",

$ crabbox stop --provider daytona silver-barnacle
deleted daytona sandbox=b367...82b9 lease=cbx_07dcd7b5c426

Still user-side prerequisites (unchanged)

• A crabbox-ready Daytona snapshot containing git, rsync, tar, sshd, and a non-root daytona user with sudo. The smoke above used daytonaio/sandbox:0.4.3 as a stand-in to validate the CLI path; the snapshot in daytona.snapshot defaults to crabbox-ready for prod use.
• DAYTONA_API_KEY (or DAYTONA_JWT_TOKEN) and optional DAYTONA_ORGANIZATION_ID in env. Never in repo config.

Quality gates on 7b08d93: go vet clean, go test -race -count=1 ./internal/cli/... ok in 4.4s (now 27 daytona tests including the 4 regression tests added from live findings), gofmt -w no changes, build OK.

Happy to convert from draft to ready-for-review on your signal, @steipete.

[zozo123]

Follow-up fixes pushed in 00be307.

Addressed the review blockers:

• provider=daytona now bypasses coordinator selection via newTargetCoordinatorClient, so logged-in Crabbox users still use the direct Daytona API path.
• ReplaceLabels now sends Daytona’s official SandboxLabels request shape: {"labels": {...}}, matching the OpenAPI schema and Go SDK wrapper.
• Reusing an existing stopped Daytona sandbox now calls StartSandbox and waits for started before minting SSH access.

Added regression coverage for all three: coordinator bypass, official label wrapper payload, and stopped-sandbox reuse before SSH token creation.

Local verification on the pushed commit:

gofmt -w internal/cli/daytona.go internal/cli/daytona_test.go internal/cli/target.go
go test -count=1 ./...
go test -race -count=1 ./internal/cli/...
go vet ./...
go build -trimpath ./cmd/crabbox
git diff --check

All passed. Scope remains unchanged: direct Daytona provider only, no Worker/coordinator, VNC, Windows/macOS, Tailscale, or CLI wrapper expansion.

[steipete]

Thanks for the careful follow-up and the live Daytona smoke. I’m leaving this as requested changes for now because the current merge result is not narrow anymore.

I fetched GitHub’s merge ref for this PR against current main (0bb34bd) and compared the actual merge result:

git fetch origin pull/32/merge:refs/tmp/crabbox-pr-32-merge
git diff --stat main..refs/tmp/crabbox-pr-32-merge

That merge result is 61 files changed, 1866 insertions(+), 3163 deletions(-), and it would delete or rewrite unrelated current-main code/docs, including:

D docs/commands/code.md
M docs/commands/webvnc.md
D docs/features/image-bake-runbook.md
D docs/features/prebaked-images.md
D internal/cli/code.go
M worker/src/fleet.ts

So before we can review/approve CI, please rebase onto current main and keep the diff limited to the Daytona provider surface. Expected shape is roughly the new Daytona provider + tests + provider/docs/changelog wiring, without removing the current code/webvnc/image-bake work.

I’m intentionally not approving the pending fork CI run while the merge ref has this unrelated rollback shape.

[steipete]

I did a deeper pass on the Daytona-specific code as well. Separate from the stale merge-ref/rebase blocker above, I found two things worth fixing before this should be considered ready:

1. The Daytona resource/class flags are currently documented and exposed, but the implementation makes them effectively unreachable/no-op.

acquireDaytona rejects an empty snapshot up front:

if daytonaSnapshot(cfg) == "" {
    return ..., exit(2, "provider=daytona requires daytona.snapshot or CRABBOX_DAYTONA_SNAPSHOT")
}

Then later it only sends cpu/memory/disk when body.Snapshot == "". Because the empty-snapshot case already returned, that branch cannot run. The test named TestAcquireDaytonaCreateBody_NoSnapshotSendsResources also now asserts the early error rather than proving resources are sent.

That means --class, --type, --daytona-cpu, --daytona-memory, and --daytona-disk look functional but do not affect creation with the default/required snapshot path. Daytona’s SDK docs also distinguish this: resources exists on create-from-image params, while create-from-snapshot params do not list resources.

Best fix: either drop/hide the resource override surface from this first snapshot-only PR, or add the proper image/base-image creation mode with the correct request shape. I’d strongly prefer the former for this narrow PR unless we know we need image creation now.

2. JWT auth should require DAYTONA_ORGANIZATION_ID locally.

The docs added here say the org ID is “only required for JWT auth”, and Daytona’s SDK docs say organizationId is required when a JWT token is provided. But newDaytonaClient currently accepts DAYTONA_JWT_TOKEN with no org ID, so doctor --provider daytona will proceed to an API request and fail later/less clearly. Please fail fast when using DAYTONA_JWT_TOKEN without DAYTONA_ORGANIZATION_ID, and add a focused test next to TestNewDaytonaClient_RequiresAuth.

Local proof I ran on the current PR head (00be307):

go test -count=1 ./...
go vet ./...
go test -race -count=1 ./internal/cli/...
go build -trimpath ./cmd/crabbox

Those pass, so this is not a compile/test-red problem; it is API-surface correctness before we bless the new provider.

[steipete]

Thanks! Will review.

[zozo123]

Rebased and pushed feat/daytona-provider onto current main (5aaa848). GitHub now reports the PR as mergeable, and the merge-ref diff is narrowed to the Daytona provider surface only: 24 files, 1752 insertions, 21 deletions.

Addressed the latest Daytona review items:

• Dropped the snapshot-incompatible resource override surface from this first pass: no --daytona-cpu, --daytona-memory, --daytona-disk, and explicit Daytona --class / --type now fail fast instead of looking functional while doing nothing. Docs now say the Daytona snapshot owns image and compute resources.
• DAYTONA_JWT_TOKEN now requires DAYTONA_ORGANIZATION_ID locally, with focused coverage next to the auth tests.
• Kept and rebased the earlier blockers: Daytona is a CoordinatorNever SSH lease backend, ReplaceLabels sends the official {"labels": {...}} wrapper shape, and stopped sandbox reuse starts/waits before minting SSH access.
• Integrated Daytona into the provider backend registry (internal/providers/daytona) instead of adding old command-level provider branches.

Verification on pushed commit b8f08f2:

go test -count=1 ./...
go test -race -count=1 ./internal/cli/...
go vet ./...
go build -trimpath ./cmd/crabbox
git diff --check origin/main...HEAD

All passed. gh pr checks currently reports no checks for this branch.