Installed system service loses a boot race against Docker Desktop and stays failed

[omry]

Summary

On a Linux host whose Docker is provided by Docker Desktop (WSL 2 integration),
a system-scope install generates a systemd unit with no ordering dependency on
Docker at all. The unit starts right after basic.target, fails because Docker
is not yet available, and exhausts its entire restart budget in about one second
— roughly 20 seconds before Docker Desktop finishes wiring up the WSL
integration.

The service then stays failed until someone starts it by hand. It has never
once survived a boot on this host.

Environment

reploy   0.7.0.dev1 [08d1ee835c, built 2026-08-15 22:36:02 PST]
OS       Ubuntu 24.04.4 LTS (WSL 2), kernel 6.6.114.1-microsoft-standard-WSL2
systemd  255 (255.4-1ubuntu8.17)
docker   Docker Desktop 4.87.0 (236836), server 29.7.2
install  --scope system --to /opt/arbiter --service arbiter

There is no docker.service on this host — Docker Desktop provides the CLI and
socket through bind mounts instead:

$ systemctl show docker.service -p LoadState
LoadState=not-found

$ ls -l /usr/bin/docker
/usr/bin/docker -> /mnt/wsl/docker-desktop/cli-tools/usr/bin/docker

Observed

The generated /etc/systemd/system/arbiter.service has an empty [Unit]
ordering section:

[Unit]
Description=Reploy Docker service (arbiter)
# Managed-By: reploy
# Reploy-Service: arbiter
# Reploy-Target: /opt/arbiter
# Reploy-Compose-Project: arbiter-03ee1bb5

[Service]
Type=notify
...
Restart=on-failure

Boot timeline (journalctl -b, boot b1eb3e86):

Time	Event
22:50:11.875	arbiter.service starts (immediately after basic.target)
22:50:11.940	fail — exec: "docker": executable file not found in $PATH
22:50:12.786	5 restarts exhausted → Start request repeated too quickly (0.91s total)
22:50:33	Docker Desktop mounts cli-tools — 21.2s too late
22:50:34.167	retry (10s start-limit window elapsed)
22:50:35.270	fail — dial unix /var/run/docker.sock: connect: no such file or directory
22:50:36.229	exhausted again (2.06s total)
22:50:38	docker.sock bind-mount activates — 1.8s too late
+5h48m	recovered only by a manual sudo systemctl start arbiter.service

The same failure occurred on the three most recent boots (Aug 19, Aug 21,
Aug 23). Two distinct error signatures appear depending on how far Docker
Desktop got:

1. reploy service-container error: ... inspect Docker endpoint: exec: "docker": executable file not found in $PATH
   — /usr/bin/docker is a dangling symlink; the cli-tools mount does not exist yet.
2. reploy service-container error: ... failed to connect to the docker API at unix:///var/run/docker.sock; check if the path is correct and if the daemon is running: dial unix /var/run/docker.sock: connect: no such file or directory
   — CLI tools mounted, socket bind-mount not yet.

Expected

An installed service survives a host reboot without manual intervention when
Docker becomes available a normal amount of time into boot.

Cause

Two independent contributing defects.

1. Backend selection ignores the Docker runtime.
hostPlatform.installBackend (internal/dockerdeploy/platform.go) dispatches on
GOOS alone:

case "linux":
    return installBackendLinuxSystemd

So WSL + Docker Desktop gets the systemd backend, which assumes a
docker.service exists, rather than the Docker Desktop backend whose
reboot-resistance story is Docker's own restart policy (as recorded in
docs/archive/WINDOWS_PORT.md and docs/archive/MACOS_PORT.md). The codebase
already detects Docker Desktop — dockerRuntimeDockerDesktop in the same file —
but that detection never reaches backend selection.

2. Absence of docker.service is treated as "no gating needed".
inspectProviderInstallHostToolsWithV1
(internal/dockerdeploy/provider_install_host_tools.go) decides whether to emit
an ordering dependency by shelling out:

result.IncludeDockerUnit = tools.run(
    CommandSpec{Name: systemctlPath, Args: []string{"cat", "docker.service"}},
    RunOptions{Context: ctx},
) == nil

That command exits 1 here, so providerInstallSystemdFileV1
(internal/dockerdeploy/provider_install_systemd.go) emits an empty ordering
block:

dockerUnit := ""
if includeDockerUnit {
    dockerUnit = "Requires=docker.service\nAfter=docker.service\n"
}

The detection is technically correct — there genuinely is no docker.service to
depend on — but the fallback should be "Docker readiness must be established
some other way", not "no gating needed".

Amplifier: the restart policy is off by an order of magnitude.
Restart=on-failure with systemd defaults gives RestartSec=100ms,
StartLimitBurst=5, StartLimitIntervalSec=10s — a ~1 second total retry budget
against a ~22 second wait. That converts a transient unavailability into a
permanent failure.

There is also no readiness wait in the service path. --docker-timeout (default
5s) is a daemon responsiveness timeout, it is not passed in the generated
ExecStart, and it would not help signature 1 regardless — that fails at
exec.LookPath before any daemon contact.

Note for whoever picks this up

The obvious narrow fix — After= on the Docker Desktop WSL mount units — does
not work. Those units are synthesized from /proc/self/mountinfo and have no
unit file:

$ systemctl show 'mnt-wsl-docker\x2ddesktop-cli\x2dtools.mount' -p FragmentPath -p SourcePath
FragmentPath=
SourcePath=/proc/self/mountinfo

systemd does not know about them at dependency-resolution time, so the ordering
constraint is silently a no-op and the unit proceeds anyway.

Suggested fix

Either or both:

• Add a bounded Docker readiness wait in _service-container before the first
  Docker call, covering both the missing-executable and missing-socket cases.
• Scale the restart policy to real Docker Desktop startup time (e.g. a longer
  RestartSec with a StartLimitIntervalSec/StartLimitBurst pair that spans
  at least a minute), so a slow Docker start is recoverable rather than fatal.

Feeding the existing dockerRuntimeDockerDesktop detection into backend
selection and/or unit generation would address the underlying mismatch.