Skip to content

e2e: close the firecracker and podman coverage gaps - #212

Merged
Shine-neko merged 5 commits into
mainfrom
test/firecracker-e2e-parity
Aug 3, 2026
Merged

e2e: close the firecracker and podman coverage gaps#212
Shine-neko merged 5 commits into
mainfrom
test/firecracker-e2e-parity

Conversation

@Shine-neko

@Shine-neko Shine-neko commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Closes coverage gaps on the runtimes nobody was testing. Every test here was run against real microVMs and containers.

The gap

e2e tests per runtime before this branch: docker 45, cloud-hypervisor 24, firecracker 9, containerd 6, podman 4.

What was added

t10_replicas.sh (firecracker) — fan out to 3 microVMs, assert 3 sockets and exactly 3 rootfs copies, converge back to 1, assert the surplus images are reaped, then delete and assert sockets and rootfs images reach zero.

t11_health_checks.sh (firecracker) — Firecracker implements neither execute_health_check nor the probes; it relies on the trait default resolving the guest through its instance_address override. Nothing proved that held. The test requires the probe message to come from health_probes::tcp_probe specifically, and asserts the loop keeps polling.

t5_podman_volumes_env.sh (podman) — bind mount readable inside the container, ro refusing a write (verified by absence of the file both in the container and on the host), and an environment variable reaching the process. Podman shares Docker's lifecycle, which is why this went untested — but bind mounts cross a user-namespace boundary when rootless.

setup.sh (firecracker) — fail fast when the binary lacks an effective CAP_NET_ADMIN. Without it every boot fails on TUNSETIFF and the deployment lands in crash_loop_back_off, which a test can only report as "did not reach running in 60s" — pointing at the runtime instead of the missing capability.

What these tests deliberately do NOT prove

Stated in each header so a green run is not over-read:

  • t10 does not prove one-VM-per-reconciliation-pass. It asserts the eventual count, which both the current implementation and the pre-Horizontal autoscaling, opt-in per deployment #211 one satisfy. Proving the pacing needs intermediate-state observation.
  • t10 does not exercise per-instance teardown. ring apply with a changed replica count replaces the deployment rather than resizing it, so the current.len() > desired branch is not reached — the same caveat CH's t9_scaledown documents.
  • t11 does not prove guest reachability. A refused connect proves the probe ran and a connect was attempted; it does not prove a packet reached the guest.
  • t11 does not assert the on_failure: alert event. It does fire, but not reliably within the test window, and I could not determine what governs the delay. Documented in the file and tracked on the board rather than asserted with a guessed timeout.

Not added

A containerd metrics test: setup.sh there requires root, so it could not be executed, and shipping an unrun test is worse than shipping none. Tracked on the board with the wider podman/containerd gap.

Verification

firecracker t1–t6, t8, t10, t11 and podman t1, t2, t4, t5 all pass locally against real VMs and containers. 824 unit tests unchanged.

Reviewed by codex over three rounds. It caught that t10 overstated what it proved, that my creating-phase explanation for the missing alert was factually wrong, that t11 accepted a scheduler-generated timeout message which would let a hang pass, and that my first cleanup trap would have deleted a developer's unrelated Ring containers. All fixed; final verdict was merge-approved.

@Shine-neko
Shine-neko merged commit 18b9183 into main Aug 3, 2026
5 checks passed
@Shine-neko
Shine-neko deleted the test/firecracker-e2e-parity branch August 3, 2026 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant