Skip to content

Flaky: TestDockerExecInteractive fails on first CI run (exit 126), passes on re-run #405

Description

@dpup

Summary

TestDockerExecInteractive (in internal/container) is flaky in CI: it fails on the initial run of nearly every push and passes on re-run, with no change to the exec code path between attempts.

Failing subtests

  • TestDockerExecInteractive/TTY_output_contains_expected_string
  • TestDockerExecInteractive/exit_code_propagation_via_ExecError

Symptom

exec_interactive_test.go:54: ExecInteractive failed: exec failed with exit code 126
--- FAIL: TestDockerExecInteractive (10.21s)
FAIL	github.com/majorcontext/moat/internal/container

Exit code 126 is "command found but cannot execute" — consistent with a TTY/exec-setup race in the Docker-in-Docker CI environment (e.g. the container/exec not being fully ready when ExecInteractive runs), rather than a logic bug.

Frequency / evidence

On PR #404 the Test job failed on the first attempt of four consecutive pushes and passed on every manual re-run:

  • run 28187501415 (failed → re-run passed)
  • run 28186749229 (failed → re-run passed)
  • run 28138719182 (failed → re-run passed)

Not reproducible locally: the full internal/container race suite (go test -race ./internal/container/...) passes repeatedly on a local Docker-in-Docker setup.

Impact

  • Blocks the first CI Test run on essentially every push; requires a manual "re-run failed jobs".
  • Wastes CI minutes and erodes signal — a real failure in this job is easy to dismiss as "the usual flake".

Suggested fixes (pick one)

  1. Stabilize: wait for container/exec readiness before the interactive exec in the test (poll until the shell is execable), or add a bounded retry around the 126-class transient.
  2. Quarantine: tag the test (build tag or t.Skip under a MOAT_FLAKY=skip env / testing.Short()) until (1) lands, so CI stops false-failing.

Surfaced while landing #404 (volume-mode workspaces); this test is pre-existing and unrelated to that change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions