Skip to content

sandbox.run_shell leaks child process on timeout (host/native/container) #40

Description

@Samurai007AK

Summary

run_shell in apodex/sandbox.py:256-267 awaits the child with a bare asyncio.wait_for(proc.communicate(), timeout=timeout) — no TimeoutError handling, no proc.kill(), no proc.wait(). On timeout the caller sees TimeoutError, but the child process keeps running: unreaped (zombie) plus still consuming CPU and still mutating the workspace after the user was told it "timed out".

Location

apodex/sandbox.py:256-267:

proc = await asyncio.create_subprocess_shell(
    command,
    cwd=cwd,
    stdout=asyncio.subprocess.PIPE,
    stderr=asyncio.subprocess.PIPE,
)
out, err = await asyncio.wait_for(proc.communicate(), timeout=timeout)

Only the bwrap path (which delegates to sandbox.commands.run) reaps; the host/native/container path above — i.e. Linux default — is affected.

Repro (verified, real code)

Child sleeps 5s and then writes a marker file; caller uses timeout=1. If the child were reaped on timeout, the marker could never appear:

asyncio.run(run_shell(cmd, cwd, 1, Strategy(HOST, "test")))  # raises TimeoutError @ ~1.0s
time.sleep(7)
os.path.exists(MARKER)  # -> True  => child survived the timeout

Actual output from the verification run:

TimeoutError raised after 1.0s (expected)
marker exists: True
CONCLUSION: child kept running after timeout -> LEAK CONFIRMED

(The run also emits unclosed transport / Event loop is closed warnings at interpreter shutdown — further symptoms of the same leaked pipes/transports.)

Impact

  • Runaway processes: a timed-out pytest/npm install/fuzzer keeps burning CPU in the background with no handle to stop it.
  • Post-timeout workspace mutation: the user believes the command was killed, but it keeps writing files — stale/conflicting results for subsequent agent steps.
  • FD/pipe + zombie accumulation over a long session (transports are never closed, children never reaped).

Suggested fix

try:
    out, err = await asyncio.wait_for(proc.communicate(), timeout=timeout)
except (asyncio.TimeoutError, TimeoutError):
    try:
        proc.terminate()
        await asyncio.wait_for(proc.wait(), timeout=5)
    except (asyncio.TimeoutError, TimeoutError):
        proc.kill()
        await proc.wait()
    raise  # or return a structured timeout result, consistently

(terminate → kill fallback; then re-raise so callers keep seeing TimeoutError, or return a dedicated timeout tuple — whichever the maintainers prefer, applied consistently.)

Environment

  • Repo: ApodexAI/FrontierAgent, branch main @ b37b624
  • Path: apodex/sandbox.py:236-267
  • Reproduced on Windows host strategy; the missing-kill code path is platform-independent (host/native/container on Linux included)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions