Repository navigation
Report an early exit 124 or 137 in the Docker backend instead of a timeout - #108
Open
willtong-aq wants to merge 4 commits into
Open
willtong-aq wants to merge 4 commits into
willtong-aq wants to merge 4 commits into
Conversation
The Docker backend runs each command under coreutils timeout(1) with --kill-after=5s and mapped both exit 124 and exit 137 to error_kind="timeout". timeout(1) exits 137 when the child dies by SIGKILL (or exits 137 itself), and it sends SIGKILL itself only after the limit. A 137 before the limit usually means a SIGKILL from elsewhere, for example the kernel out-of-memory killer, so the model was told "Command timed out after N seconds" for a command that was killed in under a second. Exit 137 is now a timeout only when the elapsed time is at least the timeout. The elapsed time is measured in the worker thread around exec_run, so the wait for a thread is not counted. An earlier 137 returns the real exit code, stdout and stderr plus a note: "Exit code 137 after X seconds, before the N second limit. This usually means the process was killed with SIGKILL, possibly by the out-of-memory killer." It sets no error_kind. format_result now also writes stdout in the error branch when it is not empty, so partial output of a timed-out command reaches the model. An error result with empty stdout formats as before.
Exit 124 was always reported as a timeout. A command can exit 124 before the limit by itself, for example when it runs its own nested `timeout 3 sleep 60` under a 60 second limit. The model was then told "Command timed out after 60 seconds" after 3 seconds, and the real exit code was hidden. Exit 124 is now a timeout only when the elapsed time is at least the timeout, the same rule as exit 137. An earlier 124 returns the real exit code, stdout and stderr plus a note: "Exit code 124 after X seconds, before the N second limit. The limit did not stop this command." It sets no error_kind.
The early exit 124 or 137 branch now appends its note to stderr and reaches the existing final return, instead of building its own result and writing a log line. The tests use a plain perf_counter patch, and the format_result test is a module-level function. The output is unchanged.
The perf_counter patch held two values, so the chown exec during provider teardown exhausted it. The error was hidden by the provider's generic exception handler. The patch now repeats the last value.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #107.
Summary
DockerCodeExecToolProviderruns commands undertimeout --kill-after=5s <N>sand mapped every exit 124 or 137 toerror_kind="timeout". Both can occur before the limit: a 137 usually means an outside SIGKILL, for example the kernel out-of-memory killer, and a 124 can come from a nestedtimeout. The model was told "Command timed out after N seconds" without the real exit code.format_resultalso dropped stdout from error results.Changes
error_kind. The elapsed time is measured in the worker thread aroundexec_run.base.pyformat_result: the error branch also writes<stdout>when stdout is not empty. An error result with empty stdout formats exactly as before.format_result.CHANGELOG.md: anUnreleased/Fixedentry.Before and after
echo allocating; python3 -c "x = b'a' * (1024 ** 3)",timeout=60, container limited to 256 MiB.Before:
After:
Validation
Against a real
python:3.12-slimcontainer limited to 256 MiB and 1 CPU:timeout 3 sleep 60withtimeout=60: exit 124 after 3.1 s, note "The limit did not stop this command."sleep 60withtimeout=5, and a command that ignores TERM withtimeout=3: still reported as timeoutsruff format --check,ruff checkandty checkpass.uv run --extra docker pytest tests/test_docker_execution.pypasses locally (17 passed).Limits
State.OOMKilledis container-wide.