Repository navigation
feat(proxy): start a proxy a task image bakes in, and fetch less per install #578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
2f3b69b
feat(proxy): start a proxy a task image bakes in, and fetch less per …
earakely-scale d671138
fix(proxy): read an image's install extension once per artifact version
earakely-scale 8aa33ea
fix(proxy): pin the bake recipe's uv image to its digest
earakely-scale 5413265
test(run): signal a run mid-step once its command has started
earakely-scale 49c2b49
refactor(proxy): keep the proxy's versions and paths in versions.sh
earakely-scale 84f01a6
test(proxy): copy versions.sh with the install a container test runs
earakely-scale File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #!/bin/sh | ||
| # Bakes the proxy into an image as it's built, so an install into one of its containers only starts it: a private | ||
| # Python and a venv holding requirements.txt in /opt/agentenv-proxy, and a marker naming the requirements it holds, | ||
| # which install.sh checks. Plain sh, for images with no bash. A task image's Dockerfile ends with | ||
| # | ||
| # COPY --from=ghcr.io/astral-sh/uv:0.13.1@sha256:4a1f3666816dafadd20ceb71eac9a7d56d7ef30279f7d64048519ab901ada336 /uv /opt/agentenv-proxy/bin/uv | ||
| # COPY agentenv-proxy /opt/agentenv-proxy/src | ||
| # RUN sh /opt/agentenv-proxy/src/bake.sh | ||
| # | ||
| # where agentenv-proxy is this folder, copied into the image's build context. | ||
| set -eu | ||
|
|
||
| here=$(cd "$(dirname "$0")" && pwd) | ||
| . "$here/versions.sh" | ||
|
|
||
| export UV_PYTHON_PREFERENCE=only-managed UV_PYTHON_INSTALL_DIR="$BAKED/python" UV_CACHE_DIR="$BAKED/cache" | ||
| "$BAKED/bin/uv" venv --quiet --python "$PYTHON_VERSION" "$BAKED/venv" | ||
| "$BAKED/bin/uv" pip install --quiet --python "$BAKED/venv/bin/python" -r "$here/requirements.txt" | ||
| rm -rf "$BAKED/cache" | ||
| "$BAKED/venv/bin/python" -c 'import hashlib, sys; print(hashlib.sha256(open(sys.argv[1], "rb").read()).hexdigest())' \ | ||
| "$here/requirements.txt" > "$BAKED/baked" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # What install.sh and bake.sh share, sourced by both. A uv bump changes UV_VERSION, both checksums and UV_IMAGE_DIGEST, | ||
| # the digest of ghcr.io/astral-sh/uv:$UV_VERSION's image index, which the bake recipe pins. | ||
| UV_VERSION=0.13.1 | ||
| UV_SHA256_X86_64=6c86040a8f045ab9adc9e3ec4cdafa03fd00e9a988ed2bf28971c7fc2d0debd0 | ||
| UV_SHA256_AARCH64=345a8308e4d1835cdfaadf692836fd3137a6d45864773f05ad7bf59442b1ba16 | ||
| UV_IMAGE_DIGEST=sha256:4a1f3666816dafadd20ceb71eac9a7d56d7ef30279f7d64048519ab901ada336 | ||
| PYTHON_VERSION=3.12 | ||
| BAKED=/opt/agentenv-proxy # where bake.sh puts the proxy in an image |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| """An agent installed into a task container on the local sandbox, played from this machine through the proxy installed | ||
| in its place: an image with the proxy baked in only starts it, one without gets uv and a venv, and both play the turn | ||
| and pass the task's tests. Needs a Docker daemon, and network access for the images, uv, Python and the proxy's | ||
| requirements.""" | ||
|
|
||
| import shutil | ||
| import subprocess | ||
| import tempfile | ||
| import uuid | ||
| from pathlib import Path | ||
|
|
||
| import pytest | ||
|
|
||
| from agent_env.a2a_agent.a2a_agent import A2AAgent | ||
| from agent_env.artifact import DockerImageArtifact | ||
| from agent_env.artifact.store import reset_artifact_store | ||
| from agent_env.config import configure, reset_config | ||
| from agent_env.task import EpisodeSpec, Task | ||
| from tst.util.proxy_bake import BAKE_RECIPE, PROXY_DIR | ||
|
|
||
| pytestmark = [pytest.mark.integration, pytest.mark.int_test_slow] | ||
|
|
||
| TASK_DOCKERFILE = """FROM mirror.gcr.io/library/debian:bookworm-slim | ||
| RUN mkdir -p /work && echo hello > /work/greeting.txt | ||
| WORKDIR /work | ||
| """ | ||
| # An agent the proxy replaces in a container needs no more than a build context. | ||
| REPLACED_DOCKERFILE = "FROM mirror.gcr.io/library/python:3.12-slim\n" | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def local_backends(monkeypatch, tmp_path): | ||
| monkeypatch.chdir(tmp_path) | ||
| monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "state")) | ||
| monkeypatch.setenv("AGENT_ENV_DOCUMENT_STORE", "local") | ||
| monkeypatch.setenv("AGENT_ENV_OBJECT_STORE", "local") | ||
| monkeypatch.setenv("AGENT_ENV_LOCAL_SANDBOX_DIR", str(tmp_path / "sandboxes")) | ||
| configure() | ||
| reset_artifact_store() | ||
| try: | ||
| yield tmp_path | ||
| finally: | ||
| reset_artifact_store() | ||
| reset_config() | ||
|
|
||
|
|
||
| @pytest.mark.asyncio | ||
| @pytest.mark.parametrize(("baked", "proxy"), [(True, "baked"), (False, "installed")], ids=["baked", "installed"]) | ||
| async def test_an_agent_installed_into_a_task_container_is_played_through_the_proxy_in_its_place(local_backends, baked, | ||
| proxy): | ||
| uid = uuid.uuid4().hex[:8] | ||
| image = _task_image(local_backends / "task-image", uid, baked) | ||
| task = Task.from_dict({"id": f"proxy-local-{uid}", "steps": [ | ||
| {"id": "box", "type": "deploy_sandbox", "sandbox_name": "box", "sandbox_mode": "vm", "sandbox_type": "local", | ||
| "exposed_ports": [8000]}, | ||
| {"id": "task", "type": "run_docker_container", "sandbox_name": "box", "container_name": "task", | ||
| "image_artifact_id": image.id, "ports": [8000], "command_override": "sleep infinity"}, | ||
| {"id": "install", "type": "install_agent", "sandbox_name": "box", "container_name": "task", | ||
| "a2a_agent_id": _replaced(uid).id, "agent_name": "solver"}, | ||
| {"id": "solve", "type": "prompt_agent", "agent_name": "solver", "prompt_id": "solve", | ||
| "prompt": "Copy greeting.txt to answer.txt."}, | ||
| {"id": "check", "type": "run_container_unit_tests_verifier", "sandbox_name": "box", "container_name": "task", | ||
| "command": "grep -qx hello /work/answer.txt"}, | ||
| ]}) | ||
|
|
||
| try: | ||
| episode = await EpisodeSpec.from_task(task).start() | ||
| try: | ||
| turn = await episode.next_turn() | ||
| async with turn.tools() as tools: | ||
| where = "test -e /opt/agentenv-proxy-8000/bin/uv && echo installed || echo baked" | ||
| assert (await tools.call("bash", {"command": f"cat greeting.txt; {where}"})).text == f"hello\n{proxy}\n" | ||
| await tools.call("bash", {"command": "cp greeting.txt answer.txt"}) | ||
| await turn.end("Copied.") | ||
| assert await episode.result() == {"check": 1.0} | ||
| finally: | ||
| await episode.abandon("the test ended") | ||
| finally: | ||
| subprocess.run(["docker", "rmi", "-f", image.image_name], capture_output=True) | ||
|
|
||
|
|
||
| def _task_image(context: Path, uid: str, baked: bool) -> DockerImageArtifact: | ||
| context.mkdir() | ||
| bake = "".join(f"{line}\n" for line in BAKE_RECIPE) if baked else "" | ||
| (context / "Dockerfile").write_text(TASK_DOCKERFILE + bake) | ||
| if baked: | ||
| shutil.copytree(PROXY_DIR, context / "agentenv-proxy", ignore=shutil.ignore_patterns("__pycache__")) | ||
| return DockerImageArtifact.put_context(f"proxy-task-image-{uid}", description="a task container", | ||
| context_path=str(context), dockerfile_path="Dockerfile", platform=None) | ||
|
|
||
|
|
||
| def _replaced(uid: str) -> A2AAgent: | ||
| with tempfile.TemporaryDirectory() as build_dir: | ||
| context = Path(build_dir) | ||
| (context / "Dockerfile").write_text(REPLACED_DOCKERFILE) | ||
| image = DockerImageArtifact.put_context(f"replaced-agent-image-{uid}", description="a replaced agent", | ||
| context_path=str(context), dockerfile_path="Dockerfile", platform=None) | ||
| return A2AAgent.put(id=f"replaced-agent-{uid}", docker_image_artifact=image) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.