Repository navigation
feat(sandbox): add the sandbox socket and the E2B provider - #2154
Conversation
…d how they apply and differ The recipe contract is the notebook's: a frozen, validated document that names the image, the source commits, the patches, the kernel artifacts and the server arguments of one configuration, and the delta that turns one recipe into another. The measurement contract in the next commit builds on RecipeDelta, so the two land together; this commit is the recipe half, taken as written on the multi-branch development branch. The contracts package starts here with an empty export list; the next commit adds the measurement names. Co-authored-by: siliangchen-amd <SiLiang.Chen@amd.com> Co-Authored-By: Claude <noreply@anthropic.com>
… the two JSON Schemas The test request a multi-branch module hands to the toolbox and the test result it gets back, as frozen dataclasses that validate on construction and round-trip through to_document() and from_document(). The two JSON Schemas describe the documents of the hosted measurement: the spec the application writes into a sandbox and the result manifest the runner signs. The schemas ship as package data. hyperloom.contracts joins the top-level layer contract between agents and inference_optimizer: it builds on the optimizer's server-argument helpers (through the recipe contract) and nothing below imports it. Nothing on main imports the package yet; it is the vocabulary the sandbox, the measurement runner and the toolbox share. Co-Authored-By: Claude <noreply@anthropic.com>
The sandbox socket is the interface the measurement layer creates and drives sandboxes through: roles, create options, run settings, output sinks, and the refusals a provider reports (denied, gone, quota exceeded, unavailable). The E2B provider implements it over the official e2b SDK against any E2B-compatible service: one API key per role, control-plane calls retried with backoff while the failure is transient, a create retried only when it carries an idempotency key, and every SDK failure classified into the socket's exceptions. The e2b SDK is the new optional dependency group ``hosted``; the test extra includes it so the provider's failure-handling tests run in CI. hyperloom.sandbox leaves the list of packages that exist only on the development branch in the architecture guard, as hyperloom.hosted did; who may import it stays governed by the owners rule beside it, which also keeps ``import e2b`` inside sandbox/e2b.py. Nothing on main imports the package yet. Co-Authored-By: Claude <noreply@anthropic.com>
Code metrics gate: PASSEDJudged: the 6 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.
A12 Single-implementation abstraction (advisory, never fails): 2
Thresholds and sources: |
…scanner does not flag The generic-api-key rule of the secret scan read KEY_MEASUREMENT as a credential (a `*_KEY` constant holding a high-entropy string). The key is a fixture the provider tests pass to the SDK and then assert never appears in a repr, error or log line, so any distinct string serves; the new value is all zeros after the role name. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…ayer Co-Authored-By: Claude <noreply@anthropic.com>
Conflict in src/hyperloom/contracts/__init__.py (main added the package with a docstring only): resolved to the multibranch-dev version, which keeps main's package docstring and adds the measurement exports. Co-Authored-By: Claude <noreply@anthropic.com>
Updated to multibranch-dev e81281d: contracts/measurement.py, the measurement spec and result JSON Schemas, test_measurement_contract.py (recipe.py and test_recipe_contract.py were already identical). Added, because measurement.py now imports them (same layer, nothing else imported): contracts/_validate.py, consistency.py, long_context.py and their tests (test_consistency_contract.py, test_long_context_contract.py, test_long_context_bar.py, long_context_checks.py). Removed: the "contracts/*.schema.json" package-data line in pyproject.toml; main's "contracts/*.json" already ships the schemas. Co-Authored-By: Claude <noreply@anthropic.com>
…ayer Co-Authored-By: Claude <noreply@anthropic.com> # Conflicts: # src/hyperloom/plugs/tests/test_architecture.py
…branch-dev e81281d Takes multibranch-dev e81281d's version of every file this PR carries: - sandbox/socket.py, sandbox/__init__.py: new SandboxCreateFailed refusal (the platform made the sandbox and it failed before it ran). - sandbox/e2b.py: a create failure naming sandbox_create_failed or a workload in phase Failed is final, not retried; a 410 idempotency_key_spent answering a create's own replay is SandboxCreateFailed, a 410 on the first send stays SandboxGone. - sandbox/tests/test_sandbox_e2b.py: tests for both (42 tests). The one fixture value the secret scan flagged stays e2b_measurement_key_000000000000. No file was added or removed; pyproject.toml and test_architecture.py keep this PR's earlier change (hosted extra; hyperloom.sandbox leaves _DEV_ONLY). Co-Authored-By: Claude <noreply@anthropic.com>
…ner does not flag Brings test_sandbox_e2b.py up to multibranch-dev d8755f4. Co-Authored-By: Claude <noreply@anthropic.com>
The measurement contract (#2153) is on main now; the contracts take main's version. Co-Authored-By: Claude <noreply@anthropic.com>
…s instead of ellipses CodeQL flags each bare `...` as a statement with no effect; the docstring-only body is the shape the other protocols in the tree (hyperloom.plugs.events) use. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…ured Board hyperloom-multibranch #156 c5098 / c4103. A SaFE key used directly against the sandbox service, from a user who is in more than one workspace, has every create refused with 400 workspace_required: the create sends no agent-sandbox.io/workspace. A system-admin key always needs one; the platform gives no default. A Claw grant's key names its own workspace and is not affected. - E2BSandboxProvider takes workspace= (from_environ reads HYPERLOOM_SANDBOX_WORKSPACE; unset or empty names none) and adds it to every create's metadata as agent-sandbox.io/workspace. A workspace the caller's metadata names itself wins; the caller's mapping is not changed. Without one the metadata is exactly the caller's, as before. - A create refused with workspace_required raises SandboxDenied naming HYPERLOOM_SANDBOX_WORKSPACE, the role and what the create named, keeping the service's own message. It is not replayed. Other refusals are unchanged. - WORKSPACE_KEY joins the caller-written agent-sandbox.io keys in the socket. Tests against the canned control plane: the workspace goes with creates of both roles and a caller's own wins; from_environ sends one only when the variable is set (unset, as on Claw, sends the caller's metadata only); the refusal names the variable and is sent once; another 400 keeps its message. Co-Authored-By: Claude <noreply@anthropic.com>
…m asked The code-metrics gate refuses a comment that points at a PR or issue: the two comments added in 333ed66 named the board item. The history is in that commit's message. Co-Authored-By: Claude <noreply@anthropic.com>
chaojhou
left a comment
There was a problem hiding this comment.
PR #2154 -- hyperloom.sandbox: the sandbox socket and its E2B provider
What it does: adds hyperloom.sandbox, taken from multibranch-dev 018cccb:
socket.pyis the interface the measurement layer will create and drive sandboxes through: roles, options, results, and theSandbox/SandboxProviderprotocols with their refusals.e2b.pyis a provider over the officiale2bSDK, with one API key per role, bounded retries, and failure classification.pyproject.tomlgains an optionalhostedextra (e2b>=2.53.1,<3), which thetestextra now pulls in.
Nothing on main imports the package yet.
Risk: high -- pyproject.toml adds a third-party SDK dependency (e2b, plus native pyqwest, connectrpc, protobuf-py) to the test extra and a new hosted extra
Decisions for a human:
- pyproject.toml -- every test install now pulls
e2band its nativepyqwestwheel; licenses are MIT/Apache-2.0/BSD;e2b2.62 needs Python >=3.11, so 3.10 resolves to 2.53.x -- is a hosted-only SDK in thetestextra acceptable? - src/hyperloom/sandbox/e2b.py -- with
E2B_API_URLunset,from_environ(:343,_params:325-331) inherits the SDK default and sends the role key tohttps://api.e2b.app. A probe at head confirmedX-API-KEYreaching that host. This is the SDK's documented default and nothing on main reaches it today -- should the provider refuse a missing endpoint before the measurement layer wires it in? - src/hyperloom/sandbox/socket.py, init.py --
SandboxandSandboxProviderhave one implementation (advisory A12 from the gate). The docstrings namehyperloom.sandbox.localand the E2B stand-in, which are not in this PR, andSandboxErrorsays "three failure classes" where five are defined (socket.py:6-8, :60; init.py:4) -- keep them byte-identical withmultibranch-dev, as the author proposes, or reword them here?
Blocking issues: 1
-
[X2] The description's sizes, totals, test count and source commit predate the last two commits [verified]
Problem: the body still describes the first push:- body "Files (6 files, +1308 / -1)",
socket.py"(199 lines)",e2b.py"(529 lines)", the test file "(532 lines)"; - Tests: "42 provider tests ... 58 passed";
- Size triggers: "768 production Python lines", with the interface module at 199 lines and the provider at 529;
- Summary: byte-identical to
multibranch-dev40fefcc.
At 4b37717,
git diff --shortstatagainst main shows 6 files, +1438/-1, andwc -lgives socket.py 203, e2b.py 571 and test_sandbox_e2b.py 616, for 814 production lines. The paragraph added with f86d8e7/4b377175b already states 571/616/203, 66 tests and 018cccb, so the body contradicts itself.
Impact: a squash merge makes this body the commit message and the release-note source, so it records sizes, a test count and a source commit that the merged tree does not have. That includes the 768-line figure the size-trigger justification rests on.
Action: update the Summary, the file bullets, the test count and the size-trigger figures to the head numbers, and name 018cccb once. No code change. - body "Files (6 files, +1308 / -1)",
Checked: the 6 files at 4b37717. Every SDK name e2b.py uses (exception classes, Sandbox.connect/get_info/set_timeout/kill/list, SandboxQuery, ApiParams keys) against e2b 2.53.1 and 2.62.0. The ordering of _classify and _is_transient (the SandboxNotRunning and SandboxUnreachable subclasses of TimeoutException). The retry deadline, and that a keyless create is never replayed (retries=0 turns off the SDK's RetryableTransport). Key handling: provider repr, exception messages, tracebacks and SDK logging carry no key; an in-memory httpx request remains on __cause__. Import of hyperloom.sandbox and .socket without e2b installed. test_architecture.py: hyperloom.sandbox leaves _DEV_ONLY, stays in _HOSTED, and rule 2 keeps import e2b in sandbox/e2b.py, as #2135 did for hyperloom.hosted. The licenses of the new transitive dependencies. Independent readers: three lenses (shape, credentials, SDK behaviour and packaging), each with two adversarial verifiers per finding | Ran: test_sandbox_e2b.py against e2b 2.53.1 and 2.62.0 in fresh venvs (50/50 each); a probe with E2B_* unset and httpx.Client.send intercepted; git diff --shortstat and wc -l at head; git merge-tree against main c18af70 (clean); CI at 4b37717: 12 test shards, coverage on 3.10 and 3.11, code-metrics, pr-hygiene and import-linter succeed | Skill: 3eec7f38c6e3 | Model: claude-opus-5-5 | Base: f60df7f | Head: 4b37717
…later SandboxError and the socket's module docstring said "three" failure classes where five are defined. The docstrings that name the local implementation, the E2B stand-in and the contract suite now say those arrive in a later PR, since this branch does not carry them. Co-Authored-By: Claude <noreply@anthropic.com>
…efault Without api_url or sandbox_url the e2b SDK falls back to E2B_API_URL / E2B_SANDBOX_URL in the process environment and then to the public E2B service, so a provider built without them sent the role's key to https://api.e2b.app. Both URLs are now required keyword arguments; a blank one, or one from_environ cannot find, raises ValueError naming its variable before any request is made. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
PR #2154 -- hyperloom.sandbox: the sandbox socket and its E2B providerWhat it does: adds
Nothing on main imports the package yet. Risk: high -- pyproject.toml adds a third-party SDK dependency (e2b, plus native pyqwest, connectrpc, protobuf-py) to the
Blocking issues: none Checked: the fc65295 delta: LGTM |
|
@chaojhou thanks for the review. All four items are addressed at fc65295. CI is green, and the files are those of [X2] Stale description: rewritten at the head.
The owner (Yihan) decided the three questions:
Could you take another look? |
X2 fixed at fc65295; merging with the maintainer's consent.
main's sandbox package (#2154) and this branch's measure package are both on main now, so the architecture test's dev-only list keeps only hyperloom_multibranch. Co-Authored-By: Claude <noreply@anthropic.com>
Summary
Adds
hyperloom.sandbox: the sandbox socket, which is the interface the measurement layer creates and drives sandboxes through, and the E2B provider that implements it over the officiale2bSDK. The measurement layer and the toolbox both create sandboxes through this socket. Nothing onmainimportshyperloom.sandboxyet, so the change is inert until the measurement layer does.The package is taken from
multibranch-devat43dffa419and matches it except for docstrings:socket.py: theSandboxandSandboxProviderprotocol methods each have a one-line docstring here instead of a bare..., because CodeQL flagged every...as a statement with no effect (the docstring-only body is the shapehyperloom.plugs.eventsuses).__init__.py,socket.py,e2b.pyand the test file: where a docstring names the local provider (hyperloom.sandbox.local), the E2B stand-in service or the contract suite, it says here that they arrive in a later PR. Those docstrings are unchanged onmultibranch-dev, where those modules exist.Files (6 files, +1493 / -1):
src/hyperloom/sandbox/socket.py(203 lines): the roles (measurement,subrun,dev),CreateOptions,RunSettings,OutputSinks,SandboxInfo,CommandResult, theSandboxandSandboxProviderprotocols, the metadata keys, and the five refusals a provider reports:SandboxDenied,SandboxGone,SandboxCreateFailed,SandboxQuotaExceeded,SandboxUnavailable(all underSandboxError).src/hyperloom/sandbox/__init__.py(40 lines): re-exports the socket's names.src/hyperloom/sandbox/e2b.py(584 lines): the provider over thee2bSDK, against any E2B-compatible service (E2B_API_URL,E2B_SANDBOX_URL).api_urlorsandbox_urlis blank, orfrom_environfindsE2B_API_URLorE2B_SANDBOX_URLunset or empty, construction raisesValueErrornaming the variable before any request is made. Without this check the SDK would fill the URL in from the process environment or default to the public E2B service (https://api.e2b.app) and send it the role's key.HYPERLOOM_SANDBOX_KEY_<ROLE>), read once, and never logged or put in an error orrepr.workspace=;from_environreadsHYPERLOOM_SANDBOX_WORKSPACE). When one is set, every create sends it as the metadataagent-sandbox.io/workspace, unless the caller's metadata names a workspace, which wins. A create refused withworkspace_requiredraisesSandboxDeniednaming that variable. Without a workspace, creates are unchanged, since a Claw grant's key names its own.SandboxGone). A create failure namingsandbox_create_failed, or a workload in phaseFailed, is final and not replayed (SandboxCreateFailed), and so is a 410idempotency_key_spentanswering the create's own replay. A 429 naminggrant_quota_exceededmeans the session's quota is full (SandboxQuotaExceeded).src/hyperloom/sandbox/tests/test_sandbox_e2b.py(658 lines, 56 tests): the provider against a canned HTTP service. It covers a particular status code, a server error that clears on the third try, a create whose sandbox failed before it ran, a replay answered with a spent key, a data plane that answers 403 or cannot be reached, key redaction, the retry budget, the workspace on every create, and a missing or blank service URL refused before any request is sent.pyproject.toml(+7): the new optional dependency grouphosted(e2b>=2.53.1,<3, imported only byhyperloom.sandbox.e2b), added to thetestextra so the tests run in CI.src/hyperloom/plugs/tests/test_architecture.py(1 line):hyperloom.sandboxleaves_DEV_ONLY, ashyperloom.hosteddid in feat(hosted): add the MCP service core with token scopes and tool registration #2135. Without this, the package's own imports ofhyperloom.sandbox.socketfail the "main imports nothing dev-only" guard. Which modules may import sandbox code is still governed bytest_rule_2_no_module_imports_a_hosted_implementation_outside_its_owners, which also keepsimport e2binsidesandbox/e2b.py. This PR's code does not importhyperloom.contracts, and.importlinterhas no sandbox lines onmultibranch-dev, so none are added.What is not in this PR
The rest of
hyperloom/sandboxonmultibranch-devis left for a follow-up:local.py,local_sandbox.py,templates.py,gpu_pool.py,gated_pool.py,identity.py,launcher.py,node.py,reaper.py,card_occupancy.py,drm_fdinfo.py, with their tests (test_sandbox_local.py,test_card_census*.py,test_card_occupancy.py,test_gated_pool.py,kernel_tree.py). They import only each other and code that is onmain(hyperloom.common.outside_facts,hyperloom.common.env_safety).hyperloom.measure(the measurement runner and sandbox pool, which are not onmain):conftest.py,contract_harness.py,contract/,test_contract_harness.py,test_e2b_bootstrap.py,test_e2b_standin.py,test_sandbox_own_tmp.py, and the E2B stand-in service they use (tests/e2b_standin/).Because this PR has only one provider, the code-metrics gate prints two advisory A12 "single implementation" notes, for
SandboxandSandboxProvider. The local provider will be the second implementation.Review template
multibranch-devwith the code. At the merge ofmain0eda8afda,pytest src/hyperloom/sandbox src/hyperloom/plugs/tests/test_architecture.pypasses on Python 3.10 (e2b 2.55.1) and 3.11 (e2b 2.62.0) with onlysrcon the path: 56 provider tests and 16 architecture tests, 72 passed on each.socket.pyis 100% covered ande2b.py88% (30 of 260 statements missed), so the changed lines are above the 80% diff-coverage bar. Also run:ruff checkandruff format --checkon the changed Python files,scripts/check_broad_suppress.py src scripts, andlint-imports --no-cache(12 kept, 0 broken).e2b.py584,socket.py203,__init__.py40), above the 400-line review trigger and under the 1000-line budget. The change has two main modules: an interface module and a provider module that implements it. The provider is under the ~800-line module trigger, and the tests are the provider's own.mainimportshyperloom.sandbox, no CLI, entry point or flag uses it, and the legacy flow behaves exactly as before. The newhostedextra installs thee2bSDK for anyone who asks for it, and thetestextra now includes it.🤖 Generated with Claude Code