Skip to content

[RFC 005] 2/4: foundation types for agentic harnesses - #1098

Merged
burtenshaw merged 1 commit into
huggingface:mainfrom
splusq:rfc-005/pr2-harness-foundation-types
Sep 28, 2026
Merged

burtenshaw merged 1 commit into
huggingface:mainfrom
splusq:rfc-005/pr2-harness-foundation-types

Conversation

@splusq

@splusq splusq commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Stack for RFC 005 — 2 of 4. Depends on #1097.

Cross-fork PRs cannot chain bases, so this targets main and its diff includes #1097. Review the top commit only: 01bb0ce.

  1. 1/4 — package split ([RFC 005] 1/4: split openenv.core.harness into a package #1097)
  2. 2/4 — this PR: foundation types
  3. 3/4 — HarnessEnvironment + subprocess + tool bridge
  4. 4/4 — production /harness route + mode wiring

What

The type layer for wrapping an external agentic harness (Claude Code, OpenClaw, Codex) as an OpenEnv environment. Types plus unit tests only — nothing runs a harness yet, that is 3/4.

module contents
config.py HarnessConfig, HarnessTransport
events.py HarnessEventType, HarnessEvent, HarnessResponse, events_to_metadata()
adapter.py AgenticHarnessAdapter ABC + error hierarchy
tools.py resolve_tool_conflicts()

Deviations from the RFC text, and why

The RFC has drifted from the code in a few places. Flagging each rather than silently following either one:

  1. ToolDefinition does not exist. The type is Tool (env_server/mcp_types.py, name/description/input_schema). Reused rather than duplicated, so inject_tools takes list[Tool]. Same for RESERVED_TOOL_NAMES, which resolve_tool_conflicts re-checks as defense in depth even though MCPEnvironment.__init__ already validates.
  2. send_message() is concrete, not abstract. Streaming is the single abstract turn primitive; send_message() drains it and pulls the response/done off the terminal event. This removes the same boilerplate from every concrete adapter, and turns "the stream ends with TURN_COMPLETE" from a convention into an enforced contract — a stream that ends without it raises rather than silently yielding an empty response.
  3. events_to_metadata() is new. Observation.metadata gets serialized over the wire, so raw pydantic events in metadata["turn_events"] would not survive. This is the one sanctioned way to put events there; 3/4 uses it and asserts the result is json.dumps-able.

Naming

The ABC is AgenticHarnessAdapter because HarnessAdapter is taken by the rollout layer (see the question at the end of #1097 — if we rename that layer, this becomes plain HarnessAdapter, matching the RFC).

Needs a decision: session_timeout_s

The RFC's field comment says "Max time for a single session/episode" but its temporal-semantics section says it bounds one turn. I implemented and documented the per-turn reading, since an episode-wide bound is not enforceable by an adapter that only sees one turn at a time. Flagging explicitly for sign-off.

Verification

82 passed

test_agentic_harness_types.py covers config defaults/validation, event JSON round-trip, events_to_metadata serializability, all six resolve_tool_conflicts branches (passthrough, env_ prefixing with schema preserved, reserved-name rejection, duplicate rejection, and both ambiguity cases where the prefixed name is also taken), and the default send_message including the missing-TURN_COMPLETE error. Back-compat and rollout suites still green. Lint clean.


Note

Low Risk
New public types and re-exports only; no subprocess, routing, or changes to rollout/collect behavior.

Overview
Adds the RFC 005 type layer for turn-based external agentic harnesses (OpenClaw, Claude Code, etc.) alongside the existing trainer-side rollout API, without running a harness yet.

New modules define HarnessConfig / HarnessTransport, normalized HarnessEvent streams and HarnessResponse, AgenticHarnessAdapter (lifecycle, tool injection with list[Tool], streaming turns) with a concrete send_message() that requires a terminal TURN_COMPLETE event, harness-specific errors, events_to_metadata() for wire-safe observation metadata, and resolve_tool_conflicts() (prefix env_, fail on reserved/duplicate/ambiguous names). openenv.core.harness re-exports these symbols and documents both layers; AgenticHarnessAdapter stays distinct from rollout HarnessAdapter.

Unit tests cover config validation, event serialization, conflict resolution branches, default send_message behavior, and rollout back-compat.

Reviewed by Cursor Bugbot for commit a1ea5d3. Bugbot is set up for automated code reviews on this repo. Configure here.

@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@cursor

cursor Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Unable to authenticate your request. Please make sure to connect your GitHub account to Cursor. Go to Cursor

@burtenshaw burtenshaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. moving on to 3 and 4

Adds the type layer for wrapping an external agentic harness (Claude Code,
OpenClaw, Codex) as an OpenEnv environment. No runtime behavior yet — this
PR is types plus their unit tests.

- `config.py`: `HarnessConfig` / `HarnessTransport`. `session_timeout_s`
  is documented as bounding ONE conversational turn, per the RFC's
  temporal-semantics section (the field comment in the RFC is ambiguous;
  flagging for reviewer sign-off).
- `events.py`: `HarnessEventType` / `HarnessEvent` / `HarnessResponse`,
  plus `events_to_metadata()`, the sanctioned JSON-safe path for putting
  events into `Observation.metadata` so they survive wire serialization.
- `adapter.py`: `AgenticHarnessAdapter` ABC and its error hierarchy.
- `tools.py`: `resolve_tool_conflicts()` for the RFC's tool-name collision
  rules (`env_` prefixing, error on ambiguity).

Two deliberate deviations from the RFC text, both because the RFC is stale
against the code:

1. The RFC's `ToolDefinition` does not exist; the type is `Tool`
   (`env_server/mcp_types.py`), reused here rather than duplicated. Same
   for `RESERVED_TOOL_NAMES`, which `resolve_tool_conflicts` re-checks as
   defense in depth.
2. `send_message()` is concrete rather than abstract. Streaming is the
   single abstract turn primitive and `send_message()` drains it, which
   removes duplication from every concrete adapter and makes the terminal
   TURN_COMPLETE event an enforced contract instead of a convention.

The ABC is named `AgenticHarnessAdapter` to avoid colliding with the
rollout layer's existing `HarnessAdapter`. Worth discussing whether to
rename the rollout classes instead and reclaim the RFC's plain names --
see the PR description.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@burtenshaw
burtenshaw force-pushed the rfc-005/pr2-harness-foundation-types branch from 6e07658 to a1ea5d3 Compare September 28, 2026 09:58
@burtenshaw
burtenshaw merged commit 62c78c7 into huggingface:main Sep 28, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants