Repository navigation
feat(protocol): yatf/1's spec, models, reader and adapter framework (Y1a, FD-6075) - #748
earakely-scale wants to merge 7 commits into
Conversation
…Y1a, FD-6075) Add agentenv_protocol.yatf: the yatf/1 spec (SPEC.md, version 1.0), its event models, a strict JSON schema generated from them and checked in, reader helpers, the adapter framework and the runner that feeds records through an adapter, plus the design's three worked examples, corrected, as the spec's fixtures. Nothing serves or reads yatf/1 yet. - Models are one tolerant tree: readers keep fields, event and part types and vocabulary values this version doesn't define, and write them back out unchanged. The checked-in schema is strict for producers. - Spec 1.0 defines the types slice 1 produces and reserves delegation, context, producer, activation and usage.checkpoint. - An adapter is an Adapter subclass declared with @yatf_adapter, whose @Native, @Aux and @finish methods write through a YatfTurn that stamps the envelope. One instance serves an A2A context; AdapterRun and replay make the same events from the same records, live or offline. - The package needs only the protocol's base dependencies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found by chaos runs against the yatf package: - Models are strict: "5" or true is no longer coerced into an integer field, and NaN and infinity are refused, so whatever parses dumps back exactly. Cost fields take an int or a float, so 0 stays 0. - dump_event no longer encodes strings, so a lone surrogate in an unknown field or an Any value doesn't raise; dump_events escapes to ASCII, as the SDK writes native logs. - parse_events raises YatfError for input that isn't UTF-8, and refuses NaN and Infinity tokens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ands out copies From review: - YatfTurn.extension keeps every key of its body, None included, so raw(None) keeps its record; reasoning always writes text, null when it was withheld. - A NaN or infinity anywhere in an event is refused: parse_event checks unknown fields and Any values too (1e400 reads as infinity), and the builder refuses an event JSON can't write as it's emitted. - YatfTurn and AdapterRun keep events as JSON text and hand out fresh copies, so a caller editing a returned event can't change the stream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A chaos run found that a field name holding a lone surrogate at a typed level is refused, while values may hold one. The spec now says so and a test pins it: a native log that cut an emoji in half still reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review: the rule against lone surrogates covers the names of an event's fields and of the objects the spec defines, known or not, not keys inside free JSON (arguments, x, a raw record). The test now pins a surrogate key under x as kept too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… runner From a maintainability pass over the yatf package: - runner.py now holds TurnContext, YatfTurn and the frame it writes, beside AdapterRun, which owns the one event buffer the turn writes into. adapter.py is only the authoring API, so nothing reaches into another module's private methods. - YatfTurn's _emit takes the body as a dict and the rest by keyword. Whether a None is written as null follows the model: a required field or an extension's own key keeps it, an optional field leaves it out. Reasoning.text is required, so withheld text is always null. - Builder methods can set is_copied_context, ATIF's flag for replayed context, which no adapter could set before. - EVENT_MODELS and CORE_TYPES come from the models' own type literals, MEDIA_KINDS from MediaPart's Literal, and the two model configs are defined once. VOCABULARIES lists every vocabulary. - New guards: every builder method takes every field of its model and every envelope field an adapter may set; SPEC.md names every type and vocabulary value; a stale schema fails with the command that regenerates it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… no more From review: AdapterRun hands each turn its one shared buffer, so a closed turn's events ran on into later turns. The turn now records where it ended, its events stop there, and writing through a closed turn, such as one an adapter kept from an earlier handler, raises instead of landing in another turn with the wrong seq. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Closing in favor of ATIF. On 2026-11-17 we decided to record agent trajectories as ATIF v1.8, Harbor's format, with an optional Why: stored yatf/1 was 11 to 21% larger in tokens than ATIF carrying the same information, most of its lead over ATIF was typed fields ATIF can hold in A new Y1a PR (FD-6075) vendors Harbor's ATIF models as |
Why
FD-6074 has every agent serve yatf/1, one validated trajectory format, beside its native trajectory. Y1 (FD-6075) puts the format itself into
agentenv_protocol, in two PRs.This is Y1a: the spec, the models, the strict schema, the reader and the adapter framework. Y1b adds the validator (V1 to V12), its CLI and the round-trip kit. Nothing serves or reads yatf/1 yet, so no existing behaviour changes.
What changes
New package
agentenv_protocol.yatf, which needs only the protocol's base dependencies, not the agent SDK:SPEC.md: the yatf/1 spec, version 1.0. It covers:It's written from the settled design and the FD-6074 decisions of 2026-11-17. Spec 1.0 defines the 13 types slice 1 produces, and reserves
delegation,context,producer,activationandusage.checkpoint.models.py: one tolerant pydantic tree (extra="allow"). A reader keeps fields, event types, part types and vocabulary values it doesn't know, and writes them back out unchanged. It takes values exactly as given:"5"andtruearen't integers);So whatever parses dumps back byte for byte. A namespaced type (
<namespace>/<name>) parses as an extension event; any other unknown type parses asUnknownEvent.yatf-1.schema.json: the strict JSON schema of one event, generated from the models and checked in. It forbids unknown fields (except in an origin,xand an extension's body), unknown types and values outside each vocabulary.python -m agentenv_protocol.yatf schemaregenerates it.reader.py:(turn, seq);adapter.py: the authoring API.Adaptersubclass declared with@yatf_adapter(native_format=…, version=…, usage_input_includes_cache=…, tested_with=…, lossy=…). The decorator checks the declaration at import.@native(type=…, where=…)and@aux(name, …)methods take records, and@finishruns at turn end. Of the handlers that match a record, the most specific one takes it.<namespace>/raw; an aux record no handler takes is dropped.from_yatfreturnsNative(log, session).runner.py: running an adapter.AdapterRunfeeds one A2A context's records through an adapter, a turn at a time, with one instance per context. It writes each turn's frame (turn.start, the inbound message,final_response,turn.end).YatfTurn, which stampsseq,id,turn,observer, the actor, the attempt andderived_from. A builder method can also setis_copied_context.eventshold only its own.replayruns stored turns through the same code, so the same records make the same events.examples/: the design's three worked examples, corrected, as the spec's fixtures:turn.start, andusage_input_includes_cacheadded;total_stepscounts step events;tool_result.positiondropped;derived_fromgiven the new shape.Also changed:
jsonschemajoins the protocol'sdevextra, for the schema tests. That adds two lines touv.lock, written with the uv CI pins.How it was tested
Unit tests. 91 tests in
packages/agentenv-protocol/tests/test_yatf_*.py, offline.pytest packages/agentenv-protocol/tests: 1729 passed, 1 skipped.ruff check .is clean.Guards against drift. These tests fail CI when code and spec part ways:
SPEC.mdnames every event type and vocabulary value;EVENT_MODELS,CORE_TYPESandMEDIA_KINDSare derived from the models rather than listed again, and the models' two configs are each defined once.End to end, on two agent-environments-admin hosts (x86_64 and aarch64), at 0fc406b. 16 combinations: Python 3.10, 3.11, 3.12 and 3.13, each on pydantic 2.11.10 (the floor) and 2.14.3. In each one, the built wheel went into a fresh venv with base dependencies only (no
a2a-sdk, nomcp), and:python -m agentenv_protocol.yatf schemais byte-identical to the checked-in schema on both pydantic versions, so the regenerate-and-diff test holds at the floor too.tst/unit/task_step/verifiers/recorded/turn_claude_code.json) into yatf/1: 16 stream-json records became 23 events.Chaos. Two rounds, one seeded run per combination each: 240 s per run in round 1, 480 s in round 2. A round-2 run made about 86k random event mutations, 440k corrupted streams and 1.2k adapter contexts with 8.7k raising handlers. Every run also did a 64-context replay on 16 threads, 40k- and 200k-record turns, and records nested 900 deep. The invariants:
YatfError, and whatever parses dumps back byte for byte, through dicts and through JSONL.YatfError, within 2 s.Chaos found four bugs, all fixed here, each with a test:
"5"andtrueinto integers.UnicodeDecodeErrorinstead ofYatfError.dump_event.Greptile found four more, also fixed:
1e400read as infinity, then refused on save;eventsrunning on into later turns.After the later commits (the maintainability pass, the closed-turn fix), the real-turn e2e still gives the same 23 events, and a local chaos run finds no failures.
Two cases are refused on purpose, and the spec says so:
integermatches5.0);One flag in round 2 was timing under load, not a bug. The x86 run on 3.13 with pydantic 2.11 read 200k records 8.3× slower than 40k, against an 8× bound, while eight runs shared that host's 8 cores. Measured alone, reading is linear with the garbage collector off (5.0× for 5× the events, then 2.0× for 2×). CPython's cyclic collector adds the rest.
🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge; no blocking issues remain.Summary
Adds
yatf/1: its spec, models, strict schema, reader, adapter framework, and worked examples.Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR Records[Native and aux records] --> Adapter Adapter --> Turn[YatfTurn] Frame[Turn frame] --> Turn Turn --> Stored[Events stored as JSON text] Stored --> Pages[Fresh event copies] Stored --> Reader[Reader and step folding]Reviews (11) · Last reviewed commit: "fix(protocol): a closed yatf turn keeps ..." · Reviewed by Greptile