refactor(ai): fake/record swap the model, not the whole agent#187
refactor(ai): fake/record swap the model, not the whole agent#187tmgbedu wants to merge 2 commits into
Conversation
The fake, record, inline-fake, and real branches each duplicated the _log_call() + _apply_schema() tail. Resolve the source once via a Transport (live / stand-in / inline fake) selected in _transport(), so each public method has a single call path and logs/applies its schema in one place. Pure internal refactor: the public prompt/stream/fake/record API and the fake/record mechanism are unchanged.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
tmgbedu
left a comment
There was a problem hiding this comment.
[Code Reviewer verdict — GitHub blocks self-approval since the PR author and the CLI's authenticated account are the same; posting as a comment instead of a formal approval]
VERDICT: APPROVE
Verified independently (not just trusting the PR description):
Diff scope — confirmed only ai/agent.py (60 lines: 19+/41-) and new ai/transport.py (142 lines) changed vs main. No other files touched.
Resolution order — _transport() checks _faked() then _match_fake(), same order as the original branches in both prompt() and stream().
_log_call exactly once per call:
prompt(): single unconditionalself._log_call("prompt", message)after_transport(...).prompt(...)resolves, for all 3 paths.stream():self._log_call("stream", message)sits at the top of the generator body, unchanged position vs. original — called once regardless of path.- Directly exercised by
tests/ai/test_agent_fake.py::test_stream_records_one_call_not_twoandassert_prompted(times=N)tests — all pass.
_apply_schema in all 3 paths — called once after _transport(...).prompt(...) returns, for Live/StandIn/InlineFake alike. tests/ai/test_agent_schema.py exercises schema parsing through the fake path and the record (StandIn) path, including round-trip replay — all pass.
Stand-in path does NOT forward model/provider_options — confirmed: StandInTransport.prompt() accepts them in its signature (shared Transport interface) but its body only calls self._stand_in.prompt(message, attachments=attachments), matching the original stand-in branch exactly.
Other subtlety verified: InlineFakeTransport.prompt() forwards model/attachments/provider_options to AgentSnapshot.resolve(...) (matches original), while InlineFakeTransport.stream() calls match.resolve(self._agent, message) with no kwargs (also matches original) — this prompt/stream asymmetry was correctly preserved.
Self-executed test runs:
tests/ai/: 223 passed ✅- Full suite excl. Postgres: 1833 passed, 7 skipped, 4 subtests passed ✅
ruff checkon both changed files: clean ✅
No SQL/ORM concerns — pure Python control-flow refactor, no DB queries touched. Byte-for-byte behavior parity confirmed.
Agent.fake()/record() previously bound an entire stand-in Agent into the container, so a faked agent bypassed its own pipeline — instructions, middleware, tools, and schema never ran. Move the seam to the model: - ModelBuilder.get_model_for() returns a registered double if one exists for the agent class, else builds the real model. Agent._build_model() uses it. - fake(responses: list) swaps in fake_chat_model(responses); record(cassette, messages) swaps in a RecordingModel that replays a cassette or, on a miss, calls the real model once and records it. - prompt()/stream() now have a single call path — the agent always runs its real pipeline, only the model is doubled. Removes the whole-agent binding machinery (make/_binding/_faked, the inline _fakes/_match_fake path, FakeAgent/RecordingAgent/AgentBinding, and the Transport indirection) now that no branch remains to dispatch.
Summary
Reworks
Agent.fake()/Agent.record()so they swap the agent's model instead of binding an entire stand-inAgentinto the container.Under the old design a faked agent was replaced wholesale, so none of its own pipeline ran —
instructions(),middleware(),tools(), and_apply_schema()were all bypassed, andprompt()/stream()each fanned out into fake / inline-fake / real branches with duplicated_log_call+_apply_schematails. Moving the seam to the model means the agent always runs its real pipeline; only the LLM boundary is doubled.What changed
ModelBuilder.get_model_for(model, provider_options)— returns a registered double if one exists for the agent class, elsebuild(). A small class-level registry (register_fake/clear_fake) replaces the container binding.Agent._build_model()calls it.fake(responses: list)→AgentFake, a context-manager/decorator that swaps infake_chat_model(responses)(replays responses in order). No network, no agent swap.record(cassette, messages=None)→AgentRecordFake, swaps in a newRecordingModelthat replays a cassette or, on a miss, calls the real model once and records it.messagesseeds initial history (defaults to none = today's behavior). Cassette auto-resolution relative to the test file is preserved.prompt()/stream()collapse to a single call path — always the real pipeline.make(),_binding(),_faked(),_transport(),_match_fake(),_fakes,transport.py, andFakeAgent/RecordingAgent/AgentBinding/_Recorder. (Confirmed zero consumers outside theaimodule.)API change (intentional)
fake()now takes a list of responses (sequential) instead of a pattern→reply dict (fnmatch).Agent.make()and theFakeAgent/RecordingAgent/AgentBinding/NoFakeResponseexports are gone. Existingtests/ai/test_agent_fake.pyandtest_agent_schema.pyare rewritten to the new API.Tests
tests/ai/: 218 passedruff check src/ tests/cleanFollow-ups (not in this PR)
docs/ai/agent.mddocuments the oldfake({...})dict API andAgent.make()— needs updating.