Skip to content

refactor(ai): fake/record swap the model, not the whole agent#187

Closed
tmgbedu wants to merge 2 commits into
devfrom
task/agent-transporter-refactor-1138
Closed

refactor(ai): fake/record swap the model, not the whole agent#187
tmgbedu wants to merge 2 commits into
devfrom
task/agent-transporter-refactor-1138

Conversation

@tmgbedu

@tmgbedu tmgbedu commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Reworks Agent.fake() / Agent.record() so they swap the agent's model instead of binding an entire stand-in Agent into the container.

Under the old design a faked agent was replaced wholesale, so none of its own pipeline raninstructions(), middleware(), tools(), and _apply_schema() were all bypassed, and prompt()/stream() each fanned out into fake / inline-fake / real branches with duplicated _log_call + _apply_schema tails. Moving the seam to the model means the agent always runs its real pipeline; only the LLM boundary is doubled.

Supersedes the earlier Transport-strategy commit on this branch (that removed the branch duplication; this removes the branches themselves).

What changed

  • ModelBuilder.get_model_for(model, provider_options) — returns a registered double if one exists for the agent class, else build(). A small class-level registry (register_fake / clear_fake) replaces the container binding. Agent._build_model() calls it.
  • fake(responses: list)AgentFake, a context-manager/decorator that swaps in fake_chat_model(responses) (replays responses in order). No network, no agent swap.
  • record(cassette, messages=None)AgentRecordFake, swaps in a new RecordingModel that replays a cassette or, on a miss, calls the real model once and records it. messages seeds initial history (defaults to none = today's behavior). Cassette auto-resolution relative to the test file is preserved.
  • prompt() / stream() collapse to a single call path — always the real pipeline.
  • Removed: make(), _binding(), _faked(), _transport(), _match_fake(), _fakes, transport.py, and FakeAgent/RecordingAgent/AgentBinding/_Recorder. (Confirmed zero consumers outside the ai module.)

API change (intentional)

fake() now takes a list of responses (sequential) instead of a pattern→reply dict (fnmatch). Agent.make() and the FakeAgent/RecordingAgent/AgentBinding/NoFakeResponse exports are gone. Existing tests/ai/test_agent_fake.py and test_agent_schema.py are rewritten to the new API.

Tests

  • tests/ai/: 218 passed
  • Full framework suite (excl. live-Postgres): 1828 passed, 7 skipped
  • ruff check src/ tests/ clean

Follow-ups (not in this PR)

  • Docs repo docs/ai/agent.md documents the old fake({...}) dict API and Agent.make() — needs updating.

The fake, record, inline-fake, and real branches each duplicated the
_log_call() + _apply_schema() tail. Resolve the source once via a Transport
(live / stand-in / inline fake) selected in _transport(), so each public
method has a single call path and logs/applies its schema in one place.

Pure internal refactor: the public prompt/stream/fake/record API and the
fake/record mechanism are unchanged.
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 14 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...tapi_startkit/src/fastapi_startkit/ai/transport.py 71.11% 13 Missing ⚠️
fastapi_startkit/src/fastapi_startkit/ai/agent.py 90.90% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@tmgbedu tmgbedu left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Code Reviewer verdict — GitHub blocks self-approval since the PR author and the CLI's authenticated account are the same; posting as a comment instead of a formal approval]

VERDICT: APPROVE

Verified independently (not just trusting the PR description):

Diff scope — confirmed only ai/agent.py (60 lines: 19+/41-) and new ai/transport.py (142 lines) changed vs main. No other files touched.

Resolution order_transport() checks _faked() then _match_fake(), same order as the original branches in both prompt() and stream().

_log_call exactly once per call:

  • prompt(): single unconditional self._log_call("prompt", message) after _transport(...).prompt(...) resolves, for all 3 paths.
  • stream(): self._log_call("stream", message) sits at the top of the generator body, unchanged position vs. original — called once regardless of path.
  • Directly exercised by tests/ai/test_agent_fake.py::test_stream_records_one_call_not_two and assert_prompted(times=N) tests — all pass.

_apply_schema in all 3 paths — called once after _transport(...).prompt(...) returns, for Live/StandIn/InlineFake alike. tests/ai/test_agent_schema.py exercises schema parsing through the fake path and the record (StandIn) path, including round-trip replay — all pass.

Stand-in path does NOT forward model/provider_options — confirmed: StandInTransport.prompt() accepts them in its signature (shared Transport interface) but its body only calls self._stand_in.prompt(message, attachments=attachments), matching the original stand-in branch exactly.

Other subtlety verified: InlineFakeTransport.prompt() forwards model/attachments/provider_options to AgentSnapshot.resolve(...) (matches original), while InlineFakeTransport.stream() calls match.resolve(self._agent, message) with no kwargs (also matches original) — this prompt/stream asymmetry was correctly preserved.

Self-executed test runs:

  • tests/ai/: 223 passed ✅
  • Full suite excl. Postgres: 1833 passed, 7 skipped, 4 subtests passed ✅
  • ruff check on both changed files: clean ✅

No SQL/ORM concerns — pure Python control-flow refactor, no DB queries touched. Byte-for-byte behavior parity confirmed.

@tmgbedu
tmgbedu changed the base branch from main to dev July 17, 2026 18:34
Agent.fake()/record() previously bound an entire stand-in Agent into the
container, so a faked agent bypassed its own pipeline — instructions,
middleware, tools, and schema never ran. Move the seam to the model:

- ModelBuilder.get_model_for() returns a registered double if one exists for
  the agent class, else builds the real model. Agent._build_model() uses it.
- fake(responses: list) swaps in fake_chat_model(responses); record(cassette,
  messages) swaps in a RecordingModel that replays a cassette or, on a miss,
  calls the real model once and records it.
- prompt()/stream() now have a single call path — the agent always runs its
  real pipeline, only the model is doubled.

Removes the whole-agent binding machinery (make/_binding/_faked, the inline
_fakes/_match_fake path, FakeAgent/RecordingAgent/AgentBinding, and the
Transport indirection) now that no branch remains to dispatch.
@tmgbedu tmgbedu changed the title refactor(ai): route Agent.prompt()/stream() through a Transport strategy refactor(ai): fake/record swap the model, not the whole agent Jul 17, 2026
@tmgbedu tmgbedu closed this Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant