The single-shot model call is the defect: every role that answers questions about a repository must be a loop

[thedancingdeveloper]

Rewritten 2026-08-06 after the original framing turned out to be too narrow. The history is kept in the comments; what follows is the current statement of the problem.

The defect is the interaction model, not any one role

Every model interaction in this harness has the same shape:

assemble a context someone guessed at → one call → parse structured text → hope

That is planner, implementer, reviewer, surveyor, assessor and inception. Each is a model asked to answer questions about a repository from a snapshot it was handed, unable to look at anything it was not given and unable to check its own answer.

That is structurally incapable of doing what claude, codex and opencode do. No prompt or format work fixes it, and the attempt is documented: over 2026-08-05 the implementer's output format was repaired four times — unified diffs → edit blocks → indentation tolerance → quoting the file back on a failed match — and delivered nothing each time. Four rendering defects were then found in the edit-block path alone (#216). The format was never the problem.

The measurement

Same item, same models, same gateway. The only variable is whether the model can look:

	single-shot	a loop
turns	1	31
reached the check command	once in four passes	yes
cargo test	never green	passed

The correct model

A loop with tools, bounded by steps and spend, is the right interaction model for anything that answers questions about a repository.

role	today	under this model
implementer	one call over planner-chosen files	loop, write access
planner	one call guessing what the implementer will need	deleted — the loop looks for itself
surveyor	one call over whatever gather() guessed	loop, read-only
assessor (adopt)	one call asked to find evidence it cannot go and find	loop, read-only
reviewer	one call over a diff, inferring whether claims hold	loop, read-only

The reviewer is the clearest case. It rejected an item saying "its tests do not derive coverage from migration-tool or the legacy SQL fixtures" — a claim it could only infer from a diff. Given a read-only environment it would grep and know.

Read-only is not a detail. A gate with write access to the tree it is judging is a gate that can be talked out of a rejection.

What is built, not bought

mini-swe-agent supplies the loop (adopted in #212–#217, behind adapters/). What this repository builds is one agentic role runner: given a role, a task, an environment (read-only or not) and a termination condition, run a bounded loop and return its result.

One runner, five callers. That is a smaller surface than five bespoke prompt-and-parse paths, which is why this simplifies the harness rather than growing it.

What survives

ModelClient (role routing, fallback chains, the retry ladder, never retrying a spend cap, per-endpoint parking, recorded answers), protocols.py (wire shape per vendor), the queue and leases, holds, the graph, budgets.py, CommandGuard, the audit, and the gates themselves. This changes how a role thinks, not what is allowed to pass.

What is revealed as scaffolding

Machinery that exists only to prop up a single-shot model:

• the planner, and context pre-selection (The context budget is a target, not a ceiling: every item pays for the whole budget whatever it needs #152)
• edit blocks and to_diff (Tier 1 of this issue), the patch apply ladder
• structured-text parsing and its format-error retries

Not deleted yet. Removal follows evidence: once the loop has delivered items through the harness, not before. Deleting the only path that has tests in favour of one that has never run in-harness is the trade AGENTS.md rejects, and it is the same staging already agreed for --session-host.

Acceptance

• An item is delivered end to end by a role runner, through agent-harness run, with the queue, gates, audit, attempt record and reviewer all seeing it. No item has ever been delivered this way — every successful loop run so far went through a standalone script.
• The reviewer verifies a claim by reading the repository rather than inferring it from a diff.
• --session-host is deprecated in help text and docs (removal still gated on delivery evidence).
• rdpapp M2 is the test: T1 and T2 are the direct comparison.

The cost, stated

mini-swe-agent becomes a dependency of the core execution path, not an opt-in extra. A pip dependency is materially different from a user-installed binary — it ships with the stack — but this makes it load-bearing, and that is the trade being accepted.

Turn count also rises from 1 to ~30 per role, per item. Against a path that has delivered nothing, the cost per delivered item is currently infinite; but nothing measures this yet and it must be, because budgets.py bounds an item and step_limit bounds a loop and the two are different units (#215).

Sub-issues

• Build the agentic role runner, and put the implementer through it #215 — the role runner, and the implementer through it. First, and blocks the rest.
• Put the surveyor through the role runner: a plan should be written by something that read the repository #224 — surveyor
• Put the assessor through the role runner: adopt asks a model to find evidence it cannot go and find #225 — assessor (adopt)
• Put the reviewer through the role runner with a READ-ONLY environment, so it verifies instead of infers #226 — reviewer, read-only
• Retire the planner and context pre-selection once the loop has delivered items #227 — retire the planner and context pre-selection, on evidence

Blind spots

• No loop-executed item has ever reached the audit, so nothing is known about what its event stream looks like, or whether pricing attributes ~30 calls to one item correctly.
• Whether a read-only environment is enough to keep the reviewer honest is untested. It could still be argued into a pass by its own reading of the code; that is a prompt question, not an architecture one, and T43: A/B whether the reviewer seeing the plan changes its verdict #84 is the experiment.

[thedancingdeveloper]

Two clarifications to Tier 1

What Tier 1 does not fix. Worth stating before anyone reads it as more than
it is. The model still gets one shot, and still sees only the context the
planner selected in advance. If it needs a file it was not given, it is still
stuck. The difference is that it now fails honestly — "that text does not
occur in the file" — instead of silently emitting a hunk header whose
arithmetic is wrong. Reading on demand and correcting its own failed edit are
Tier 2's job, and Tier 1 should not be described as closing that gap.

Concretely, the three refusal rules, since "refused rather than guessed" is the
whole safety argument:

• not found — the model's mistake costs an attempt, never a corrupt tree;
• found more than once — text appearing twice is not a location, and taking
  the first match is how an edit lands in the wrong function and passes review
  because the diff looked plausible. The model is told to include more
  surrounding context to make it unique;
• any edit invalid — none are applied, because a half-applied set runs the
  checks against a tree nobody intended and shows the reviewer a diff nobody
  wrote.

This overlaps #187. Tier 1 resolves the edit path against the item's
worktree and refuses anything outside it, symlinks included — so a model naming
../../.ssh/id_rsa is refused by construction. That is a piece of the boundary
#187 asks for, arriving here for a different reason.

It is only a piece. #187 is about the agent's shell, which an in-process
loop does not remove: Tier 2 gains file tools, and a check command still runs
arbitrary commands. So #187 stays open, and this should not be read as
closing it — only as meaning the file-write half of it is already done for the
in-process path, and needs a matching answer for anything that executes.