Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/system-specs/modules/acp-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -588,6 +588,43 @@ This leverages kiro-cli's `promptCapabilities.image: true` capability. The LLM r

The summary carries **no message content** — only counts, types, and sizes — which is a hard requirement (issue #6022): the kiro-cli data dir is fenced precisely because it holds SSO tokens, so the diagnostics must never record block text, image bytes, or tool arguments. This lets an operator tell a stale/invalid model id apart from a structurally malformed payload the next time a turn is rejected as `Improperly formed request` (see the `_RE_MALFORMED_REQUEST` classifier), without ever exposing what the turn contained. The helper is defensive by contract: it never raises into the live prompt path (a malformed block list yields a partial/minimal summary), so a diagnostics failure can never break a turn.

### Turn-boundary loss diagnostics (content-free)

Two places in `AcpSessionHandle.prompt` could destroy or omit a turn's evidence
silently. Both now report, and both report **only** a count or the bare fact —
never frame text, tool arguments, tool results, or frame SIZE, since a size leaks
response length.

- **Pre-turn stale drain.** The drain empties the session queue of frames left by
an abandoned turn (see the cancel-unacked / stale / tool-stall / timeout paths,
which synthesize a terminal and return while the real kiro-cli turn keeps
emitting). Permission REQUESTS are answered rather than dropped; everything else
is discarded, which used to happen with no count and no log. It now counts the
discarded frames and emits **one** WARNING per turn carrying that count — one
line regardless of how many frames drained, so a burst cannot flood the log.
This matters downstream: a turn whose terminal was destroyed here reaches the
dashboard as an empty response with no attributable cause. The count is NOT
bridged into `chat_runner` — see the note below.
- **A prompt stream that ends without a terminal.** `_dispatch_events`
synthesizes an `EVENT_COMPLETE` on every exit path it knows about, so a
consumer that never receives one is looking at a path that has none. The
generator warns when it exhausts CLEANLY having yielded no terminal.
"Cleanly" is what makes the line spam-free: a consumer close
(`GeneratorExit`), a cancellation, and any raised error all skip it, and each
is already logged by whoever caused it. The terminal is marked as delivered
BEFORE its `yield`, so a consumer that closes the stream on the terminal is not
reported as having lost it.

**Deliberately not bridged to the runner.** The drain count stays inside this
layer. Reaching `chat_runner` would mean a new field on the `AcpEvent` /
`LLMEvent` provider contract plus plumbing through `_dispatch` and the provider,
and `scripts/check_agent_sdk_boundary.py` baselines the dashboard modules at a
count that may not grow — a materially larger change than the fault it would
report. Unknown `sessionUpdate` discriminants in `_dispatch.py` are still ignored
silently for a related reason: `parse_session_update` is a pure function on a hot
path with no per-session state, so a bounded log there needs a dedupe set it does
not have, and an unbounded one would log per frame.


## AcpRuntime & AcpSessionHandle (session multiplexing)

Expand Down
48 changes: 47 additions & 1 deletion docs/system-specs/modules/session.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,8 @@ send time.
cancellation is treated as a transient provider failure and recovered
through a bounded three-rung ladder driven by `slot._empty_response_retries`:
1. **first empty** → the ORIGINAL message is silently re-queued at the
front of the slot queue (no visible card);
front of the slot queue (no visible card). Reached ONLY by a turn with no
activity — see the productive-turn exclusion below;
2. **second empty** (the same-message retry also produced nothing) → ONE
synthetic continue nudge (`_EMPTY_AUTO_CONTINUE_MSG` — a DIFFERENT
message, since re-sending the identical prompt tends to reproduce the
Expand All @@ -153,6 +154,51 @@ send time.
3. **third empty** (the nudge also produced nothing) → terminal notice card
asking the user to send a message; the counter resets so the next
genuine user turn gets a fresh budget.

**A PRODUCTIVE turn never reaches rung 1.** "Empty" at this branch means only
that the FINAL assistant segment is empty, which is not the same as "the turn
did nothing": `assistant_text` is reset at every tool boundary, so a turn that
streamed an answer and then called a tool arrives here with its answer already
flushed, persisted and on screen, and a tool-only turn arrives here having run
real side effects. Rung 1 re-queues the user's own message, so for either shape
it re-executes completed tool calls (a second `send_message`, a second write, a
second PR) and re-derives an answer the user has already read — observed in the
field as two consecutive billed `end_turn` turns, each with a preamble and
successful tool calls, both classified empty and the first verbatim-replayed.
`chat_utils.EmptyTurnActivity.productive` is the guard: a flushed visible
segment, a dispatched tool call, or thinking. A productive turn skips to rung 2,
which carries `_ACTIVITY_NO_REPLY_CONTINUE_MSG` instead — the same
`EMPTY_RESPONSE_RECOVERY_PREFIX` marker (so no new recovery card or locale pair
is needed) with a body that does NOT claim the turn produced nothing, because
that body is read by the model and would invite it to redo work whose side
effects already landed. Its notice card differs for the same reason. The ladder
bound is unchanged: a productive turn spends the same budget, it simply never
spends it on a replay. `_produced_visible_output` deliberately does NOT cover
this case — its narrow meaning (only the mid-turn resets that are not tool
boundaries: steer cut, compaction, clear, agent switch) is load-bearing for the
promise-only guard.

**Turn-end diagnostics.** The branch emits ONE privacy-safe WARNING per empty
verdict, after the rung is chosen, naming a closed `cause` and `rung` plus
booleans: `provider_empty`, `tool_only`, `thinking_only`, `visible_partial`,
`no_terminal_event`, `synthetic_completion` or `other`
(`chat_utils.classify_empty_turn`, ranked most-specific first), and `replay` /
`continue` / `give_up`. `EmptyTurnActivity` carries whether a terminal
`EVENT_COMPLETE` arrived, whether the provider SYNTHESIZED it, the terminal stop
reason normalised onto a closed set (`chat_utils.normalize_stop_reason` — an
omitted reason answers `absent`, which is a distinct observation from a clean
`end_turn` and must not be laundered into one, and an unrecognised backend
string answers `other` rather than being echoed), whether text streamed, whether
a visible segment was flushed at a tool boundary, whether tools ran, whether
thinking ran, and whether the provider reported ANY billing dimension. Every
field is a bool or a closed constant by contract: no prompts, responses,
thinking, tool arguments or results, paths, identities, token counts or costs.
The predecessor logged only `Empty model response (attempt N)`, which could not
separate a provider that generated nothing from a turn whose answer a tool
boundary flushed away from a turn no terminal event ever closed — three faults
with three different owners, and one field incident hit all three in three
consecutive attempts.

Recovery rungs 1–2 skip persistence/consolidation/success-recording (the
empty turn is never saved) and preserve all other retry budgets. Synthetic
recovery messages (`_SYNTHETIC_RECOVERY_MSGS`: the post-transient CONTINUE
Expand Down
52 changes: 52 additions & 0 deletions src/kiro_crew/acp/session_handle.py
Original file line number Diff line number Diff line change
Expand Up @@ -906,6 +906,15 @@ async def _run_turn(
# from an abandoned turn (or routed here for a backend child between
# turns) gets the fail-closed reject; the live turn's requests are
# handled by the dispatch loop as before.
# A DROPPED frame is invisible to every layer above: the abandoned turn's
# output vanishes here with nothing to show it existed, and a turn that
# loses its terminal this way reaches the dashboard as an empty response
# with no attributable cause. Count them and say how many, ONCE. Never
# what they were: a frame carries model text, tool arguments and tool
# results, and none of that belongs in a log — nor its size, which leaks
# response length. The count is bounded by the queue, and the log line is
# one per turn regardless of how many frames drained.
_stale_dropped = 0
while True:
try:
stale = self._queue.get_nowait()
Expand Down Expand Up @@ -979,6 +988,19 @@ async def _run_turn(
_stale_sid if _stale_sid != self._session_id else ""
),
)
else:
# Everything that is not a permission request is DISCARDED, which
# is correct (it belongs to a turn nobody is reading any more) but
# was silent. Count it.
_stale_dropped += 1

if _stale_dropped:
logger.warning(
"pre-turn drain discarded %d leftover frame(s) from a prior "
"abandoned turn on this session; those frames — possibly "
"including that turn's terminal — reached no consumer",
_stale_dropped,
)

self.last_prompt_stats = self.last_prompt_stats.carry_over()

Expand Down Expand Up @@ -1019,6 +1041,16 @@ async def _run_turn(
_mark(self._session_id, False)
raise

# Did a terminal reach the consumer, and did this generator finish of its
# own accord? Together these answer a question no layer above can: the
# dashboard reads "no EVENT_COMPLETE" as an empty response and cannot tell
# whether the backend never closed the turn or the consumer simply walked
# away. Only a CLEAN exhaustion is reported, which is what makes the
# warning spam-free: a consumer close (GeneratorExit), a cancellation, and
# any raised error all leave `_exhausted_clean` False and are already
# logged by whoever caused them.
_yielded_terminal = False
_exhausted_clean = False
try:
# Surface any drain-time rejections (see the pre-turn drain above)
# as crew-card activity before the turn's own events — the user
Expand Down Expand Up @@ -1052,6 +1084,11 @@ async def _run_turn(
# single choke point because `_dispatch_events` yields from 15
# places and every one of them funnels through this `async for`.
self._parked_since = time.monotonic()
if event.kind == EVENT_COMPLETE:
# Set BEFORE the yield: a consumer that closes the stream ON
# the terminal still received it, and marking it after would
# report a lost terminal that was in fact delivered.
_yielded_terminal = True
try:
yield event
finally:
Expand All @@ -1064,11 +1101,26 @@ async def _run_turn(
if self._parked_since is not None:
self._parked_total += time.monotonic() - self._parked_since
self._parked_since = None
# Reached only when the dispatch loop returned on its own — not on a
# close, a cancel, or an exception.
_exhausted_clean = True
finally:
if _mark is not None:
_mark(self._session_id, False)
if not self._turn_done.is_set():
self._turn_done.set()
if _exhausted_clean and not _yielded_terminal:
# The dispatch loop synthesizes a terminal on every path it knows
# about (timeout, stale, tool stall, cancel-unacked), so reaching
# here means one of its exits has none — and the consumer is left
# deciding what an unclosed turn means. Content-free by
# construction: this line carries no count, no text and no ids,
# because the only fact it has to report is that it happened.
logger.warning(
"prompt stream for this session ended without a terminal "
"completion event; the caller will see the turn as producing "
"nothing"
)

# ── Turn park state (readable from OUTSIDE the turn) ──

Expand Down
Loading
Loading