diff --git a/README.md b/README.md index 67f370fe..b5512e04 100644 --- a/README.md +++ b/README.md @@ -1527,8 +1527,8 @@ Gemini exhaustion goes straight to Opus; see `--antigravity-models`. Antigravity turns are single-shot and its usage totals are estimated because `agy` does not expose token counts. A model that repeatedly answers in chat prose with no recoverable structured payload also -falls back to the next chain model. When `agy` supports `--json-schema`, -structured turns use agy's schema-constrained output. Disable it with +falls back to the next chain model. When `agy` supports `--output-format`, +structured turns read the single JSON answer from agy's JSON envelope. Disable it with `--antigravity-structured-output off`; see [Structured output](docs/local_agent_loop.md#structured-output). diff --git a/docs/architecture/decisions/execution-and-concurrency.md b/docs/architecture/decisions/execution-and-concurrency.md index 11d49be0..b400b16c 100644 --- a/docs/architecture/decisions/execution-and-concurrency.md +++ b/docs/architecture/decisions/execution-and-concurrency.md @@ -65,3 +65,4 @@ never rewritten or deleted. The detail doc describes the current design. - 2026-11-20: A non-`SUCCESS` agy envelope's decoded error is a typed provider diagnostic on `AgentResult` and is classified as the terminal frame in every failure phase. A fallback response that fails validation after such an envelope is classified as a provider failure, with no repair and no format-failure fallback (#2029). - 2026-11-20: The agy structured-output schema avoids non-string enums, which Gemini rejects with INVALID_ARGUMENT; `schema_version` is a plain integer validated by the protocol parsers. A structured attempt whose provider diagnostics (non-`SUCCESS` envelope, stdout outside the envelope, or the silent attempt's own agy log) show a schema-naming INVALID_ARGUMENT is replayed once per turn without schema flags, with a warning, a dedicated attempt slot, no retry, no format-failure count and no repair. Model prose is never evidence (#2039). - 2026-11-20: `run-tests` refuses a request before reservation or launch when a registered worktree link or private environment of the broker checkout is no longer intact, naming it in a no-test `cancelled` frame; the pre-check is fail-open and never poisons the checkout. Coder prompts name the coder checkout's registered links as shared operator infrastructure. An uncitable receipt for a citable launcher family names the rejected argument through a static explainer that shares the recognizer's argument walk; the allow-list and persisted reasons are unchanged (#2036). +- 2026-11-20: agy's `structured_output` keeps only the properties the request schema declares, so the loose schema stripped every review field. Structured Antigravity turns now pass `--output-format json` with no `--json-schema` (the probe needs only `--output-format`), and `structured_output` is never the answer source. The answer is the response file, else the single expected-kind (or `agent_unavailable`) JSON object recovered from the envelope `response`, counted as top-level objects over the whole response before any marker handling, rendered with the payload's own footer state and signature in code, else the marker text. Several candidate objects, a wrong kind, or an out-of-span footer whose state differs is a typed tool-authored ambiguity on `AgentResult`; the validated turn records it as an `agent-unavailable` format failure that feeds #2029's format accounting, with no repair and no classification from model text. Reviewer repair refuses envelope-only `{schema_version, kind, state}` payloads (#2049). diff --git a/docs/architecture/execution-and-concurrency.md b/docs/architecture/execution-and-concurrency.md index 8a276143..d33857e9 100644 --- a/docs/architecture/execution-and-concurrency.md +++ b/docs/architecture/execution-and-concurrency.md @@ -530,13 +530,13 @@ and an all-groups-cooling state stops before invoking. Only when every group carries a parsed long reset is `QuotaResetExceededError` raised. Memory is not persisted across processes. -**Antigravity structured output (#2029, #2039).** -`agents/antigravity_structured.py` adds agy's `--json-schema` channel (string -enums only) for eligible kinds after a cached probe. A renderable -`structured_output` wins; validators stay authoritative. No-payload turns get -a retry-free `format-fallback`. A schema-naming INVALID_ARGUMENT in provider -diagnostics (not model prose) replays the dispatch once per turn without schema -flags: no retry, format failure or repair. See the +**Antigravity structured output (#2029, #2039, #2049).** +`agents/antigravity_structured.py` requests agy's schema-free +`--output-format json` envelope for eligible kinds. Answer: response file, else +the one expected-kind object in `response` (footer/signature added in code), +else the marker; `structured_output` is ignored. Ambiguity (several objects, +wrong kind, contradicting footer) is a trusted, repair-free format failure +feeding `format-fallback`. Envelope-only reviews are refused repair. See the [CLI guide](../local_agent_loop.md#structured-output). **Quota reset wait.** `--wait-for-quota-reset DURATION` (off by default) turns diff --git a/docs/local_agent_loop.md b/docs/local_agent_loop.md index e3afa04e..4c187b1a 100644 --- a/docs/local_agent_loop.md +++ b/docs/local_agent_loop.md @@ -1145,11 +1145,11 @@ The default Antigravity chain is `Gemini 3.8 Flash (High)` → `Claude Opus 5.5 ### Structured output -When the installed `agy` advertises `--output-format` and `--json-schema`, structured turns ask agy for schema-constrained output. These are PR and plan reviews, plan state and revisions, coder follow-ups, issue implementations and task results. agent-loop writes a loose per-kind JSON Schema beside the private response file and passes `--output-format json --json-schema `. It reads the payload from the envelope's `structured_output` and appends the footer marker and signature itself. A narrating model therefore still yields a usable response. A usable `structured_output` takes precedence over the response file. The response file, and then the stdout marker, remain the fallback when `structured_output` is missing or the status is not `SUCCESS`. agent-loop's own validators stay authoritative. A non-`SUCCESS` envelope is treated as an operational failure reported by agy. Its decoded error is classified as the final provider line, so model narration cannot hide it. Auth and billing errors stop without retry or fallback, and quota or capacity errors take the quota path. A fallback response that then fails validation is never counted as a format failure. A turn that must carry content outside the JSON keeps the response-file channel. Such turns include those with signed human requirements to acknowledge, a required direct-discussion acknowledgement, or a reviewer's resolved-requirements marker. Repair, classification and discuss turns also keep it. +When the installed `agy` advertises `--output-format`, structured turns ask agy for its JSON envelope with `--output-format json`. These are PR and plan reviews, plan state and revisions, coder follow-ups, issue implementations and task results. No request schema is sent: agy's `--json-schema` channel keeps only the properties the schema declares in `structured_output`, which stripped every review field (#2049), so `structured_output` is never used as the answer. agent-loop takes the answer in this order: a non-empty response file; otherwise exactly one JSON object of the expected kind (or `agent_unavailable`), bare or in a single ```json fence, in the envelope's `response` text; otherwise the stdout marker in that text. A recovered object gets its footer marker (carrying the object's own state) and signature appended in code, so a model that writes its payload only in chat still yields a usable response; an object without a state is never given one and falls back to the marker text. Candidates are counted over the whole `response`, before the stdout marker is considered. Two or more top-level objects (including one before and one after the marker), a single object of another kind or without a kind, or a state footer outside the object that contradicts it is a format failure: no object is picked, no repair runs, and the failure is classified only from a tool-authored diagnostic, so it counts toward the format fallback to the next chain model. agent-loop's own validators stay authoritative. A reviewer payload carrying only `schema_version`, `kind` and `state` is never accepted and is refused repair. A non-`SUCCESS` envelope is treated as an operational failure reported by agy, and no payload is recovered from its prose. Its decoded error is classified as the final provider line, so model narration cannot hide it. Auth and billing errors stop without retry or fallback, and quota or capacity errors take the quota path. A fallback response that then fails validation is never counted as a format failure. A turn that must carry content outside the JSON keeps the response-file channel. Such turns include those with signed human requirements to acknowledge, a required direct-discussion acknowledgement, or a reviewer's resolved-requirements marker. Repair, classification and discuss turns also keep it. -The schema uses only constructs Gemini function declarations accept: `enum` appears only on string values, so `schema_version` is a plain integer and the validators check its value. If agy or the provider still rejects agent-loop's request schema (an `INVALID_ARGUMENT` that names the schema, reported in a non-`SUCCESS` envelope, in stdout outside the envelope, or in the silent attempt's own agy CLI log), agent-loop logs a warning and replays that turn once without the schema flags. The replay uses the same prompt and model, consumes no retry, is not a format failure and gets no repair; a failed replay is classified normally. The next turn tries structured output again. Model prose that quotes such an error never triggers the replay. Use `--antigravity-structured-output off` to disable the channel persistently. +If agy or the provider still rejects agent-loop's structured request (an `INVALID_ARGUMENT` that names a request schema, reported in a non-`SUCCESS` envelope, in stdout outside the envelope, or in the silent attempt's own agy CLI log), agent-loop logs a warning and replays that turn once without the JSON flags. The replay uses the same prompt and model, consumes no retry, is not a format failure and gets no repair; a failed replay is classified normally. The next turn tries structured output again. Model prose that quotes such an error never triggers the replay. Use `--antigravity-structured-output off` to disable the channel persistently. -The capability is probed once per agy command with `agy --help`. A probe that fails, an older agy, or `--antigravity-arg` values that already set `--output-format` or `--json-schema` keep the legacy channel. `--antigravity-structured-output off` (default `auto`) disables the schema flags explicitly. When a prompt is too large for argv, the short `--print` directive names the response file (or the structured result) and tells the model not to answer in chat. +The capability is probed once per agy command with `agy --help`. A probe that fails, an older agy, or `--antigravity-arg` values that already set `--output-format` or `--json-schema` keep the legacy channel. `--antigravity-structured-output off` (default `auto`) disables the JSON envelope explicitly. When a prompt is too large for argv, the short `--print` directive names the response file (or asks for exactly one JSON object as the final answer) and tells the model not to answer in chat. Every `agy --print` call passes `--print-timeout` from `--antigravity-print-timeout-seconds` (default `600`, i.e. ten minutes), which diff --git a/src/coding_review_agent_loop/agents/antigravity.py b/src/coding_review_agent_loop/agents/antigravity.py index 227e1686..579a46fe 100644 --- a/src/coding_review_agent_loop/agents/antigravity.py +++ b/src/coding_review_agent_loop/agents/antigravity.py @@ -50,7 +50,8 @@ ) from .antigravity_structured import ( AgyEnvelope, - SCHEMA_FILE_NAME, + AmbiguousPayload, + RecoveredPayload, envelope_provider_error, envelope_request_rejection, STRUCTURED_KIND_FOOTERS, @@ -59,8 +60,8 @@ parse_agy_json_envelope, render_structured_output_text, structured_output_signature, + recover_response_payload, structured_request_rejection_line, - write_structured_output_schema, ) from .replacement import classify_provider_executable_replacement_interruption from ..worktree_hints import dependency_hint_environment @@ -1426,7 +1427,7 @@ def structured_request_rejection_evidence( """Provider-owned evidence that agy rejected the request schema (#2039). Call only for a structured attempt. Nothing counts when the attempt - produced a renderable ``structured_output`` or a response file, or timed + produced a payload rendered from the envelope (#2049) or a response file, or timed out. Sources, in order: a non-SUCCESS envelope's error/status; stdout outside any parsed envelope; raw stdout of a nonzero exit with no envelope; this silent attempt's own agy log. A SUCCESS envelope's @@ -1468,44 +1469,63 @@ def _select_structured_turn_text( *, kind: str, signature: str, -) -> tuple[str, AgentTextSource, str | None]: - """(text, text_source, provider error) for a JSON-mode turn (#2029). - - Precedence: a SUCCESS envelope whose ``structured_output`` renders, then a - non-empty response file, then the public-response marker in the - envelope's prose (or raw stdout when no envelope parsed). A non-empty - response file does not prove a usable contract, so a renderable payload - wins over it; the file stays on the result for audit only. - - A non-SUCCESS envelope is an operational failure reported by agy, not - model prose: its decoded error comes back separately as the provider - error, so classification can read it as the terminal provider frame in - every failure phase (narration in ``response`` can never displace it). +) -> tuple[str, AgentTextSource, str | None, str | None]: + """(text, text_source, provider error, ambiguity) for a JSON-mode turn. + + Precedence (#2029, #2049): + + 1. A non-SUCCESS envelope is an operational failure reported by agy, not + model prose: its decoded error comes back separately as the provider + error, so classification can read it as the terminal provider frame in + every failure phase. The response file, else the marker text of the + envelope's prose, is the fallback content; no payload is recovered + from a failed envelope's prose. + 2. A non-empty response file. + 3. A SUCCESS envelope whose ``response`` holds exactly one expected-kind + (or agent_unavailable) JSON object: rendered in code with the footer + carrying the payload's own state and the signature. + 4. An ambiguous ``response`` (several candidate objects, a wrong kind, + or a contradicting footer): empty text plus a tool-authored diagnostic + the validated turn records as a format failure. + 5. Otherwise the public-response marker in the envelope's prose (or raw + stdout when no envelope parsed). + + agy's ``structured_output`` field is never the answer source: it holds + only declared schema keys, so it could displace the model's full payload. """ envelope: AgyEnvelope | None = parse_agy_json_envelope(stdout) - if envelope is not None and envelope.succeeded and envelope.structured_output is not None: - rendered = render_structured_output_text(envelope.structured_output, kind, signature) - if rendered is not None: - if response_file_text is not None: - log( - config, - "Antigravity returned both structured_output and a response file; " - "using structured_output (the file is kept for audit only)", - ) - return rendered, "structured_output", None - log(config, "Antigravity structured_output was not renderable; falling back") - provider_error = ( - envelope_provider_error(envelope) - if envelope is not None and not envelope.succeeded - else None - ) - if response_file_text is not None: - return response_file_text, "response_file", provider_error - if envelope is not None: + if envelope is not None and not envelope.succeeded: + provider_error = envelope_provider_error(envelope) + if response_file_text is not None: + return response_file_text, "response_file", provider_error, None text, source = _strip_public_response_marker(envelope.response) - return text, source, provider_error - text, source = _strip_public_response_marker(stdout) - return text, source, provider_error + return text, source, provider_error, None + if envelope is not None and envelope.structured_output is not None: + log( + config, + "Antigravity envelope carried structured_output; ignored (never the answer source)", + ) + if response_file_text is not None: + return response_file_text, "response_file", None, None + if envelope is None: + text, source = _strip_public_response_marker(stdout) + return text, source, None, None + recovery = recover_response_payload(envelope.response, kind) + if isinstance(recovery, AmbiguousPayload): + diagnostic = recovery.diagnostic + log(config, f"{diagnostic}; recording a format failure") + return "", "stdout", None, diagnostic + if isinstance(recovery, RecoveredPayload): + rendered = render_structured_output_text(recovery.payload, kind, signature) + if rendered is not None: + return rendered, "structured_output", None, None + log( + config, + "Antigravity response object could not be rendered (missing state); " + "falling back to the marker text", + ) + text, source = _strip_public_response_marker(envelope.response) + return text, source, None, None def _parse_model_catalog(raw: str) -> set[str]: @@ -1628,7 +1648,6 @@ def run( or not antigravity_structured_output_supported(runner, config) ): structured_kind = None - schema_path: Path | None = None args = [ config.antigravity_cmd, "--model", @@ -1645,10 +1664,9 @@ def run( if session_id: args += ["--conversation", session_id] if structured_kind is not None: - # The schema sits beside the private response file, outside the - # checkout, and is removed after the turn. - schema_path = write_structured_output_schema(response_path.parent, structured_kind) - args += ["--output-format", "json", "--json-schema", str(schema_path)] + # JSON envelope only: no request schema, because agy's + # structured_output keeps only declared schema keys (#2049). + args += ["--output-format", "json"] # The prompt is the value of --print (must be last), not a positional. args += [ "--print", @@ -1925,8 +1943,6 @@ def strip_injected_prefix() -> None: log(config, f"{seat_label} could not remove its shared agy settings lock holder record: {exc}") fcntl.flock(settings_lock, fcntl.LOCK_UN) settings_lock.close() - if schema_path is not None: - schema_path.unlink(missing_ok=True) log(config, f"Antigravity ({model}) finished; log: {log_path}") # Prefer the public response file the prompt asks the agent to write; else @@ -1940,8 +1956,14 @@ def strip_injected_prefix() -> None: response_path.unlink(missing_ok=True) response_file_text = None provider_error_text: str | None = None + structured_ambiguity: str | None = None if structured_kind is not None: - message_text, text_source, provider_error_text = _select_structured_turn_text( + ( + message_text, + text_source, + provider_error_text, + structured_ambiguity, + ) = _select_structured_turn_text( config, result.stdout, response_file_text, @@ -1997,6 +2019,7 @@ def strip_injected_prefix() -> None: antigravity_log_path=agy_log_path, antigravity_log_quota_evidence=agy_log_evidence, antigravity_structured_request_rejection=request_rejection, + antigravity_structured_ambiguity=structured_ambiguity, ) @staticmethod diff --git a/src/coding_review_agent_loop/agents/antigravity_structured.py b/src/coding_review_agent_loop/agents/antigravity_structured.py index 2dc87e15..31adfe22 100644 --- a/src/coding_review_agent_loop/agents/antigravity_structured.py +++ b/src/coding_review_agent_loop/agents/antigravity_structured.py @@ -1,23 +1,27 @@ -"""agy schema-constrained output for structured Antigravity turns (#2029). +"""agy JSON-envelope output for structured Antigravity turns (#2029, #2049). Some Gemini models answer in chat markdown even when a turn asks for a JSON -response file. agy can constrain the final answer with ``--output-format json ---json-schema ``: the CLI then prints a JSON envelope whose -``structured_output`` field holds a schema-valid object even when the model's -``response`` prose is narration. +response file. agy's ``--output-format json`` wraps the turn in a JSON +envelope whose ``response`` field holds the model's final answer text. No +request schema is sent: agy's ``--json-schema`` channel keeps only the +properties a schema declares in ``structured_output`` (#2049), so a loose +schema stripped every review field and a full nested schema is not accepted +by Gemini function declarations (#2039). This module holds the pure pieces of that channel: * a cached capability probe, so an older agy keeps the legacy channel; * the eligibility rule for a turn (allowlisted kind, and no content the contract requires outside the JSON object, footer and signature); -* a loose per-kind schema that also admits the shared ``agent_unavailable`` - alternative; -* a tolerant envelope parser and a renderer that rebuilds the public text - (JSON, footer marker, signature) in code. - -The schema only shapes the payload. The protocol validators stay -authoritative for every rendered response. +* a tolerant envelope parser; +* single-object recovery from the envelope's ``response`` text, which is a + format failure when the text holds more than one candidate object, an + object of another kind, or a footer contradicting the payload; +* a renderer that rebuilds the public text (JSON, footer marker, signature) + in code. + +``structured_output`` is never the answer source. The protocol validators +stay authoritative for every rendered response. """ from __future__ import annotations @@ -26,6 +30,7 @@ import json import os from pathlib import Path +import re import shutil import tempfile import threading @@ -52,8 +57,10 @@ } AGENT_UNAVAILABLE_KIND = "agent_unavailable" _AGENT_UNAVAILABLE_FOOTER = "" +# Operator pass-through of either flag means the operator owns the format. STRUCTURED_OUTPUT_FLAGS = ("--output-format", "--json-schema") -SCHEMA_FILE_NAME = "agy-structured-schema.json" +# The capability probe needs only the envelope flag: no schema is sent (#2049). +CAPABILITY_FLAGS = ("--output-format",) CAPABILITY_PROBE_TIMEOUT_SECONDS = 20.0 _capability_cache: dict[str, bool] = {} @@ -67,11 +74,12 @@ def antigravity_structured_kind_for_turn( reviewer_requirement_ids: Sequence[str] | None, requires_direct_discussion_ack: bool, ) -> str | None: - """The kind this turn may request through agy's schema flags, else None. + """The kind this turn may request through agy's JSON envelope, else None. A turn whose contract needs content beyond the JSON object, footer and signature keeps the response-file channel, because ``structured_output`` - cannot carry that content and the renderer must never invent it: + recovered from the envelope cannot carry that content and the renderer + must never invent it: * surfaced signed requirement IDs (the human-requirements acknowledgement marker and section between the JSON and the footer); @@ -96,7 +104,7 @@ def operator_args_set_output_format(args: Sequence[str]) -> bool: def help_text_supports_structured_output(help_text: str) -> bool: cleaned = strip_ansi(help_text or "") - return all(flag in cleaned for flag in STRUCTURED_OUTPUT_FLAGS) + return all(flag in cleaned for flag in CAPABILITY_FLAGS) def _probe_agy_help(runner: Runner, config: "AgentLoopConfig") -> str | None: @@ -134,7 +142,7 @@ def _capability_cache_key(config: "AgentLoopConfig") -> str: def antigravity_structured_output_supported( runner: Runner, config: "AgentLoopConfig" ) -> bool: - """Whether agy advertises both schema flags; probed once per command path. + """Whether agy advertises ``--output-format``; probed once per command path. ``off`` mode, operator pass-through format args, and a probe that errors, times out or exits nonzero all mean unsupported. A failed probe is cached @@ -169,35 +177,6 @@ def reset_structured_output_capability_cache() -> None: _capability_cache.clear() -def structured_output_schema(kind: str) -> dict[str, object]: - """A loose JSON Schema for ``kind`` that also admits ``agent_unavailable``. - - Only constructs Gemini function declarations accept are used (#2039): - ``enum`` appears only on string-typed values and every enum value is a - non-empty string. An integer enum is rejected by the provider with - INVALID_ARGUMENT, so ``schema_version`` is a plain integer here and the - protocol validators keep enforcing ``schema_version == 1``. ``state`` is - optional because an agent_unavailable payload has none; the role - validators still require it. - """ - return { - "type": "object", - "properties": { - "schema_version": {"type": "integer"}, - "kind": {"type": "string", "enum": [kind, AGENT_UNAVAILABLE_KIND]}, - "state": {"type": "string", "enum": ["approved", "blocking"]}, - }, - "required": ["schema_version", "kind"], - "additionalProperties": True, - } - - -def write_structured_output_schema(directory: Path, kind: str) -> Path: - path = directory / SCHEMA_FILE_NAME - path.write_text(json.dumps(structured_output_schema(kind), indent=2), encoding="utf-8") - return path - - @dataclass(frozen=True) class AgyEnvelope: status: str | None @@ -332,6 +311,105 @@ def envelope_request_rejection(envelope: AgyEnvelope | None) -> str | None: return None +# --- Single-object recovery from the envelope ``response`` (#2049) ----------- + +AMBIGUITY_MULTIPLE_CANDIDATES = "multiple-candidates" +AMBIGUITY_WRONG_KIND = "wrong-kind" +AMBIGUITY_FOOTER_STATE_CONFLICT = "footer-state-conflict" + +_STATE_FOOTER_RE = re.compile( + r"" +) +_AMBIGUITY_DESCRIPTIONS = { + AMBIGUITY_MULTIPLE_CANDIDATES: "candidate JSON objects", + AMBIGUITY_WRONG_KIND: "JSON object of an unexpected kind", + AMBIGUITY_FOOTER_STATE_CONFLICT: "JSON object whose state contradicts a footer outside it", +} + + +@dataclass(frozen=True) +class RecoveredPayload: + """Exactly one expected-kind (or agent_unavailable) object was found.""" + + payload: dict[str, object] + + +@dataclass(frozen=True) +class AmbiguousPayload: + """The response cannot yield one payload; carries no model text.""" + + reason: str + candidate_count: int + + @property + def diagnostic(self) -> str: + """A tool-authored diagnostic built only from the reason and count.""" + description = _AMBIGUITY_DESCRIPTIONS.get(self.reason, "unusable JSON objects") + return ( + f"Antigravity response carried {self.candidate_count} {description} " + f"({self.reason}); no payload selected" + ) + + +def _top_level_objects(text: str) -> list[tuple[int, int, dict[str, object]]]: + """Decoded top-level JSON objects as (start, end, value), in order. + + Only a ``{`` that decodes counts; an object nested inside an earlier + candidate is consumed by it. Scalars and arrays are never candidates. + """ + decoder = json.JSONDecoder() + found: list[tuple[int, int, dict[str, object]]] = [] + index = text.find("{") + while index != -1: + try: + value, end = decoder.raw_decode(text, index) + except ValueError: + index = text.find("{", index + 1) + continue + if isinstance(value, dict): + found.append((index, end, value)) + index = text.find("{", end) + else: # pragma: no cover - a value decoded at "{" is always a dict + index = text.find("{", index + 1) + return found + + +def recover_response_payload( + response: str, kind: str +) -> RecoveredPayload | AmbiguousPayload | None: + """Recover the single answer object from an envelope's ``response`` text. + + Candidates are counted over the complete normalized response, before and + independent of any public-response marker. ``None`` means no candidate. + Two or more candidates, one candidate of another kind (or without a + string kind), or a state footer outside the candidate whose state differs + from the payload's are ambiguous. Fields are never merged, picked or + edited. Never raises. + """ + try: + text = strip_ansi(response or "").replace("\r\n", "\n").replace("\r", "\n") + candidates = _top_level_objects(text) + except Exception: # noqa: BLE001 - recovery never fails a turn + return None + if not candidates: + return None + if len(candidates) > 1: + return AmbiguousPayload(AMBIGUITY_MULTIPLE_CANDIDATES, len(candidates)) + start, end, payload = candidates[0] + payload_kind = payload.get("kind") + if not isinstance(payload_kind, str) or payload_kind not in {kind, AGENT_UNAVAILABLE_KIND}: + return AmbiguousPayload(AMBIGUITY_WRONG_KIND, 1) + state = payload.get("state") + payload_state = state.strip() if isinstance(state, str) else "" + outside = text[:start] + "\n" + text[end:] + for match in _STATE_FOOTER_RE.finditer(outside): + if match.group(1).strip() != payload_state: + return AmbiguousPayload(AMBIGUITY_FOOTER_STATE_CONFLICT, 1) + return RecoveredPayload(payload) + + def structured_output_signature(config: "AgentLoopConfig", model: str) -> str: """The Antigravity signature label, seat-labelled when a seat is active.""" base = f"Google Antigravity: {model}" @@ -342,7 +420,7 @@ def structured_output_signature(config: "AgentLoopConfig", model: str) -> str: def render_structured_output_text( payload: object, kind: str, signature: str ) -> str | None: - """Rebuild the public text from a ``structured_output`` payload, else None. + """Rebuild the public text from a recovered payload, else None. A role payload gets its kind's footer carrying the payload's own state; an agent_unavailable payload gets the dedicated footer. The renderer @@ -367,8 +445,8 @@ def render_structured_output_text( def oversized_prompt_directive(response_path: Path, *, structured: bool) -> str: """The short ``--print`` value used when the task moves into GEMINI.md.""" channel = ( - "Return your answer as the requested structured result (the JSON object the " - f"schema describes); if you also write a file, write it to {response_path}." + "Return exactly one JSON object as your final answer, as the task describes; " + f"if you also write a file, write it to {response_path}." if structured else f"Write your complete answer to the public response file {response_path}." ) diff --git a/src/coding_review_agent_loop/agents/base.py b/src/coding_review_agent_loop/agents/base.py index 15cca990..2dae02a0 100644 --- a/src/coding_review_agent_loop/agents/base.py +++ b/src/coding_review_agent_loop/agents/base.py @@ -21,8 +21,10 @@ from ..review_images_transport import ImageInputs, ImageDeliveryProof AgentName = Literal["claude", "codex", "gemini", "antigravity"] -# ``structured_output`` is an agent-authored payload from agy's schema channel -# (#2029); like ``response_file`` it was not scraped from stdout. +# ``structured_output`` is an agent-authored payload rendered in code from the +# agy JSON envelope (#2029, #2049): the single JSON object recovered from the +# envelope's ``response`` text, never agy's own ``structured_output`` field. +# Like ``response_file`` it was not scraped from stdout. AgentTextSource = Literal["response_file", "structured_output", "stdout_marker", "stdout"] AGENT_AUTHORED_TEXT_SOURCES = frozenset({"response_file", "structured_output"}) @@ -101,6 +103,12 @@ class AgentResult: # showing agy rejected agent-loop's request schema (INVALID_ARGUMENT # naming the schema). Never derived from model prose. antigravity_structured_request_rejection: str | None = None + # Antigravity JSON mode only (#2049): a tool-authored diagnostic set when + # the envelope's ``response`` held more than one candidate JSON object, an + # object of another kind, or a footer contradicting the payload. Built + # only from a fixed reason code and a count, never from model text; the + # validated turn records it as a format failure with no repair. + antigravity_structured_ambiguity: str | None = None image_delivery_proof: ImageDeliveryProof | None = None diff --git a/src/coding_review_agent_loop/cli.py b/src/coding_review_agent_loop/cli.py index 7a143ac5..14209d5e 100644 --- a/src/coding_review_agent_loop/cli.py +++ b/src/coding_review_agent_loop/cli.py @@ -520,7 +520,7 @@ def add_common(subparser: argparse.ArgumentParser) -> None: choices=("auto", "off"), default="auto", help=( - "Use agy --output-format json --json-schema for eligible structured turns " + "Use agy --output-format json (no request schema) for eligible structured turns " "when the installed agy supports it (default: auto). 'off' keeps the " "legacy response-file channel." ), diff --git a/src/coding_review_agent_loop/repair.py b/src/coding_review_agent_loop/repair.py index 07b84eaf..33d1fd77 100644 --- a/src/coding_review_agent_loop/repair.py +++ b/src/coding_review_agent_loop/repair.py @@ -1717,6 +1717,9 @@ def _issue_implementation_instruction( ) +_REVIEW_ENVELOPE_ONLY_KEYS = frozenset({"schema_version", "kind", "state"}) + + def require_recoverable_review_substance(raw: str, *, expected_kind: str) -> None: """Refuse reviewer repair when the source carries no review substance. @@ -1746,6 +1749,14 @@ def require_recoverable_review_substance(raw: str, *, expected_kind: str) -> Non f"Reviewer repair refused: the source declares kind {kind!r}, not the " f"expected `{expected_kind}`." ) + if set(payload) <= _REVIEW_ENVELOPE_ONLY_KEYS: + # An envelope-only payload (for example agy's key-stripped + # structured_output, #2049) carries no summary, finding or + # disposition; repair could only invent the review around it. + raise ReviewSubstanceIntegrityError( + f"Reviewer repair refused: the `{expected_kind}` source carries only " + "envelope keys (schema_version, kind, state) and no review substance." + ) return if not (REVIEW_KIND_UNIQUE_FIELDS[expected_kind] & set(payload)): raise ReviewSubstanceIntegrityError( diff --git a/src/coding_review_agent_loop/validated_agent.py b/src/coding_review_agent_loop/validated_agent.py index 8a9f8033..3f459452 100644 --- a/src/coding_review_agent_loop/validated_agent.py +++ b/src/coding_review_agent_loop/validated_agent.py @@ -1800,7 +1800,8 @@ def _antigravity_stop_error( validation_acquisition = result audit_result = result if result.text_source == "structured_output": - # agy's structured_output won (#2029): the rendered payload is this + # The payload recovered from the agy envelope won (#2029, #2049): + # the rendered payload is this # invocation's authoritative artifact. A response file the model # also wrote is audit-only, so artifact salvage, response-file # status and structured-candidate recovery see the rendered @@ -2248,6 +2249,27 @@ def _antigravity_stop_error( # but yields to typed runner retry decisions (#1226, #1598). should_retry = False last_failure_category = "non-retryable" + elif ( + backend_agent == "antigravity" + and antigravity_structured_kind is not None + and result.antigravity_structured_ambiguity is not None + ): + # The agy envelope's response held several candidate JSON objects, + # an object of another kind, or a contradicting footer (#2049). + # That is a pure format failure: classify from the tool-authored + # diagnostic only (never raw output or rejected payload prose, so + # no quota/auth/capacity path), skip repair, and let the #2029 + # per-model format accounting decide retry or format-fallback. + diagnostic = result.antigravity_structured_ambiguity + last_error = diagnostic + classification_text = diagnostic + last_classification_text = diagnostic + last_failure_category = "agent-unavailable" + should_retry = True + format_failure = True + if usage_record is not None: + usage_record.validation_status = "invalid" + log(config, f"{agent_name}: {diagnostic}; no repair (format failure)") elif not text.strip(): last_error = "agent response was empty" classification_text, provider_verdict = _agent_failure_classification( diff --git a/tests/test_antigravity.py b/tests/test_antigravity.py index 90768a1d..6c1d0153 100644 --- a/tests/test_antigravity.py +++ b/tests/test_antigravity.py @@ -2422,27 +2422,43 @@ def _payload(text): return json.JSONDecoder().raw_decode(text.lstrip())[0] -def _agy_envelope(structured=None, *, status="SUCCESS", response=_NARRATED_PROSE, **extra): +_DECLARED_KEYS = ("schema_version", "kind", "state") + + +def _agy_envelope(payload=None, *, status="SUCCESS", response=_NARRATED_PROSE, **extra): + """An agy envelope; ``payload`` is the model's full answer object. + + Like agy 1.5.12 (#2049), the payload is carried in ``response`` as one + fenced JSON block after the narration, while ``structured_output`` holds + only the keys the old request schema declared. + """ envelope = {"status": status, "response": response, **extra} - if structured is not None: - envelope["structured_output"] = structured + if payload is not None: + envelope["response"] = f"{response}\n\n```json\n{json.dumps(payload, indent=2)}\n```\n" + envelope.setdefault( + "structured_output", {key: payload[key] for key in _DECLARED_KEYS if key in payload} + ) # PTY start-up chrome and CRLF line endings around the envelope. return "\x1b[2mStarting agy 1.5.12\x1b[0m\r\n" + json.dumps(envelope) + "\r\n" -class _SchemaCapturingRunner(FakeRunner): - """Captures each agy argv and the schema file content while agy runs.""" +class _ResponseDirCapturingRunner(FakeRunner): + """Records the private response directory's contents while agy runs.""" def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) - self.schemas: list[dict | None] = [] + self.response_dir_listings: list[list[str] | None] = [] def run_with_log(self, args, *, cwd, **kwargs): - if list(args[:1]) == ["agy"] and "--json-schema" in args: - path = Path(args[args.index("--json-schema") + 1]) - self.schemas.append(json.loads(path.read_text(encoding="utf-8"))) - elif list(args[:1]) == ["agy"]: - self.schemas.append(None) + if list(args[:1]) == ["agy"]: + prompt = args[-1] + match = re.search(r"(/\S*?/response\.md)", prompt) + listing: list[str] | None = None + if match is not None: + directory = Path(match.group(1)).parent + if directory.exists(): + listing = sorted(path.name for path in directory.iterdir()) + self.response_dir_listings.append(listing) return super().run_with_log(args, cwd=cwd, **kwargs) @@ -2474,29 +2490,40 @@ def _structured_config(tmp_path, kind="pr_review", **overrides): ], ids=["pr_review", "plan_review"], ) -def test_structured_output_is_used_when_the_model_narrates(tmp_path, monkeypatch, kind, payload_text, parse): +def test_stripped_structured_output_never_displaces_the_response_payload( + tmp_path, monkeypatch, kind, payload_text, parse +): + """#2049: structured_output holds only declared keys; response holds the payload.""" from coding_review_agent_loop.agents.antigravity import AntigravityBackend _enable_agy_structured_output(monkeypatch) structured = _payload(payload_text) - runner = _SchemaCapturingRunner(antigravity_outputs=[(_agy_envelope(structured), 0)]) + stdout = _agy_envelope(structured) + stripped = json.loads(stdout.split("\r\n")[1])["structured_output"] + assert set(stripped) == set(_DECLARED_KEYS) # the live agy 1.5.12 shape + runner = _ResponseDirCapturingRunner(antigravity_outputs=[(stdout, 0)]) config = _structured_config(tmp_path, kind) result = AntigravityBackend().run(runner, config, "Review this PR.", run_id="r", role="reviewer") cmd = runner.commands[-1][0] assert cmd[cmd.index("--output-format") + 1] == "json" - assert cmd.index("--json-schema") < cmd.index("--print") - schema = runner.schemas[-1] - assert schema["properties"]["kind"]["enum"] == [kind, "agent_unavailable"] - assert set(schema["required"]) == {"schema_version", "kind"} - assert not Path(cmd[cmd.index("--json-schema") + 1]).exists() + assert cmd.index("--output-format") < cmd.index("--print") + assert "--json-schema" not in cmd + assert runner.response_dir_listings[-1] is not None + assert not any(name.endswith(".json") for name in runner.response_dir_listings[-1]) + assert not any(name.endswith(".json") for name in _os.listdir(result.response_file_path.parent)) assert result.text_source == "structured_output" assert result.response_file_text is None assert _NARRATED_PROSE not in result.text assert result.text.rstrip().endswith("-- Google Antigravity: Gemini 4 Hydrogen (Medium)") + assert _payload(result.text) == structured # the full payload, not the stripped one + assert structured["summary"] in result.text + for item in structured.get("blocking_items", []): + assert item in result.text parsed = parse(result.text) assert parsed.state == structured["state"] + assert parsed.summary == structured["summary"] @pytest.mark.parametrize( @@ -2550,11 +2577,12 @@ def test_missing_structured_output_falls_back_to_file_then_marker( ["# Review\n\nLooks fine, approving.", structured_pr_review(state="blocking", blocking_items=["Other."])], ids=["invalid-markdown-file", "valid-but-different-file"], ) -def test_structured_output_wins_over_a_non_empty_response_file(tmp_path, monkeypatch, file_text): +def test_non_empty_response_file_wins_over_the_envelope_payload(tmp_path, monkeypatch, file_text): + """#2049: a response file outranks the payload recovered from response prose.""" from coding_review_agent_loop.agents.antigravity import AntigravityBackend _enable_agy_structured_output(monkeypatch) - structured = _payload(structured_pr_review(summary="From structured_output.")) + structured = _payload(structured_pr_review(summary="From the envelope prose.")) runner = FakeRunner( antigravity_outputs=[(_agy_envelope(structured), 0)], public_response_outputs=[file_text], @@ -2563,24 +2591,20 @@ def test_structured_output_wins_over_a_non_empty_response_file(tmp_path, monkeyp result = AntigravityBackend().run(runner, config, "Review this PR.", run_id="r", role="reviewer") - assert result.text_source == "structured_output" - assert result.response_file_text == file_text.strip() # retained for audit - assert "From structured_output." in result.text - assert file_text.strip() not in result.text # never merged - assert parse_pr_review(result.text, reviewer="Antigravity").state == "approved" + assert result.text_source == "response_file" + assert result.text == file_text.strip() + assert "From the envelope prose." not in result.text # never merged + assert result.antigravity_structured_ambiguity is None @pytest.mark.parametrize("returncode", [0, 1], ids=["zero-exit", "nonzero-exit"]) -def test_validated_turn_accepts_structured_output_over_invalid_file_without_repair( +def test_validated_turn_accepts_the_recovered_envelope_payload_without_repair( tmp_path, monkeypatch, returncode ): _enable_agy_structured_output(monkeypatch) structured = _payload(structured_pr_review(summary="From structured_output.")) runner = FakeRunner( antigravity_outputs=[(_agy_envelope(structured), returncode)], - # A valid but different review in the file: neither salvage nor - # structured-candidate recovery may prefer it. - public_response_outputs=[structured_pr_review(state="blocking", blocking_items=["From file."])], ) config = make_config(tmp_path, antigravity_model="Gemini 4 Hydrogen (Medium)", agent_max_retries=0) repairs: list[object] = [] @@ -2605,7 +2629,6 @@ def record_repair(*_a, **_k): assert repairs == [] assert response.marker_value.state == "approved" assert "From structured_output." in response.text - assert "From file." not in response.text assert len([c for c, _ in runner.commands if c[:1] == ["agy"]]) == 1 @@ -2696,7 +2719,7 @@ def record_repair(*_a, **_k): @pytest.mark.parametrize( "case", - ["old-help", "probe-error", "probe-nonzero", "off", "operator-output-format"], + ["old-help", "probe-error", "probe-nonzero", "off", "operator-output-format", "operator-json-schema"], ) def test_capability_gate_keeps_legacy_args(tmp_path, monkeypatch, case): from coding_review_agent_loop.agents import antigravity_structured @@ -2721,8 +2744,12 @@ def probe(runner, config): overrides["antigravity_structured_output"] = "off" if case == "operator-output-format": overrides["antigravity_args"] = ("--output-format=text",) + if case == "operator-json-schema": + overrides["antigravity_args"] = ("--json-schema=/tmp/operator-schema.json",) config = _structured_config(tmp_path, **overrides) - runner = FakeRunner(antigravity_outputs=[("first review", 0), ("second review", 0)]) + # A recoverable payload in an envelope must not be used on the legacy channel. + envelope = _agy_envelope(_payload(structured_pr_review())) + runner = FakeRunner(antigravity_outputs=[("first review", 0), (envelope, 0)]) first = AntigravityBackend().run(runner, config, "Review this PR.", run_id="r", role="reviewer") second = AntigravityBackend().run(runner, config, "Review this PR.", run_id="r", role="reviewer") @@ -2732,9 +2759,10 @@ def probe(runner, config): assert "--json-schema" not in cmd assert "--output-format" not in cmd assert first.text == "first review" and first.text_source == "stdout" - assert second.text == "second review" + assert second.text == envelope and second.text_source == "stdout" # no envelope recovery + assert second.antigravity_structured_ambiguity is None # Probed at most once per command path; never for off/operator overrides. - assert len(probes) == (0 if case in {"off", "operator-output-format"} else 1) + assert len(probes) == (0 if case in {"off", "operator-output-format", "operator-json-schema"} else 1) def test_capability_probe_runs_once_when_supported(tmp_path, monkeypatch): @@ -2746,7 +2774,21 @@ def test_capability_probe_runs_once_when_supported(tmp_path, monkeypatch): AntigravityBackend().run(runner, config, "Review.", run_id="r", role="reviewer") AntigravityBackend().run(runner, config, "Review.", run_id="r", role="reviewer") assert probes == ["agy"] - assert all("--json-schema" in cmd for cmd, _ in runner.commands if cmd[:1] == ["agy"]) + agy = [cmd for cmd, _ in runner.commands if cmd[:1] == ["agy"]] + assert agy and all( + cmd[cmd.index("--output-format") + 1] == "json" and "--json-schema" not in cmd for cmd in agy + ) + + +def test_capability_probe_needs_only_output_format(): + from coding_review_agent_loop.agents.antigravity_structured import ( + help_text_supports_structured_output, + ) + + assert help_text_supports_structured_output("Usage: agy\n --output-format string text|json\n") + assert help_text_supports_structured_output(_AGY_HELP_WITH_SCHEMA) + assert not help_text_supports_structured_output("Usage: agy\n --json-schema string\n") + assert not help_text_supports_structured_output("Usage: agy\n --print string\n") @pytest.mark.parametrize( @@ -2814,7 +2856,7 @@ def test_validated_turn_threads_eligibility_into_the_agy_argv( tmp_path, monkeypatch, expected_kind, turn_kwargs, flags_expected ): _enable_agy_structured_output(monkeypatch) - runner = _SchemaCapturingRunner(antigravity_outputs=[("ok", 0)]) + runner = FakeRunner(antigravity_outputs=[("ok", 0)]) config = make_config(tmp_path, agent_max_retries=0) _run_validated_agent( @@ -2829,10 +2871,8 @@ def test_validated_turn_threads_eligibility_into_the_agy_argv( ) cmd = [c for c, _ in runner.commands if c[:1] == ["agy"]][-1] - assert ("--json-schema" in cmd) is flags_expected + assert "--json-schema" not in cmd assert ("--output-format" in cmd) is flags_expected - if flags_expected: - assert runner.schemas[-1]["properties"]["kind"]["enum"] == [expected_kind, "agent_unavailable"] @pytest.mark.parametrize("role", ["repair", "semantic-dedupe"]) @@ -2878,7 +2918,8 @@ def test_oversized_directive_names_the_output_channel(tmp_path, monkeypatch, str assert all(prompt not in arg for arg in cmd) assert str(result.response_file_path) in directive assert "Do not answer in chat" in directive - assert ("structured result" in directive) is structured + assert ("exactly one JSON object" in directive) is structured + assert "schema" not in directive assert len(directive.encode("utf-8")) < 2_000 @@ -2917,46 +2958,6 @@ def test_non_success_envelope_error_is_a_typed_provider_diagnostic(tmp_path, mon ) -def _walk_schema_nodes(node, path="$"): - if isinstance(node, dict): - yield path, node - for key, value in node.items(): - yield from _walk_schema_nodes(value, f"{path}.{key}") - elif isinstance(node, list): - for index, value in enumerate(node): - yield from _walk_schema_nodes(value, f"{path}[{index}]") - - -from coding_review_agent_loop.agents.antigravity_structured import ( # noqa: E402 - STRUCTURED_KIND_FOOTERS as _STRUCTURED_KIND_FOOTERS, -) - - -@pytest.mark.parametrize("kind", sorted(_STRUCTURED_KIND_FOOTERS)) -def test_structured_schema_uses_only_gemini_compatible_enums(tmp_path, kind): - from coding_review_agent_loop.agents.antigravity_structured import ( - structured_output_schema, - write_structured_output_schema, - ) - - written = json.loads(write_structured_output_schema(tmp_path, kind).read_text(encoding="utf-8")) - for schema in (structured_output_schema(kind), written): - enum_nodes = 0 - for path, node in _walk_schema_nodes(schema): - if "enum" not in node: - continue - enum_nodes += 1 - assert node.get("type") == "string", f"{path}: enum on a non-string type" - assert node["enum"], f"{path}: empty enum" - for value in node["enum"]: - assert isinstance(value, str) and value.strip(), f"{path}: bad enum value {value!r}" - assert schema["properties"]["schema_version"] == {"type": "integer"} - assert schema["properties"]["kind"]["enum"] == [kind, "agent_unavailable"] - assert schema["properties"]["state"]["enum"] == ["approved", "blocking"] - assert schema["required"] == ["schema_version", "kind"] - assert enum_nodes == 2 - - def test_validated_turn_rejects_structured_output_with_wrong_schema_version(tmp_path, monkeypatch): _enable_agy_structured_output(monkeypatch) structured = _payload(structured_pr_review()) @@ -2977,7 +2978,7 @@ def test_validated_turn_rejects_structured_output_with_wrong_schema_version(tmp_ ) assert "schema_version" in str(excinfo.value) cmds = [c for c, _ in runner.commands if c[:1] == ["agy"]] - assert len(cmds) == 1 and "--json-schema" in cmds[0] + assert len(cmds) == 1 and "--output-format" in cmds[0] and "--json-schema" not in cmds[0] @pytest.mark.parametrize( @@ -3152,8 +3153,8 @@ def test_validated_turn_replays_once_without_schema_after_request_rejection(tmp_ cmds = _agy_cmds(runner) assert len(cmds) == 2 - assert "--json-schema" in cmds[0] and "--output-format" in cmds[0] - assert "--json-schema" not in cmds[1] and "--output-format" not in cmds[1] + assert "--output-format" in cmds[0] and "--json-schema" not in cmds[0] + assert "--output-format" not in cmds[1] assert cmds[0][cmds[0].index("--model") + 1] == cmds[1][cmds[1].index("--model") + 1] # Same prompt (each attempt has its own response path), not a corrective re-prompt. assert re.sub(r"turn-[0-9a-f]+", "turn-X", cmds[0][-1]) == re.sub(r"turn-[0-9a-f]+", "turn-X", cmds[1][-1]) @@ -3183,7 +3184,7 @@ def test_validated_turn_replays_after_silent_exit_with_rejection_in_agy_log( cmds = _agy_cmds(runner) assert len(cmds) == 2 - assert "--json-schema" in cmds[0] and "--json-schema" not in cmds[1] + assert "--output-format" in cmds[0] and "--output-format" not in cmds[1] assert repairs == [] assert sum(_REPLAY_WARNING in line for line in logs) == 1 assert "After silent rejection." in response.text @@ -3207,7 +3208,7 @@ def test_failed_replay_is_classified_normally_without_a_second_fallback(tmp_path cmds = _agy_cmds(runner) assert len(cmds) == 2 - assert "--json-schema" not in cmds[1] + assert "--output-format" not in cmds[1] assert sum(_REPLAY_WARNING in line for line in logs) == 1 assert "exited with 3" in str(excinfo.value) @@ -3276,5 +3277,308 @@ def test_success_envelope_prose_quoting_the_rejection_gets_no_replay(tmp_path, m cmds = _agy_cmds(runner) assert not any(_REPLAY_WARNING in line for line in logs) - # Any later attempt keeps the schema flags: no free schema-off attempt. - assert all("--json-schema" in cmd for cmd in cmds) + # Any later attempt keeps the JSON flag: no free structured-off attempt. + assert all("--output-format" in cmd for cmd in cmds) + + +# --- Envelope response recovery and ambiguity (#2049) ------------------------- + +_PROSE_NOISE_BEFORE = ( + "1. I read the diff.\n2 files changed; tests pass: true, flaky: null.\n" + "See step [2] and the list [1, 2, 3].\n" +) +_PROSE_NOISE_AFTER = "\nDone. Count: 3, ok: true, missing: null, items [2].\n" + + +def _recover(response, kind="pr_review"): + from coding_review_agent_loop.agents.antigravity_structured import recover_response_payload + + return recover_response_payload(response, kind) + + +def _fenced(payload): + return f"```json\n{json.dumps(payload, indent=2)}\n```" + + +@pytest.mark.parametrize("fenced", [True, False], ids=["fenced", "bare"]) +def test_recovery_counts_one_object_amid_prose_scalars_and_arrays(fenced): + from coding_review_agent_loop.agents.antigravity_structured import RecoveredPayload + + payload = _payload(structured_pr_review(state="blocking", blocking_items=["Fix [2] now."])) + body = _fenced(payload) if fenced else json.dumps(payload) + outcome = _recover(_PROSE_NOISE_BEFORE + body + _PROSE_NOISE_AFTER) + assert isinstance(outcome, RecoveredPayload) + assert outcome.payload == payload # never edited + + +def test_recovery_ignores_a_footer_quoted_inside_the_object(): + from coding_review_agent_loop.agents.antigravity_structured import RecoveredPayload + + payload = _payload(structured_pr_review( + summary="The doc example shows as a footer.", + )) + outcome = _recover("Narration.\n" + _fenced(payload)) + assert isinstance(outcome, RecoveredPayload) and outcome.payload["state"] == "approved" + + +def test_recovery_accepts_a_matching_footer_and_agent_unavailable(): + from coding_review_agent_loop.agents.antigravity_structured import RecoveredPayload + + assert isinstance(_recover(structured_pr_review()), RecoveredPayload) + unavailable = {"schema_version": 1, "kind": "agent_unavailable", "retryable": True, + "category": "tooling", "summary": "x", "suggested_action": "y"} + assert isinstance(_recover(_fenced(unavailable)), RecoveredPayload) + assert _recover("No JSON here, just [1, 2] and true.") is None + assert _recover("") is None + # An undecodable brace is skipped, not counted. + assert _recover("A {broken brace and prose.") is None + + +@pytest.mark.parametrize( + "response,reason,count", + [ + ( + _fenced(_payload(structured_pr_review())) + "\n" + _fenced(_payload(structured_pr_review(state="blocking", blocking_items=["x"]))), + "multiple-candidates", 2, + ), + ( + json.dumps(_payload(structured_pr_review(state="blocking", blocking_items=["Before."]))) + + f"\n{PUBLIC_RESPONSE_MARKER}\n" + structured_pr_review(summary="After the marker."), + "multiple-candidates", 2, + ), + ("Intro {\"note\": 1}\n" + _fenced(_payload(structured_pr_review())), "multiple-candidates", 2), + (_fenced(_payload(structured_plan_review())), "wrong-kind", 1), + (_fenced({"schema_version": 1, "state": "approved", "summary": "x"}), "wrong-kind", 1), + (_fenced(_payload(structured_pr_review())) + "\n", "footer-state-conflict", 1), + ], + ids=["two-fences", "across-marker", "stray-object", "other-kind", "no-kind", "footer-conflict"], +) +def test_recovery_ambiguity_carries_only_a_reason_code_and_count(response, reason, count): + from coding_review_agent_loop.agents.antigravity_structured import AmbiguousPayload + + outcome = _recover(response) + assert isinstance(outcome, AmbiguousPayload) + assert (outcome.reason, outcome.candidate_count) == (reason, count) + assert outcome.diagnostic.startswith(f"Antigravity response carried {count} ") + assert reason in outcome.diagnostic and "no payload selected" in outcome.diagnostic + for model_text in ("After the marker", "Review complete", "Before."): + assert model_text not in outcome.diagnostic + + +def test_backend_recovers_a_fenced_payload_when_no_response_file_exists(tmp_path, monkeypatch): + from coding_review_agent_loop.agents.antigravity import AntigravityBackend + + _enable_agy_structured_output(monkeypatch) + payload = _payload(structured_plan_review(state="blocking", blocking_plan_issues=["Step 3 is unsafe."])) + stdout = _agy_envelope(response=_PROSE_NOISE_BEFORE + _fenced(payload) + _PROSE_NOISE_AFTER) + runner = FakeRunner(antigravity_outputs=[(stdout, 0)]) + config = _structured_config(tmp_path, "plan_review") + + result = AntigravityBackend().run(runner, config, "Review this plan.", run_id="r", role="reviewer") + + assert result.response_file_text is None + assert result.text_source == "structured_output" + assert result.antigravity_structured_ambiguity is None + assert "" in result.text + assert result.text.rstrip().endswith("-- Google Antigravity: Gemini 4 Hydrogen (Medium)") + parsed = parse_plan_review(result.text, reviewer="Antigravity") + assert parsed.state == "blocking" + assert "Step 3 is unsafe." in result.text + + +def test_backend_blank_state_object_falls_through_to_the_marker_path(tmp_path, monkeypatch): + from coding_review_agent_loop.agents.antigravity import AntigravityBackend + + _enable_agy_structured_output(monkeypatch) + payload = _payload(structured_pr_review()) + payload["state"] = " " + response = "Narration.\n" + _fenced(payload) + runner = FakeRunner(antigravity_outputs=[(_agy_envelope(response=response), 0)]) + + result = AntigravityBackend().run(runner, _structured_config(tmp_path), "Review.", run_id="r", role="reviewer") + + assert result.text_source == "stdout" + assert result.text == response # not rendered, no state invented + assert "AGENT_STATE" not in result.text and "Google Antigravity" not in result.text + assert result.antigravity_structured_ambiguity is None + + +def test_backend_ambiguous_response_returns_empty_text_and_a_typed_diagnostic(tmp_path, monkeypatch): + from coding_review_agent_loop.agents.antigravity import AntigravityBackend + + _enable_agy_structured_output(monkeypatch) + response = structured_pr_review(summary="First.") + f"\n{PUBLIC_RESPONSE_MARKER}\n" + structured_pr_review(summary="Second.") + stdout = _agy_envelope(response=response) + runner = FakeRunner(antigravity_outputs=[(stdout, 0)]) + + result = AntigravityBackend().run(runner, _structured_config(tmp_path), "Review.", run_id="r", role="reviewer") + + assert result.text == "" and result.text_source == "stdout" + assert result.raw_output == stdout # kept for audit + assert result.antigravity_structured_ambiguity == ( + "Antigravity response carried 2 candidate JSON objects (multiple-candidates); " + "no payload selected" + ) + assert result.provider_error_text is None + + +def test_backend_non_success_envelope_never_recovers_from_prose(tmp_path, monkeypatch): + from coding_review_agent_loop.agents.antigravity import AntigravityBackend + + _enable_agy_structured_output(monkeypatch) + payload = _payload(structured_pr_review()) + stdout = _agy_envelope(payload, status="ERROR", error="RESOURCE_EXHAUSTED (code 429)") + runner = FakeRunner(antigravity_outputs=[(stdout, 0)]) + + result = AntigravityBackend().run(runner, _structured_config(tmp_path), "Review.", run_id="r", role="reviewer") + + assert result.provider_error_text == "RESOURCE_EXHAUSTED (code 429)" + assert result.text_source == "stdout" + assert "Google Antigravity" not in result.text # not rendered in code + assert result.antigravity_structured_ambiguity is None + + +def test_validated_turn_accepts_a_single_recovered_object_amid_noise(tmp_path, monkeypatch): + _enable_agy_structured_output(monkeypatch) + payload = _payload(structured_pr_review(state="blocking", blocking_items=["Fix calc.py:3."])) + response = _PROSE_NOISE_BEFORE + json.dumps(payload) + _PROSE_NOISE_AFTER + runner = FakeRunner(antigravity_outputs=[(_agy_envelope(response=response), 0)]) + config = make_config(tmp_path, antigravity_model="Gemini 3.8 Flash (High)", agent_max_retries=0) + logs: list[str] = [] + repairs: list[int] = [] + + response_obj = _run_structured_review_turn(runner, config, logs, repairs) + + assert repairs == [] + assert response_obj.marker_value.state == "blocking" + assert "Fix calc.py:3." in response_obj.text + + +_PROVIDER_LIKE = "quota exhausted; RESOURCE_EXHAUSTED (code 429); unauthorized: invalid api key" + + +def _ambiguous_envelope(): + first = _payload(structured_pr_review(summary=f"First: {_PROVIDER_LIKE}")) + second = _payload(structured_pr_review(state="blocking", blocking_items=[_PROVIDER_LIKE])) + return _agy_envelope(response=f"{_fenced(first)}\n{_PROVIDER_LIKE}\n{_fenced(second)}") + + +def _spy_classifiers(monkeypatch): + from coding_review_agent_loop import validated_agent + + calls: list[str] = [] + real_capacity = validated_agent.classify_antigravity_capacity + real_failure = validated_agent._agent_failure_classification + + def capacity(text, **kwargs): + calls.append(f"capacity:{text}") + return real_capacity(text, **kwargs) + + def failure(result, **kwargs): + calls.append("failure-classification") + return real_failure(result, **kwargs) + + monkeypatch.setattr(validated_agent, "classify_antigravity_capacity", capacity) + monkeypatch.setattr(validated_agent, "_agent_failure_classification", failure) + return calls + + +def test_ambiguity_is_a_trusted_format_failure_that_falls_back_down_the_chain(tmp_path, monkeypatch): + _enable_agy_structured_output(monkeypatch) + calls = _spy_classifiers(monkeypatch) + valid = _agy_envelope(_payload(structured_pr_review(summary="From Model B."))) + runner = FakeRunner( + antigravity_outputs=[(_ambiguous_envelope(), 0), (_ambiguous_envelope(), 0), (valid, 0)], + ) + config = make_config( + tmp_path, + antigravity_models=("Model A", "Model B"), + agent_max_retries=1, + agent_retry_backoff_seconds=0, + ) + logs: list[str] = [] + repairs: list[int] = [] + + response = _run_structured_review_turn(runner, config, logs, repairs) + + models = [c[c.index("--model") + 1] for c in _agy_cmds(runner)] + assert models == ["Model A", "Model A", "Model B"] + assert repairs == [] + assert calls == [] # no capacity/quota/auth classification of model text + assert any("Model A produced no recoverable structured payload 2 times" in line for line in logs) + assert not any("quota" in line.lower() and "skipping" in line.lower() for line in logs) + assert not any("agent response was empty" in line for line in logs) + assert response.marker_value.state == "approved" + assert "From Model B." in response.text + + +def test_ambiguity_classification_text_is_only_the_trusted_diagnostic(tmp_path, monkeypatch): + _enable_agy_structured_output(monkeypatch) + calls = _spy_classifiers(monkeypatch) + runner = FakeRunner(antigravity_outputs=[(_ambiguous_envelope(), 0)]) + config = make_config(tmp_path, antigravity_model="Gemini 3.8 Flash (High)", agent_max_retries=0) + logs: list[str] = [] + repairs: list[int] = [] + from coding_review_agent_loop import validated_agent + + captured: list[dict] = [] + real_diagnostics = validated_agent._failed_run_diagnostics + + def diagnostics(**kwargs): + captured.append(kwargs) + return real_diagnostics(**kwargs) + + monkeypatch.setattr(validated_agent, "_failed_run_diagnostics", diagnostics) + + with pytest.raises(AgentInvocationError) as excinfo: + _run_structured_review_turn(runner, config, logs, repairs) + + diagnostic = ( + "Antigravity response carried 2 candidate JSON objects (multiple-candidates); " + "no payload selected" + ) + assert excinfo.value.failure_category == "agent-unavailable" + assert repairs == [] and calls == [] + assert captured[-1]["classification_text"] == diagnostic + assert captured[-1]["failure_reason"] == diagnostic + assert len(_agy_cmds(runner)) == 1 + + +@pytest.mark.parametrize( + "kind,parse", + [ + ("pr_review", lambda text: parse_pr_review(text, reviewer="Antigravity")), + ("plan_review", lambda text: parse_plan_review(text, reviewer="Antigravity")), + ], + ids=["pr_review", "plan_review"], +) +def test_bare_approved_envelope_payload_never_yields_an_approval(tmp_path, monkeypatch, kind, parse): + """#2049 item 3: {schema_version, kind, state: approved} is no review.""" + from coding_review_agent_loop import validated_agent + + _enable_agy_structured_output(monkeypatch) + bare = {"schema_version": 1, "kind": kind, "state": "approved"} + runner = FakeRunner(antigravity_outputs=[(_agy_envelope(response=_fenced(bare)), 0)]) + config = make_config(tmp_path, antigravity_model="Gemini 3.8 Flash (High)", agent_max_retries=0) + backend_repairs: list[int] = [] + + def record_backend_repair(*_a, **_k): + backend_repairs.append(1) + return None + + monkeypatch.setattr(validated_agent, "attempt_repair", record_backend_repair) + with pytest.raises(AgentInvocationError) as excinfo: + _run_validated_agent( + runner, + agent="antigravity", + config=config, + prompt="Review.", + marker_description=f"structured {kind}", + validate=parse, + use_repair=True, + repair_expected_kind=kind, + role="reviewer", + ) + assert backend_repairs == [] + assert excinfo.value.failure_category == "agent-unavailable" + assert "carries only envelope keys" in str(excinfo.value) diff --git a/tests/test_antigravity_quota_groups.py b/tests/test_antigravity_quota_groups.py index 981c2748..666535ed 100644 --- a/tests/test_antigravity_quota_groups.py +++ b/tests/test_antigravity_quota_groups.py @@ -1410,12 +1410,9 @@ def test_json_mode_quota_frame_is_classified_and_falls_back_by_quota( tmp_path, monkeypatch, returncode, response ): """The decoded 429 frame stays terminal for classification in every phase (#2029).""" - import json as json_module - _structured_on(monkeypatch) failure = _json_envelope(status="ERROR", response=response, error=LIVE_SAMPLE) - valid_payload = json_module.JSONDecoder().raw_decode(structured_pr_review())[0] - success = _json_envelope(status="SUCCESS", response="prose", structured_output=valid_payload) + success = _json_envelope(status="SUCCESS", response="prose\n" + structured_pr_review()) with workdir_claims.workdir_claim_scope(): response, error, models, logs, repairs = _review_turn( tmp_path, [(failure, returncode), (success, 0)], models=(GEM1, OPUS), @@ -1429,13 +1426,10 @@ def test_json_mode_quota_frame_is_classified_and_falls_back_by_quota( def test_json_mode_silent_envelope_still_reads_the_agy_log(tmp_path, monkeypatch, _isolated_agy_log_dir): - import json as json_module - _structured_on(monkeypatch) # Only agy's known no-output warning surrounds an empty envelope. silent = _json_envelope(AGY_SILENT_WARNING, status="ERROR", response="") - valid_payload = json_module.JSONDecoder().raw_decode(structured_pr_review())[0] - success = _json_envelope(status="SUCCESS", response="", structured_output=valid_payload) + success = _json_envelope(status="SUCCESS", response=structured_pr_review()) runner = AgyLogRunner( antigravity_outputs=[(silent, 0), (success, 0)], log_dir=_isolated_agy_log_dir, agy_logs=[AGY_429_LOG, None], diff --git a/tests/test_repair.py b/tests/test_repair.py index 7f2d64fd..62075303 100644 --- a/tests/test_repair.py +++ b/tests/test_repair.py @@ -5858,3 +5858,41 @@ def test_configured_reviewer_repair_carrying_dispositions_field_is_accepted(tmp_ assert parsed.dropped_human_requirement_dispositions is True assert parsed.blocking_items == without.blocking_items assert parsed.dispositions == without.dispositions + + +# --- Issue #2049: envelope-only reviewer payloads ------------------------------ + + +@pytest.mark.parametrize("expected_kind", ["pr_review", "plan_review"]) +@pytest.mark.parametrize("state", ["approved", "blocking", None]) +@pytest.mark.parametrize("fenced", [False, True], ids=["bare", "fenced"]) +def test_envelope_only_reviewer_payload_is_refused_repair(expected_kind, state, fenced): + payload = {"schema_version": 1, "kind": expected_kind} + if state is not None: + payload["state"] = state + body = json.dumps(payload) + source = f"```json\n{body}\n```" if fenced else body + with pytest.raises(ReviewSubstanceIntegrityError, match="only envelope keys"): + require_recoverable_review_substance(source, expected_kind=expected_kind) + + +def test_envelope_only_approval_fails_the_strict_parsers(): + from coding_review_agent_loop.protocol import parse_plan_review, parse_pr_review + + for kind, footer, parse in ( + ("pr_review", "AGENT_STATE", parse_pr_review), + ("plan_review", "AGENT_PLAN_STATE", parse_plan_review), + ): + text = ( + json.dumps({"schema_version": 1, "kind": kind, "state": "approved"}) + + f"\n\n\n-- Google Antigravity: Gemini" + ) + with pytest.raises(AgentLoopError): + parse(text, reviewer="Antigravity") + + +def test_envelope_plus_one_review_field_is_still_admitted(): + require_recoverable_review_substance( + json.dumps({"schema_version": 1, "kind": "pr_review", "state": "approved", "summary": "LGTM."}), + expected_kind="pr_review", + ) diff --git a/tests/test_validated_agent_recovery.py b/tests/test_validated_agent_recovery.py index dba98274..134d2bef 100644 --- a/tests/test_validated_agent_recovery.py +++ b/tests/test_validated_agent_recovery.py @@ -2375,7 +2375,7 @@ def record_repair(*_a, **_k): ) cmds = [c for c, _ in runner.commands if c[:1] == ["agy"]] - assert all("--json-schema" in cmd for cmd in cmds) + assert all("--output-format" in cmd and "--json-schema" not in cmd for cmd in cmds) assert [c[c.index("--model") + 1] for c in cmds] == ["Model A"] assert excinfo.value.failure_category == "non-retryable" assert repairs == []