Quest: Run Codex code review from the README as an Actions check

[taras]

Quest outcome

Run repository code review as an explicit root README operation. XMD invokes
Codex through its ACPX Agent integration, and GitHub Actions presents the
result in the review job's Markdown summary instead of creating or updating a
pull-request issue comment.

A contributor can inspect the README to understand the review and can run the
same entrypoint without knowing about hidden .reviews/ documents:

./dist/xmd run --plugin ./packages/code-review-agent/mod.ts README.md#Review --approve-reads

In CI, the PR Review job is the GitHub Actions check. Its summary contains the
review report. The Quest does not create a second custom Check Run.

The bundled Git Plugin, including its GitHub adapter, is active by default in the run/workflow profile established by #822. The supported local and CI command explicitly selects the trusted code-review Plugin before importing the README; merely having that optional package or its component names in the checkout does not activate it. The GitHub adapter remains inert until a GitHub-backed operation invokes it.

Current gap

The review workflow currently invokes hidden executable documents, sends their
prompts to hosted models selected outside the README, and publishes the result
as a marked pull-request comment and separate inline review comments. Most of
the local and hosted documents are duplicated.

CI also loads the reviewer from the pull request checkout while supplying the
credentials used by the reviewer. A pull request can therefore alter the code
that reviews it before that code is trusted.

The current composition gives separate policies broad pull-request and
diagnostic objects, lets some of them invoke the model independently, and has no
documented, bounded way for the model to obtain evidence omitted from its
prompt or correct a malformed final response.

End-to-end contract

The root README visibly composes one review pipeline:

exact pull-request subject
  → deterministic environment, change and lint evidence
  → independently testable review-check components
  → one review request plus the available read-only XMD syntax
  → at most four Codex Prompt turns through ACPX
  → validated structured findings
  → rendered Markdown report
  → GitHub Actions job summary

Each review check is its own component. It receives the immutable pull-request
subject and only the evidence relevant to its question. It returns plain
structural data and receives no credential, provider handle, publication capability or mutable shared review state. A deterministic check may return its
result directly. Agent-assisted checks contribute typed requests to one logical
review so Codex can apply shared context and reconcile overlapping findings.
Separate report sections do not create separate Agent turns.

The review uses <Agent name="codex">, one named review session and ordinary
<Prompt> turns. It does not use the native interactive <Session.Launch>
surface or the standalone OpenAI Codex GitHub Action.

Before the first Prompt, the document obtains a compact syntax catalog from the
same read-only evaluation profile that will answer later information requests.
That generated catalog appears in the initial Prompt exactly once; it is not a
hand-maintained list. Codex may use bare <Syntax /> to inspect that available
surface or <Syntax names={...} /> to read selected component documentation.
Documentation never grants execution control.

Codex receives the repository policy, the applicable check requests, selected
evidence, and access to inspect the exact pull-request subject. Its response is
parsed against a fixed schema as either:

• a complete review for the supplied head commit; or
• an information request carrying a reason and a read-only XMD program.

An information request is a short circuit. It contains no provisional
findings, is not rendered as review output, and runs through <Evaluate allow={["read"]}> under the trusted host's review-read profile. The profile
includes File, Glob, Syntax and Json, exact pull-request changes, and
pull-request comments, reviews and checks. A request may compose those reads and
render only the observations it wants returned to the next Prompt. It cannot
write, execute a command, reach an undeclared network surface or obtain a
credential.

The README owns one bound of four total Prompt turns for the logical review.
Information continuations and schema-repair continuations share that bound. A
malformed response receives its schema diagnostics while another turn remains;
a refused information request receives its safe normalized reason. No fifth
Prompt starts, and a fourth response that is not a valid complete review ends
the job without publishing findings.

Only a complete, schema-valid response becomes the report. Deterministic
sensors remain evidence for the review; Codex supplies judgment rather than
replacing facts the repository can compute directly. The renderer organizes
the final result by check and does not need another Agent turn.

Review findings are advisory. A successful review with findings leaves the job
successful and makes the findings visible in its summary. Failure to collect
required evidence, obtain the syntax catalog, start or complete a Codex turn,
execute an authorized information request, obtain a valid complete review
within four turns, or publish the summary fails the job.

Trust boundary

CI executes the xmd binary, bundled default Git Plugin with its GitHub adapter, explicitly selected code-review Plugin, README entrypoint, review components, read profile, policy, prompt and response schema from the trusted base revision. The
pull request is supplied as subject data and may not replace those sources
during its own review.

The Codex session uses ACPX's approve-reads permission policy and receives no
credential capable of changing repository collaboration state. Information
requests receive only the exact component forms and host operations declared by
the review-read profile. GitHub's job summary channel needs no issue-comment or
Checks API write permission.

This is a trusted CI-agent profile, not an operating-system sandbox. The Quest does not
claim that ACPX prevents every direct filesystem or network action an agent
process could attempt. The ephemeral runner, withheld mutation credentials,
trusted reviewer source, declared permission policy and bounded XMD evaluation
profile define the delivered boundary. Stronger portable Agent enforcement
remains separate work.

Child map

• Make executable document entrypoints configurable #533 — make a document require an exact target so README help remains inspection-only and a bare invocation cannot start review. It is open and has no remaining dependency in this Quest.
• Make review and analysis components independent of repository include paths #83 — package the first-party review graph outside checkout-relative component includes and preserve its anti-shadowing contract. It was delivered by PR 🛡️ Take the review component graph out of the checkout (#83) #817.
• Quest: Extend XMD with trusted installed plugins #818 — load explicitly selected trusted Plugins before root import and supply one declaration catalog to run, syntax, validation, Plan and nested execution. It was delivered by PR ✨ Install trusted Plugins before a document runs (#818) #821 and migrated the code-review package away from CLI-specific attachment.
• Make Git and GitHub available in XMD runs and workflows #822 — extract the complete repository collaboration surface, including the lazy GitHub adapter, into one bundled default Git Plugin. It is open and can proceed in parallel with Make executable document entrypoints configurable #533.
• Read a pull request and its changes from executable documents #816 — provide normalized details and exact changes for one known pull-request URL through the Git/GitHub boundary. It is open and follows Make Git and GitHub available in XMD runs and workflows #822.
• Run Codex code review from the root README as an Actions check #534 — compose the check components and bounded Codex review in the root README, use the bundled GitHub adapter, explicitly select the code-review Plugin from the trusted base, treat the pull request as subject data, and deliver the report through the Actions job summary. It is open and depends on Make executable document entrypoints configurable #533 and Read a pull request and its changes from executable documents #816.

Completion

The Quest is complete when:

• README help identifies the exact review target without executing it;
• the bundled Git Plugin, including its GitHub adapter, is active once, while the supported local and CI command explicitly selects the trusted code-review Plugin and invokes the README target through XMD;
• each review check owns one typed evidence-and-question boundary while one
  logical Codex review judges all applicable Agent-assisted checks;
• the initial Prompt receives the catalog produced by the exact read profile,
  and information requests can inspect that profile through <Syntax />;
• XMD selects Codex through ACPX and permits at most four Prompt turns shared by
  information requests and schema repairs;
• only a complete review for the exact head commit passes structured response
  validation and reaches the report;
• the Actions review job shows the rendered report in its Markdown summary and
  creates no pull-request or inline review comment;
• findings remain advisory while incomplete review and infrastructure failures
  fail the job;
• CI runs trusted reviewer sources and treats the pull request as subject data;
• the Agent and generated information requests receive no GitHub
  collaboration-state write permission; and
• focused positive and negative controls distinguish each of those behaviors.

Related work

• Search GitHub issues and pull requests in the current repository #815 owns bounded listing of recent pull requests for analysis workflows. It
  is independent of this Quest because review starts from the current pull
  request's known URL.
• Keep comments and strings out of unused-declaration review findings #51 and Review bot false positives on docs/spec PRs from markdown code samples #53 own remaining false-positive cases in review findings. They are
  quality follow-ups, not prerequisites for moving the review pipeline.
• Quest: Restrict XMD workflow effects to explicitly granted permissions #536 owns general least-privilege enforcement across XMD workflows.
• Certify ACP agents against real backends through xmd run #743 owns opt-in qualification of ACP Agents against their real backends; it
  is not part of ordinary review CI.

Out of scope

• Creating a separate custom Check Run or line annotations through the Checks
  API.
• Making review findings block delivery.
• Launching Codex's native interactive UI.
• Claiming a general Agent, process, filesystem or network sandbox.
• Moving the repository-analysis workflow into the README.

[taras]

Current and proposed code-review flow

GitHub continues to trigger the workflow. The proposed change makes
README.md#Review the visible XMD entrypoint, and the resulting PR Review
Actions job is the check. It does not create a separate Check Run through the
Checks API.

flowchart LR
  subgraph Current["Current CI review"]
    direction TB
    C1["PR opened, synchronized,<br/>or reopened"]
    C2["Check out PR revision<br/>with reviewer code"]
    C3["Build XMD and run hidden<br/>.reviews/ReviewPR.md"]
    C4["Collect base…head diff,<br/>file statistics and PR body"]

    C5["Deterministic heuristics<br/>thresholds, regexes and Oxlint"]
    C6["Qwen via DeepInfra<br/>comment cleanup and<br/>extraneous-code judgment"]

    C7["Render scope, bloat,<br/>slop and correctness sections"]
    C8["Create or update PR<br/>conversation comment"]
    C9["Post separate inline review<br/>with comment-removal suggestions"]
    C10["Later runs may repeat<br/>findings across revisions"]

    C1 --> C2 --> C3 --> C4
    C4 --> C5
    C4 --> C6
    C5 --> C7
    C6 --> C7
    C7 --> C8
    C6 --> C9 --> C10
  end

  subgraph Proposed["Proposed README and Codex review"]
    direction TB
    P1["PR event starts<br/>PR Review job"]
    P2["Load XMD, README target,<br/>policies and prompts from trusted base"]
    P3["Run<br/>xmd run README.md#Review"]
    P4["Treat PR as subject data<br/>at an exact head SHA"]
    P5["Collect diff, repository rules,<br/>PR context, lint and environment evidence"]
    P6["Normalize evidence<br/>changed-line attribution,<br/>baseline comparison and semantic churn"]
    P7["Codex through ACPX<br/>approve-reads; no mutation credentials"]
    P8{"Response satisfies fixed schema<br/>and finding evidence contract?"}

    P9["Reconcile stable finding IDs<br/>across channels and revisions"]
    P10["Render one Markdown report<br/>to Actions job summary"]
    P11["Findings are advisory:<br/>job succeeds"]
    P12["Evidence, agent, schema or<br/>publication failure: job fails"]

    P1 --> P2 --> P3 --> P4 --> P5 --> P6 --> P7 --> P8
    P8 -- Yes --> P9 --> P10 --> P11
    P8 -- No --> P12
  end

Loading

What gets checked

Concern	Current mechanism	Proposed mechanism
PR size	Fixed 400/800-line, 20-file and 5-directory thresholds	Report semantic size separately from generated or mechanical churn; request a split only when Codex can name an independently deliverable boundary
Description and issue context	Description length, linked-issue regex and simple keyword checks	Read the complete description and suppress concerns already explained by the author
Scope	Directory breadth and config/source mixing	Identify a specific unrelated change or architectural divergence and explain its consequence
Dependencies	Warn when package.json changes without the word dependenc in the description	Compare the actual dependency change with repository policy and the author's explanation
Abstractions	Filename patterns and consumers visible in the diff	Apply established architecture vocabulary and lifecycle rules to the changed abstraction
Static analysis	Run Oxlint over whole changed files and summarize rule counts	Compare base to head and consider only introduced or changed-line diagnostics; require a concrete consequence
Comment quality	Send adjacent comment/code pairs to the model and look for narration	Distinguish narration from rationale about invariants, ownership, ordering, cancellation, identity and compatibility
Extraneous code	Qwen reviews a truncated diff preview when the PR exceeds 20 lines	Codex evaluates normalized evidence and may inspect the exact PR subject under read-only permissions
Release configuration	Warn when release configuration changes without the release spec	Retain as deterministic repository evidence
Repeated findings	Location-based partial state plus fresh inline reviews	Stable identities reconcile current, resolved, moved, repeated and superseded findings
Clean result	Post “no concerns” sections into the PR conversation	Show success in the Actions summary without adding a PR comment
Review findings	Write to PR comments and inline reviews	Write to the PR Review job summary; findings remain non-blocking
Infrastructure failure	Mixed into the workflow outcome	Missing evidence, Codex failure, invalid schema or summary publication failure fails the job

The proposed evidence contract is:

detect a signal
  → establish base-to-head or changed-line attribution
  → identify an observable consequence or worthwhile cleanup
  → reconcile it with findings from earlier revisions
  → publish one actionable result

The attribution, rationale-aware comment review, reconciliation and semantic-size
steps come from the ten-PR feedback analysis. They refine review quality on top
of this Quest's accepted README, ACPX, Codex, trust-boundary and Actions-summary
contract.