Skip to content

feat(drain): post replies to waiting review threads (STA-351 PR B) - #175

Merged
SutuSebastian merged 14 commits into
mainfrom
agent/sta-351-drain-b
Nov 17, 2026
Merged

SutuSebastian merged 14 commits into
mainfrom
agent/sta-351-drain-b

Conversation

@SutuSebastian

Copy link
Copy Markdown
Contributor

Summary

STA-351 PR B of 2: verdict drain --post answers pending review-thread replies on the host. PR A (#174) added plan and drafts. Spec: STA-351 spec — verdict drain.

 verdict drain owner/repo 42 [--agent <name>] [--post] [--json]
   plan (refresh threads → drain.json) → --agent: draft drain.replies.json
+  --post:
+    local checks first: ReadPlan → CheckDrafts → ThreadReply capability → secret scan   # zero host calls on refusal
+    lock <bundle>/drain (two --post runs serialize)
+    refresh threads + self login
+    per draft: skip resolved | stale-thread (someone spoke since the draft) | already-replied (ledger)
+      NeutralizeMentions + marker `<!-- verdict-reply thread=… to=… -->`
+      gh api --method POST …/pulls/N/comments/{root}/replies -f body=…   |  glab …/discussions/{id}/notes (stdin JSON)
+      append the replies.json ledger entry right after each write
+      withdraw → resolve the thread (existing resolve write)
+    drain.json receipt: posted[] / failed[] / posted_at; any failure → drain-incomplete (one JSON envelope, exit 1)
  • The one-run path drain R N --agent A --post plans, drafts, validates and posts in one run.
  • Idempotent:
    • The ledger is written per write.
    • A posted reply reads back as author_kind: verdict, so the next run sees the thread as answered.
    • A rerun after a partial failure posts nothing twice.
  • Hosts: GitHub and GitLab post. Others refuse provider-unimplemented before any read.
  • Discovery: drain is now listed in SKILL.md. That moves only the prompts_hash lines, the run_id and byte counts derived from it, and the provenance pin.

Evidence

  • Before: --post was an unknown flag.
  • After:
    • New evals: drain-post (answer + withdraw post 2 marked replies and 1 resolve; the ledger has 2 entries; the rerun makes 0 writes; the one-run --agent --post path works) and drain-stale. drain-refusals adds a secret-shaped body and a Forgejo --post (0 writes each).
    • Unit tests:
      • TestPostPartialFailureRerunIsIdempotent
      • TestPostLocalRefusalsMakeZeroHostCallsAndKeepThreadsUnchanged
      • TestPostCallOrderLocalChecksThenReadsThenWrites
      • TestPostConcurrentRunsSerialize (fails 3/3 runs without the lock)
      • TestPostIncompleteEmitsSingleJSONEnvelope
      • TestReplyMarkerRoundTrip
  • Reviews:
    • Round 1, Gemini 4 Hydrogen (independent of the Hy4 build): requested changes. The one-run path skipped plan and agent; next omitted --post; host reads came before the local checks; there were two JSON envelopes.
    • Fix round, built by Gemini 4 Hydrogen after a builder switch.
    • Final review: a direct review by a different-family reviewer. It approved, and each fix was mutation-probed.
  • Ladder at 81832f89b: all green.
    • make test-race, make vet, make deadcode, make cross, doc-coverage, make schemas-check, make ts-types-check, make tsgen && make build && make clients-ts and make size pass.
    • make evals: 140 passed, 0 failed.
    • golangci-lint (host and GOOS=windows): 0 issues.

Merge Danger

Door: two-way

--post is opt-in, and it is the only path in this PR that writes to a host. Plan and draft modes are unchanged from #174.

Blast Radius: small

  • Host side: the first verdict verb that replies inside threads. A wrong reply is visible on the PR, but it only goes out on an explicit --post, after the secret scan and stale checks.
  • Locally: SKILL.md hashes move in about 104 goldens.
  • Size: the baselines rise; headroom stays tight on windows/amd64 and darwin/arm64.

Follow-ups (advisory):

  • drain.json posted[] reflects only the last --post run; replies.json is the durable record.
  • The one-run path refreshes threads twice.
  • The plan is written outside the post lock.

@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 144 files, which is 44 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: dd8ba38c-2f49-4c9c-a11e-7941762fcb2d
Commits

Reviewing files that changed from the base of the PR and between f306775 and 32a54a2.

Files selected for processing: 144

  • .changeset/drain-post.md
  • apps/docs/content/04-roadmap/index.mdx
  • bench/runner/testdata/tool-free-prompt.golden
  • clients/ts/src/refusal-codes.gen.ts
  • cmd/verdict/usage.go
  • docs/manual.md
  • docs/troubleshooting.md
  • evals/README.md
  • evals/cases/drain-plan.sh
  • evals/cases/drain-post.sh
  • evals/cases/drain-refusals.sh
  • evals/cases/drain-stale.sh
  • internal/core/drain/ledger.go
  • internal/core/drain/ledger_test.go
  • internal/core/drain/plan.go
  • internal/core/drain/post.go
  • internal/core/drain/post_test.go
  • internal/core/hint.go
  • internal/core/hint_test.go
  • internal/core/hostcaps.go
  • internal/core/hostcaps_test.go
  • internal/core/post/host.go
  • internal/core/post/host_reply.go
  • internal/core/post/host_reply_test.go
  • internal/core/post/secretvalues.go
  • internal/core/refusalcodes.go
  • internal/core/testdata/hints.golden
  • internal/core/threads/reply_marker_test.go
  • internal/core/threads/threads.go
  • internal/drain/post.go
  • internal/drain/post_test.go
  • internal/drain/verb.go
  • internal/drain/verb_test.go
  • internal/drain/verb_unix_test.go
  • internal/fetch/testdata/author-advisory.golden
  • internal/fetch/testdata/azure-happy.golden
  • internal/fetch/testdata/base-sha-json.golden
  • internal/fetch/testdata/base-sha-note.golden
  • internal/fetch/testdata/bitbucket-happy.golden
  • internal/fetch/testdata/bitbucket-paging.golden
  • internal/fetch/testdata/budget-over-consented.golden
  • internal/fetch/testdata/budget-unknown.golden
  • internal/fetch/testdata/budget-unreadable.golden
  • internal/fetch/testdata/budget-unset.golden
  • internal/fetch/testdata/budget-within.golden
  • internal/fetch/testdata/cap-override-beats-chunk.golden
  • internal/fetch/testdata/cap-override.golden
  • internal/fetch/testdata/controls-base-absent.golden
  • internal/fetch/testdata/dump-json.golden
  • internal/fetch/testdata/dump-text.golden
  • internal/fetch/testdata/empty-diff.golden
  • internal/fetch/testdata/file-paging.golden
  • internal/fetch/testdata/forgejo-happy.golden
  • internal/fetch/testdata/gitcafe-base-note.golden
  • internal/fetch/testdata/gitcafe-happy.golden
  • internal/fetch/testdata/gitlab-base-note.golden
  • internal/fetch/testdata/gitlab-base.golden
  • internal/fetch/testdata/gitlab-happy.golden
  • internal/fetch/testdata/gitlab-text.golden
  • internal/fetch/testdata/happy-json.golden
  • internal/fetch/testdata/happy-text.golden
  • internal/fetch/testdata/local-branch.golden
  • internal/fetch/testdata/local-dump-chunk.golden
  • internal/fetch/testdata/local-dump-text.golden
  • internal/fetch/testdata/local-paths.golden
  • internal/fetch/testdata/local-range.golden
  • internal/fetch/testdata/local-range3.golden
  • internal/fetch/testdata/local-staged.golden
  • internal/fetch/testdata/local-stash.golden
  • internal/fetch/testdata/local-untracked.golden
  • internal/fetch/testdata/local-worktree-text.golden
  • internal/fetch/testdata/local-worktree.golden
  • internal/fetch/testdata/over-cap-chunk-text.golden
  • internal/fetch/testdata/over-cap-chunk.golden
  • internal/fetch/testdata/over-cap-knob.golden
  • internal/fetch/testdata/ratelimit-retry.golden
  • internal/fetch/testdata/sensitive-kb-head.golden
  • internal/fetch/testdata/sensitive-kb.golden
  • internal/fetch/testdata/since-alias.golden
  • internal/fetch/testdata/threads-degraded.golden
  • internal/provenance/prompts_test.go
  • internal/record/testdata/body/mermaid-stripped.golden
  • internal/record/testdata/body/non-ascii.golden
  • internal/record/testdata/body/receipt-collision.golden
  • internal/record/testdata/chunk/drift.golden
  • internal/record/testdata/chunk/unattested.golden
  • internal/record/testdata/chunk/unknown-chunk-claim.golden
  • internal/record/testdata/chunk/valid.golden
  • internal/record/testdata/claims/verify-triage-resolved/rereview.golden
  • internal/record/testdata/degenerate/basis-bad.golden
  • internal/record/testdata/degenerate/basis-ok.golden
  • internal/record/testdata/degenerate/blob-link-local.golden
  • internal/record/testdata/degenerate/brief-valid.golden
  • internal/record/testdata/degenerate/carried-footer.golden
  • internal/record/testdata/degenerate/confidence-valid.golden
  • internal/record/testdata/degenerate/fenced-body-section.golden
  • internal/record/testdata/degenerate/missing-marker-ok.golden
  • internal/record/testdata/degenerate/normalized-fenced.golden
  • internal/record/testdata/degenerate/normalized-prose.golden
  • internal/record/testdata/degenerate/normalized-weak-shapes.golden
  • internal/record/testdata/incremental/non-touching/rereview.golden
  • internal/record/testdata/incremental/reopen/rereview.golden
  • internal/record/testdata/kb/absent.golden
  • internal/record/testdata/kb/bad-id-shape.golden
  • internal/record/testdata/kb/bad-policy-env.golden
  • internal/record/testdata/kb/bad-shape.golden
  • internal/record/testdata/kb/broken-store.golden
  • internal/record/testdata/kb/counter-adds/rereview.golden
  • internal/record/testdata/kb/duplicate.golden
  • internal/record/testdata/kb/empty.golden
  • internal/record/testdata/kb/false-positive/rereview.golden
  • internal/record/testdata/kb/joins-idempotent/rereview.golden
  • internal/record/testdata/kb/no-store.golden
  • internal/record/testdata/kb/null.golden
  • internal/record/testdata/kb/retired.golden
  • internal/record/testdata/kb/string-counters/rereview.golden
  • internal/record/testdata/kb/truncated.golden
  • internal/record/testdata/kb/unknown.golden
  • internal/record/testdata/kb/valid.golden
  • internal/record/testdata/matrix/n10/first.golden
  • internal/record/testdata/matrix/n10/rereview.golden
  • internal/record/testdata/matrix/n200/first.golden
  • internal/record/testdata/matrix/n200/rereview.golden
  • internal/record/testdata/matrix/n50/first.golden
  • internal/record/testdata/matrix/n50/rereview.golden
  • internal/record/testdata/overrides/dir-as-fragment.golden
  • internal/record/testdata/overrides/eq-threshold.golden
  • internal/record/testdata/overrides/valid.golden
  • internal/record/testdata/provider/gitlab.golden
  • internal/record/testdata/refuse/empty-prior.golden
  • internal/record/testdata/refuse/false-prior.golden
  • internal/record/testdata/refuse/invalid-utf8-review.golden
  • internal/record/testdata/refuse/null-prior.golden
  • internal/record/testdata/source/dirty-head.golden
  • internal/record/testdata/source/dirty-string.golden
  • internal/record/testdata/uncite/null-keeps/rereview.golden
  • internal/record/testdata/verify/contradict-exit/rereview.golden
  • internal/record/testdata/verify/contradict-timeout/rereview.golden
  • internal/record/testdata/verify/on-addressed/rereview.golden
  • scripts/size-baseline.txt
  • scripts/verdict-complete.sh
  • skills/verdict/SKILL.md
  • skills/verdict/prompts/address-reviews.md
  • skills/verdict/references/commands.md

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@changeset-bot

changeset-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 32a54a2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@stainless-code/verdict Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@SutuSebastian
SutuSebastian merged commit b17943d into main Nov 17, 2026
21 checks passed
@SutuSebastian
SutuSebastian deleted the agent/sta-351-drain-b branch November 17, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant