feat: autonomy loops — post-run verification, /goal evaluator, CI watch, autonomy budget#59
Merged
Conversation
The deterministic "trust but verify" pass: fingerprint every workspace repo (HEAD + porcelain status) before a run, and after a cleanly successful run re-execute the check gate of exactly the repos the run changed, using the repo's OWN runner — Taskfile (task --dry probe), Makefile (make -n probe) or package.json scripts — preferring check > ci > test > tests > lint. A repo named on the retry list is re-gated even when unchanged, so a no-op "fix" run cannot silently exit a red-gate loop. Failures carry the output tail and a ready-to-inject fix-up prompt; a missing runner is a skip, not a failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/goal arms a per-chat objective persisted across restarts (single JSON file,
atomic rewrite, mirroring the other stores). The evaluator prompt scopes a
FRESH provider session to read-only inspection plus running checks and pins
the reply to a single JSON verdict; the parser scans balanced {...} spans
from the last one backwards and accepts the first carrying an "achieved"
key, so fences or stray prose around the object never break a round. The
fix-up prompt relays the verdict's unmet points back to the working session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User-triggered runs are bounded by the per-user cost cap and scheduled jobs by their creator's cap, but autonomy-originated runs (CI fix-ups, post-run verification fix-ups, goal-evaluator rounds) had no owner to meter. This store accumulates their cost per chat per UTC day — key "<chat>#<day>", atomic JSON rewrite, days older than the retention window pruned on write — and answers the reactive Allowed gate. A nil store disables the feature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… loop After a cleanly successful run (and only then), the service now runs an autonomy tail on the dispatcher lane: first the deterministic gate re-run over the repos this run changed (core/verify, snapshot taken at run start), then — on a clean tree — the /goal evaluator, an independent fresh-session judge with no shared context, whose unmet points loop back as a fix-up run. Both loops are bounded: consecutive verification fix-ups per chat, and evaluation rounds per goal, with the counters reset by a real user message. Follow-ups go through the new InjectAuto: like Inject it enqueues an auto-resume marker and uses the blocking Submit (a fix-up for work already paid for must survive a busy lane and a deploy), but the run is attributed to the AutoUser sentinel so its cost lands on the chat's per-day autonomy budget — which also gates the injection itself, pausing autonomous work (with one daily notice) once the day's budget is spent while direct user messages keep working. Scheduled fires and poller relays are untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ge greens Closes the CI feedback loop without a human relay. A background loop discovers every duck/<chatid>/... branch checked out in the per-chat workspaces (reading .git/HEAD and .git/config directly — no git exec) and polls the git host for that branch's CI state: GitHub check-runs with a legacy combined-status fallback, or Gitea's combined commit status, with the head SHA resolved via the slash-safe /commits?sha= query. A red build emits one event per (repo, branch, SHA) — deduplicated in a small durable store, across restarts — for the adapter to inject as a fix-up prompt. Optionally (AutoMerge) a branch whose CI is green gets its open PR merged, removing the last human touchpoint for deployments that want it. Like core/poller it polls because the bot can reach the host while inbound webhooks often can't. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ope) ENABLE_POST_VERIFY (on) + POST_VERIFY_MAX_FIXES/TIMEOUT for the post-run gate re-run; GOAL_MAX_ATTEMPTS / EVALUATOR_MODEL / GOAL_EVAL_* and the goal store path for the /goal evaluator; ENABLE_CI_WATCH (off) + CI_POLL_INTERVAL (floored to 30s) + ENABLE_AUTO_MERGE (off) + the dedupe-state path, enabled for github.com or a Gitea API URL and deliberately NOT requiring ENABLE_PR_REVIEW; AUTO_TASK_MAX_COST_PER_DAY + store path for the per-chat autonomy budget; AUTO_APPROVE_SCOPE normalized fail-safe to "off" so a typo can never silently skip the human approval gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The workspace renderer substitutes a fifth placeholder,
${AUTO_APPROVE_SCOPE}, defaulting an unset field to the fail-safe "off"
(always wait for scope approval). The tester subagent pins model: sonnet —
running suites and reporting failures verbatim is routine work that does not
need the lead model, per the cost advice of scaling the model to the role.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…watch /goal joins the reserved set (menu, guard test, both adapters): arm a goal, show it, /goal off to disarm. It is therefore no longer forwarded to the provider's own slash-command system — the bot-level loop persists across turns, meters the budget, and survives restarts, which the in-session command could not. Both mains assemble the post-run config (verification, goal evaluator, autonomy budget) with the usual non-fatal store opens; the Telegram main additionally starts the CI watch when configured, injecting red-build fix-ups into the owning chat and announcing auto-merged PRs. The VK Service seam gains the three goal methods. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the "prefer independent, flag-gated PRs" guidance: a single feature's work in a single repo must never be split across several PRs. Structure it as a sequence of coherent, reviewable commits on the ONE duck/<chatid>/<slug> branch and open ONE PR; review/CI follow-ups push more commits to the same PR. A multi-repo feature still gets exactly one PR per affected repo, and oversized asks are staged via a plan doc where each stage is one self-contained PR. The bottom-up merge rule stays as a recovery path for a chain that should not have existed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The chatgpt.com/codex installer resolves the release and its asset digests
through the unauthenticated GitHub API, which rate-limits shared CI runners
and then reports the platform assets missing ("Could not find Codex package
or platform npm release assets for Codex 0.143.0") — this broke the arm64
leg of the VK image build (the Telegram image survived on layer cache). One
arch-matched, retried, direct tarball download is deterministic and mirrors
how gh and task are already installed in these images. The /opt/codex
install-state dir goes away with the installer that needed it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Lane deadlock (critical): InjectAuto is called from postRun, which runs ON the lane's own worker; the blocking Submit against that lane's full 64-slot queue deadlocked the chat forever. Now TrySubmit first, and on a full lane the blocking wait moves to a detached goroutine (Submit is shutdown-safe; the auto-resume marker covers a crash mid-wait). Regression test fills the queue and asserts InjectAuto returns. - Durable fix-round cap: the consecutive-round counter now rides INSIDE the fix-up prompt (verify.FixPrompt embeds "Auto-fix round N of M", ParseFixRound reads it back), so a deploy mid-loop resumes at the same round instead of resetting an in-memory map — POST_VERIFY_MAX_FIXES is no longer per-process-lifetime. The verifyFixes map is gone. - Per-message git cost: the pre-run snapshot is now served from a cache refreshed at the END of each run (after delivery), so the 2-git-commands- per-repo cost leaves the user-visible critical path; only the first run after boot pays a synchronous Take. - Kill grace: verify.runCmd SIGTERMs the gate's process group and escalates to SIGKILL only after 2s, mirroring core/claude — a timed-out runner can tear down containers and temp state instead of orphaning them. - GOAL_MAX_ATTEMPTS floor: a non-positive cap stopped the /goal loop after its first evaluation round; config.EffectiveGoalMaxAttempts floors it to 1 (ArmGoal guards too), and the goal-cap notice no longer fuses the verdict summary into the next sentence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The branch-name convention routes host events to chats and was parsed independently by the poller and the CI watch — and the two disagreed: the poller accepted an empty chat segment (emitting a PRComment with an empty ChatID for a "duck//slug" ref, dropped only later by the adapter's ParseInt guard) while the CI watch rejected it. The contract now lives in core/teambranch with one ChatID() parser; the poller adopts it (an empty chat id is now skipped and marked read, like any non-team branch). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A green build now emits CIGreen (once per repo+branch+SHA, durable dedupe) whenever auto-merge did not consume the event. This closes the most-reported hole in the bot's feedback loop: an agent that told its user "I'll report when CI finishes" had NO wake-up signal for success — the PR poller only relays new review comments — so a green build ended the conversation until the user pinged. Auto-merge keeps its behavior (merge + notice, no extra event); a transient PR lookup/merge failure retries next cycle without consuming the green. The dedupe store is now pruned every scan against the live checkout set (a switched-away or deleted branch has no future observation to dedupe), so it tracks live branches instead of growing toward the wholesale-wipe backstop — whose reset would have re-emitted one duplicate fix-up per still-red branch. The watch also adopts core/teambranch for chat routing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both mains now assemble the post-run config via internal/autonomy.Build — the verbatim buildAutonomy copy in cmd/duck-vk is gone, so a wiring change cannot silently desynchronize the adapters (the same drift risk the review flagged). The Telegram CI-watch wiring routes the new CIGreen event as an InjectAuto prompt telling the agent to deliver the CI verdict it promised (or briefly confirm and continue), in the user's language. Docs updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…port back" The no-standby rule kept being rationalized around with a false belief: the agent assumes "the poller will wake me when the CI job finishes" and ends its turn promising to report. It won't be woken: the poller only relays NEW review comments from other people, and nothing starts a turn on a green build (only a deployment with CI watch enabled does, which the agent cannot see from inside the workspace). Spell that out and require waiting for a CI verdict in the FOREGROUND (gh pr checks --watch / a bounded API poll loop), or saying explicitly that the result will NOT be seen without a ping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eturns The going-silent failure is broader than CI: any "I'll report back when X finishes" strands the user, because nothing wakes a finished turn. This adds the missing primitive so waiting is possible instead of forbidden-and-lied- about: a durable one-shot store (single JSON file, atomic rewrite, ids seeded across restarts) plus a 30s firing loop. Delays parse from a filename stem (15m / 2h / 1d), floored to 1m and capped at 48h (longer horizons belong in /schedule), with a per-chat cap of 10 pending items — each is an unattended, model-cost-bearing run. Take-then-fire: a crash drops at most one firing rather than duplicating it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…promises
Two halves of the same fix. The SWEEP runs after every terminal: files the
run wrote into the workspace's followup/ directory (filename = delay, content
= prompt — documented in a convention appended to every workspace CLAUDE.md)
are scheduled into the durable store with a confirmation notice, and fired
later through InjectAuto — budget-gated, AutoUser-attributed, auto-resumed.
The NUDGE is the mechanical safety net for the promise itself: prompt rules
demonstrably fail here (the model rationalizes "the poller will wake me"),
so when a cleanly finished run's FINAL answer matches a conservative set of
first-person return promises (EN + RU: "I'll report back", "доложу",
"отпишусь", "сообщу, как только", …) while having scheduled no follow-up,
ONE corrective run is injected: do the work now in the foreground, schedule a
real followup/<delay>.md, or retract the promise. A marker in the nudge's own
prompt guarantees it can never chain into a nudge loop; asking the USER to
follow up ("напиши, когда…") deliberately does not match. Both mains wire
the loop via the shared internal/autonomy builder; ENABLE_PROMISE_NUDGE=true
by default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The standby rules were CI-specific after the last pass; the failure is not.
The hard rule now names the follow-up convention as the one legal way to end
a turn that must wait ("followup/<delay>.md, tell the user when you'll be
back"), lists the RU phrasings that keep slipping through ("вернусь с
результатом", "доложу, когда закончится") as the bug, and the CI paragraph
points at the same mechanism. .env.example and README document the follow-up
convention and the promise nudge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
flock is DuckBug's own bot, but the team had no way to see the project's live telemetry. The MCP config writer is now generic (WriteMCPConfig takes a named server map; entries carry an optional Authorization: Bearer header and the file stays 0600 since it may hold the token), and both mains assemble it from config: context7 as before (ENABLE_CONTEXT7), plus DuckBug when DUCKBUG_MCP_TOKEN is set — the token is the on/off switch, no separate flag. DUCKBUG_MCP_URL defaults to the cloud endpoint (https://duckbug.io/api/mcp); self-hosted deployments point it at <their-duckbug>/api/mcp. The workspace agreement tells the team when to reach for the tools (debugging production issues, verifying a fix stopped the errors). Claude backend only — codex has its own MCP mechanism. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… role The compose file passes .env wholesale (env_file), but the VK env template and the Ansible role enumerate variables explicitly and had none of the new surface. Both now carry the MCP block (ENABLE_CONTEXT7, DUCKBUG_MCP_TOKEN/URL) and the full autonomy block (post-run verification, /goal evaluator, CI watch + auto-merge, scope auto-approval, the per-day autonomy budget, follow-ups + promise nudge); the Ansible role gets matching defaults and renders the DuckBug token from vault_duckbug_mcp_token. Telegram's .env.example gains the previously undocumented ENABLE_CONTEXT7 alongside the DuckBug block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
govulncheck started failing every CI run on GO-2026-5856 — the Encrypted Client Hello privacy leak in crypto/tls, published for go1.26.4 and fixed in go1.26.5 — regardless of the diff under test. Bump the module toolchain and the dev-tools image; the runtime images are node-based and unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Four opt-in autonomy loops that move the bot up the delegation ladder — from "the agent checks its own work" to "the agent runs without you" — each with a hard stop condition, plus two safety rails and a team-policy update.
ENABLE_POST_VERIFY, default on) — after a cleanly successful run, the bot itself re-runs the changed repos' own check gate (task/make/npm, preferringcheck > ci > test > tests > lint) and, on red, injects a fix-up run with the real failure output — up toPOST_VERIFY_MAX_FIXESconsecutive rounds per chat. Previously failed repos are re-gated even when a "fix" run touched nothing. The agent's "tests pass" is verified, not trusted./goalevaluator — a new reserved command (Telegram + VK): the user states a done-criterion once; after every completed run an independent fresh-session judge (no shared context with the working session) inspects the workspace, re-runs checks, and returns a strict JSON verdict. Not met → the unmet points loop back as a fix-up; met → 🎯 and the goal disarms. Bounded byGOAL_MAX_ATTEMPTS;EVALUATOR_MODELcan pick a cheaper judge.ENABLE_CI_WATCH, default off) — polls CI state on theduck/*branches checked out in the workspaces (GitHub check-runs with a legacy fallback, or Gitea combined status; no inbound webhooks). A red build injects "CI is red — fix and push" into the owning chat, exactly once per commit (durable dedupe).ENABLE_AUTO_MERGEadditionally merges a branch's PR once CI is green — full hands-off mode, off by default.AUTO_TASK_MAX_COST_PER_DAYcaps what autonomy-originated runs (CI/verify/goal fix-ups, via the newService.InjectAuto) may spend per chat per UTC day, with a single daily notice when exhausted. Direct messages and scheduled jobs (already gated by their creator's cost cap) are unaffected.AUTO_APPROVE_SCOPE(off|trivial|standard|all, default off) — lets covered planner complexities skip the "confirm scope & wait" step; unknown values fail safe tooff. The tester subagent now runs onmodel: sonnet(routine role, cheaper model).Notes for review
/goalwas previously forwarded to the provider's own slash-command system; it is now bot-reserved because the bot-level loop persists across turns, meters the budget, and survives restarts. Trade-off called out incore/chat/reserved.go.InjectScheduled) and poller relays are untouched; the newInjectAutois a separate path (blocking submit + auto-resume marker +AutoUsercost attribution).DOCKER_HOSTis unset — bounded by the fix-round cap, output tail shows the cause.README.*.md) are not updated — English only.Test plan
task fmt/task lint— 0 issues;task tests(go test -race ./...) — all packages green..git, both host clients via httptest, dedupe, auto-merge), chat post-run (full goal loop, attempt cap, budget gate, capped verify fix loop with a real red gate), config knobs, workspace placeholder rendering, reserved-set guard tests updated.🤖 Generated with Claude Code