Durable approval gates, cost caps, approver scoping, signed webhooks - #6
Merged
Conversation
…bhooks Six governance gaps found by mining recurring pains across humanlayer, n8n, dagu, dify, langfuse, litellm and agentgateway, then checked against what draftcat already ships. Common theme: the gate must hold when things go wrong. 1. Durable approval gates (pending_approvals + reconcileInterruptedApprovals) A gate lived only in an in-process poll loop: the draft went out, the engine blocked on a ticker, and action_approvals was written only once a decision arrived. A redeploy or crash mid-approval left NO trace at all — the operator saw a live-looking message attached to a run that no longer existed, and the compliance queries could not see the hole either. Now a row is written before the draft is sent, resolved on any terminal decision, and reconciled to `interrupted` on the next boot: an audit row so UnapprovedActions counts it, plus a message telling the operator the action did NOT run. Resuming the run is not attempted — the in-memory data context is gone after a crash — so an unknown outcome is never treated as approval. 2. Cost caps in money (budgets.per_day_cost / per_pipeline_cost) Per-call cost was already computed from the model rates and thrown away. It now accumulates and blocks. The check lives inside check(), which already guards every LLM call, so a call site added later cannot spend uncapped by forgetting to wire it. Enforced between calls: a pre-call estimate needs the response token count, which does not exist yet, so one in-flight call may overshoot by at most a step. 0 = off, existing configs unaffected. 3. Per-operator approver scoping (steps[].approvers) allowed_users was one flat list and quorum only a count, so anyone who could approve an inbox draft could also release an invoice. approvers intersects with allowed_users and never widens it, so a step cannot grant rights to someone the channel does not already trust. 4. channel: slack validated OK but was never implemented The validator's allow-list carried "slack" while no Slack channel existed and step.Channel was never read at runtime, so approvals silently went to Telegram. A gate routing somewhere the operator is not watching is worse than no gate, because it still looks like it held. internal/channels is now the single list both the validator and the engine read; an unimplemented channel is a hard error, and the engine rejects a step naming a channel it is not running. README's "Telegram / Slack" claim corrected too. 5. Body-bound webhook signatures (webhook.require_signature) A bearer token proves only that the caller once saw it: the body is not bound and a captured header replays forever — and this endpoint STARTS pipelines. Adds X-Draftcat-Signature: t=<unix>,v1=<hmac-sha256(t.body)>, a skew window, and single-use signatures via the existing dedup table. Verified whenever present so a broken signer fails loudly, demanded when require_signature is on. Bearer auth is unchanged and still required. 6. Config validation on the boot path (validate.CheckAtStartup) The validator was thorough but opt-in, so a bad config booted fine and failed hours later mid-pipeline. Errors are now fatal at startup, warnings logged; DRAFTCAT_SKIP_VALIDATE=1 overrides. Shares one check body with `draftcat validate` so the two cannot drift. 44 new tests. The three security-critical fixes were mutation-checked: re-adding phantom slack, letting approvers widen past allowed_users, and removing the replay guard each fail their test. Verified end-to-end against the binary — a bad config refuses to boot, the escape hatch overrides, and a simulated crashed gate reconciles to interrupted with the audit row written. Existing behavior is unchanged where nothing is configured: no cost keys, no approvers, and no signature header all mean today's semantics.
- Governance: durable approval gates, cost budgets, approver scoping, boot-path config validation. - Webhook section: signed-request format, skew window, single-use signatures. - Configuration: per_day_cost / per_pipeline_cost and steps[].approvers, with the between-calls enforcement caveat spelled out. - Correct two stale claims: the approval step never supported Slack, and pre-push runs `draftcat validate`, not `--strict` (see .lefthook.yml). - Commands: note that the engine validates before starting; add audit-verify.
Adds the "New in vX.Y.Z" highlights block above the demo, carrying v0.4.0 and v0.3.1 — the two most recent releases. The README top is the shop window; a release whose highlights never land there is invisible to visitors. v0.3.1 is v0.3.0 plus a state-init fix, so its line carries the v0.3 substance (quorum, signed receipts, exporters) and says so rather than advertising a lint fix or misattributing v0.3.0's features. Also refreshes Status, which was three releases stale: - claimed v0.2.2 - listed the OTLP/Prometheus exporter as "planned" — shipped in v0.3.0 - listed "Slack approval" as planned, which this branch made a hard error; replaced with an explicit note that Telegram is the only implemented channel - adds resuming an interrupted pipeline at its approval step as real future work, since v0.4.0 only records the interruption
Triaged the five open issues (all from 2026-03-22, when this was FixClaw and Slack-first). Three are obsolete or already shipped; two had real work left. #2 config validation on startup — the boot-path gate landed earlier on this branch; these are the checks the issue asked for that were still missing: - a pipeline with no steps is now an error. It schedules, runs, and does nothing, which reads as success in the logs. - typo'd step types and action names get a did-you-mean hint, which the issue asked for by name ("unknown type 'ia' — did you mean 'ai'?"). Uses Damerau-Levenshtein, not plain Levenshtein: transposition has to count as one edit or "ia" -> "ai" scores 2 and falls outside the cutoff on a short word, losing exactly the suggestion most worth making. The cutoff is tight on purpose — a confidently wrong suggestion is worse than none. #4 pipeline execution logs — adds `draftcat runs [pipeline] [--json]`. Runs and approval decisions have been recorded in SQLite for a while but nothing could read them back without an sqlite3 client; an audit trail you cannot query is half an audit trail, which is what the issue was really about ("easy to review, search, and archive"). The issue asked for a JSON file per run under logs/. That would duplicate state.db into a second unindexed copy with its own retention problem, so --json prints the same archivable output on stdout instead — pipeable to a file, jq, or a log shipper. Approvals are joined onto their run by timestamp window; the scheduler refuses to start a pipeline already running, so runs of one pipeline never overlap and the window is unambiguous. Per-step timings and token counts stay in the observability spans, which is their right home. This is the governance record: what ran, who decided what. Not implemented, with reasons: #1 asks for Telegram "alongside Slack" and is inverted — Telegram is the only channel and Slack never existed. #3's dry-run ships as `draftcat test`. #5 (Teams) needs Adaptive Cards plus a bot for interactive approvals; a real project, not a config flag.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six governance gaps, found by mining recurring pains across humanlayer, n8n, dagu, dify, langfuse, litellm and agentgateway, then checked against what draftcat already ships. Common theme: the gate must hold when things go wrong — restart, misconfig, replay, overspend, wrong approver, wrong channel.
Where nothing is configured, behavior is unchanged: no cost keys, no
approvers, and no signature header all mean today's semantics.What's in here
1. Durable approval gates —
pending_approvals+reconcileInterruptedApprovalsA gate lived only in an in-process poll loop: the draft went out, the engine blocked on a ticker, and
action_approvalswas written only once a decision arrived. A redeploy or crash mid-approval left no trace at all — the operator saw a live-looking message with buttons attached to a run that no longer existed, andUnapprovedActionscouldn't see the hole either.A row is now written before the draft is sent, resolved on any terminal decision, and reconciled to
interruptedon the next boot: an audit row so the compliance queries count it, plus a message telling the operator the action did not run.Deliberately not resuming the run — the in-memory data context is gone after a crash. A gate whose outcome is unknown is never treated as an approval.
2. Cost caps in money —
budgets.per_day_cost/per_pipeline_costPer-call cost was already computed from the model rates and thrown away. It now accumulates and blocks. The check sits inside
check(), which already guards every LLM call, so a call site added later can't spend uncapped by forgetting to wire it.Enforced between calls — a pre-call estimate needs the response token count, which doesn't exist yet — so one in-flight call may overshoot by at most a step. Documented; bound it with
per_step_tokens.0= off.3. Per-operator approver scoping —
steps[].approversallowed_userswas one flat list and quorum only a count, so whoever could approve an inbox draft could also release an invoice. Quorum says how many; this says which ones. It intersects withallowed_usersand never widens, so a step can't grant rights to someone the channel doesn't already trust.4.
channel: slackvalidated OK but was never implementedThe validator's allow-list carried
slackwhile no Slack channel existed andstep.Channelwas never read at runtime — so approvals silently went to Telegram. A gate routing somewhere the operator isn't watching is worse than no gate, because it still looks like it held.internal/channelsis now the single list both the validator and the engine read. Unimplemented channel = hard error; the engine also rejects a step naming a channel it isn't running. The honest fix was deleting the phantom entry, not building Slack. The README's "Telegram / Slack" claim was wrong too, and is corrected.5. Body-bound webhook signatures —
webhook.require_signatureA bearer token proves only that the caller once saw it: the body isn't bound and a captured header replays forever — and this endpoint starts pipelines. Adds
X-Draftcat-Signature: t=<unix>,v1=<hmac-sha256(t + "." + body)>, a skew window, and single-use signatures via the existing dedup table.Verified whenever present (so a broken signer fails loudly), demanded when
require_signature: true. Bearer auth is unchanged and still required. The existing constant-time compare and body size limit were already good.6. Config validation on the boot path —
validate.CheckAtStartupThe validator was thorough but opt-in, so a bad config booted fine and failed hours later mid-pipeline. Errors are now fatal at startup, warnings logged.
DRAFTCAT_SKIP_VALIDATE=1overrides. Shares one check body withdraftcat validateso the two can't drift.Verification
44 new tests. Build,
go vet,gofmt, full suite,go run . validate, andgolangci-lint --new-from-revall pass; no new lint issues against the 139-issue baseline. Both commits build and test independently.The three security-critical fixes were mutation-checked — each was deliberately broken to confirm the test actually bites:
slackto the registryapproverswiden pastallowed_usersEnd-to-end against the built binary: a bad config refuses to boot (exit 1, names the offending step), the escape hatch overrides,
draftcat testis unchanged, and a simulated crashed gate reconciled correctly —pending→0,interrupted→1, audit row written, andUnapprovedActionsnow sees it.Review notes
main.gocarries four of the six changes, which is why this is one feature commit rather than six — splitting it would have produced non-building intermediates.internal/channelsexists specifically so the validator and engine can't drift again; adding a name there without an implementation reintroduces the bug.