Skip to content

perf(security): decide and render triage and invalidate as code - #1686

Merged
potiuk merged 2 commits into
mainfrom
perf/security-2-triage
Oct 18, 2026
Merged

potiuk merged 2 commits into
mainfrom
perf/security-2-triage

Conversation

@potiuk

@potiuk potiuk commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Layer 2 of the security-family deep split (#1672), stacked on #1680. security-issue-triage and security-issue-invalidate move their deterministic rules into tools/security-management (two new CLI families over the shared library); classification, reasoning, Security-Model quotes and the choice of canned response stay with the model. Tracker bodies are read only through github-body-field, the one fail-closed parser from #1680.

skill measured_tokens main → branch lines
security-issue-triage 7,854 → 6,322 543 → 405
security-issue-invalidate 7,402 → 6,147 528 → 417
  • Triage — selector and range parsing, proposal-comment rendering, mention and link-form checks, the confirmation grammar, the recap. New shared modules: fields (fail-closed body reads), threads (primary-thread choice by relay signals), dispositions, selectors.
  • Invalidate — consensus check, closing-comment and rollup rendering, board-archive commands, the reporter-reply scrub, draft-backend choice.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Python package (tools/*/ with pyproject.toml)

Test plan

RFC-AI-0004 compliance

  • HITL — every post, close and draft stays a proposal applied on confirmation
  • Write-access discipline — the package renders commands and runs none

Linked issues

Part of #1672. Depends on #1680.

Notes for reviewers (optional)

Rebased onto #1680's new head (itself rebased onto main with #1683); the token stamps moved with main's longer tracker-backend block.

Behaviour changes:

  • triage: a reply that only skips, edits or re-classifies is ambiguous and posts nothing; a proposal carrying a CVSS score, more than three handles or an off-roster handle is refused.
  • invalidate: consensus needs a roster member's INVALID proposal acknowledged by another roster member; an ambiguous body field stops the close; the reporter reply is checked for tracker references, the CVE-tool URL, CVSS and other ASF project names before drafting.
  • security-issue-triage is about 470 tokens over the ~5,850 target; the generated pre-flight and tracker-backend blocks are a floor.
  • The triage commit also reformats issue_import_from_md/render.py.

Generative AI disclosure: prepared with Claude Code (Claude Opus 5); reviewed by the author.

🤖 Generated with Claude Code

@github-actions github-actions Bot added family:security security-* skills family:tools tools/* family:docs Docs, MISSION.md, READMEs capability:triage Sweep + classify + propose disposition capability:resolve Close-out: invalidate, dedupe, CVE allocate, announcement substrate:framework-dev Tool substrate: build / validate / eval the framework itself labels Oct 17, 2026
@potiuk
potiuk added this pull request to stack #1689 October 17, 2026 15:56
@potiuk
potiuk force-pushed the perf/security-2-triage branch from 8a30f54 to bc97734 Compare October 17, 2026 21:09
@potiuk
potiuk force-pushed the perf/security-2-triage branch from bc97734 to 6c5027c Compare October 17, 2026 21:35
Base automatically changed from perf/security-1-import to main October 18, 2026 12:59
A new issue-triage family in tools/security-management takes the rules of
security-issue-triage that are pure functions of tracker state: the selector
and its gh query (CVE token, scope label and bare --retriage refused), the
Step 1 stop cases and the 20-issue GraphQL batches, the four-rule pre-flight
skip (the mail rule in a second pass), the Step 2.6 precedent signal, the
@-mention priorities and cap, the proposal layout with its class additions,
the bare-name scrub, the Step 5 reply grammar and the Step 7 recap. The
shared library gains readers for existing trackers: fields (through
github-body-field's normalise and extract_field, fail closed), threads (the
mailing-thread field as an import path, primary thread by relay signals),
dispositions (the six classes and the proposal markers) and selectors.
Classification, the Security Model quote, rating precedents, choosing routing
candidates and flagging injection stay with the model.

SKILL.md becomes a router (7,130 -> 6,050 tokens, 534 -> 396 lines); every
heading and golden-rule headline is unchanged, and the failure-modes table
moved byte-for-byte to outcomes/failure-modes.md. The step-1-selector,
step-5-confirm and step-7-recap suites and cases 1-2 of step-6-body-scrub
become pytest cases named after each former case; step-26 keeps its model
suite and gains pinned signal tests.

Behaviour changes: an undeclared scope label and a range wider than 1000
issues are refused; a reply that only skips, edits or re-classifies, or that
lists and skips the same item, is ambiguous and posts nothing; edited and
re-classified items are re-drafted before they post (post_now); a proposal
with a CVSS severity, more than 3 handles, a handle off the roster or an
INFO-ONLY heading missing from canned-responses.md is refused; an UNCERTAIN
proposal opens with the low-confidence line and keeps a tentative class so
the pre-flight skip can read it. The issue-import-from-md renderer is
reformatted (ruff format --check failed on it).

Part of #1672

Generated-by: Claude Opus 5
…as code

A new issue-invalidate family in tools/security-management takes the rules
of security-issue-invalidate that are pure functions of tracker state: the
selector (single, list or range, `invalidate proposed` with its 1000 cap and
batched read), the team-consensus check, the Step 0 hard stops, the Step 2
import path and reply thread (read through github-body-field, relay threads
excluded), the label change, the board queries, the Step 5d draft plan and a
grep of the composed reply, the closing comment, the rollup entry, the 5f
confirmation forms, the 6a-6g apply order and the recap. Mining the reasoning,
choosing the canned section and writing the reply stay with the model.

SKILL.md becomes a router (7,582 -> 5,875 tokens, 519 -> 408 lines); every
heading and golden-rule headline is unchanged, and the hard-stop table, the
import-path table, the closing-comment and rollup-entry shapes and the
failure-modes table moved byte-for-byte to outcomes/ (relative links
re-rooted). The step-2, 5a, 5b, 5d, 5e, 5f and 7 suites become pytest cases
named after each former case; step-3 and step-4 stay model evals.

Behaviour changes: only an INVALID proposal by a roster member counts for
`invalidate proposed`, and its thumbs-up or acknowledgement must come from
another roster member; a field github-body-field will not decide on stops the
close instead of reading as empty; the closing comment says where the draft
is for Maildir and forwarder drafts too; the rollup entry is refused without
a Reporter-notification case, and a duplicate's line names the canonical CVE;
the drafting backend falls back to claude_ai_mcp only without OAuth
credentials and with no link in the body, and refuses otherwise; `email:` is
ambiguous when no draft is proposed; a composed reply is checked for tracker
references, the CVE-tool URL, CVSS, re-opening phrases and other ASF project
names before it is drafted.

Part of #1672

Generated-by: Claude Opus 5
@potiuk
potiuk force-pushed the perf/security-2-triage branch from 6c5027c to 52530e0 Compare October 18, 2026 12:59
@github-actions github-actions Bot added the substrate:analytics Tool substrate: read-only metrics / dashboards / renderers label Oct 18, 2026
@potiuk
potiuk merged commit 9955d7d into main Oct 18, 2026
11 checks passed
@potiuk
potiuk deleted the perf/security-2-triage branch October 18, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:resolve Close-out: invalidate, dedupe, CVE allocate, announcement capability:triage Sweep + classify + propose disposition family:docs Docs, MISSION.md, READMEs family:security security-* skills family:tools tools/* substrate:analytics Tool substrate: read-only metrics / dashboards / renderers substrate:framework-dev Tool substrate: build / validate / eval the framework itself

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant