Skip to content

feat: automatic issue triage over free GitHub Models (v2.1.0) - #11

Merged
leonardocandiani merged 1 commit into
mainfrom
feat/v2.1.0-issue-triage
Jun 5, 2026
Merged

leonardocandiani merged 1 commit into
mainfrom
feat/v2.1.0-issue-triage

Conversation

@leonardocandiani

Copy link
Copy Markdown
Owner

What

Automatic, advisory issue triage. When an issue is opened/edited/reopened, a classify job asks GitHub Models (free in Actions over the GITHUB_TOKEN, no secret) for strict JSON — suggested labels, possible duplicate, missing info, severity, summary — and a deterministic apply job acts on it. It never closes, assigns, or merges; a human stays in the merge path.

This is the tournament-champion approach ("Zero-Cost Deterministic Triage") — highest value at zero cost with a blast radius that physically cannot reach code, secrets, or a merge.

Safe by construction

  • Least privilege. The workflow holds issues: write + models: read + contents: read and nothing else. A prompt injection in an issue body can, at worst, suggest a wrong label.
  • Untrusted-data boundary. The issue body is a separate user message wrapped in <issue_body>; the system message declares it DATA to classify, never instructions.
  • Allowlist, never blocklist. Model labels are intersected with the repo's live label set (gh label list). Hallucinated labels are dropped; the workflow never creates labels.
  • Graceful degradation + idempotency. Rate limit / malformed output → needs:human-triage fallback. The advisory comment is marker-keyed and updated in place (no spam on re-trigger).

What's included

  • templates/workflows/issue-triage.yml.template (auto-wired by the existing workflows walker)
  • templates/.github/ISSUE_TEMPLATE/{bug_report,feature_request,config} (+ new apply.ts walker)
  • templates/scripts/seed-labels.sh.template — deterministic, human-run label seeding (auto-wired by the existing scripts walker)
  • templates/rules/09-issue-triage.md.template + CLAUDE.md pointer (passes validate-claude-md-sync)
  • Engine: placeholders.ts (ISSUES_TRIAGE, TRIAGE_MODEL, issues config type), schema issues block
  • Version bump 2.0.2 → 2.1.0; README + CHANGELOG

EMPIRICAL VALIDATION (real environment)

Validated on a fixture (apply.ts --repo-path):

  • Generation — apply produces issue-triage.yml, the 3 ISSUE_TEMPLATE files, seed-labels.sh, rule 09, and the CLAUDE.md pointer. Placeholders substituted: runs-on: ubuntu-latest (runner=github), if: ${{ 'github-models' == 'github-models' }}, --arg model "openai/gpt-4o-mini", ISSUE_TEMPLATE with the real project/maintainer/repo. The off variant generates if: ${{ 'off' == 'github-models' }} (workflow no-ops).
  • Shell soundness — both run: blocks pass bash -n on bash 3.2 (strictest) and shellcheck -S error (zero errors). YAML valid. (Refactored the system-prompt heredoc out of $(...) to dodge bash 3.2's heredoc-in-command-sub bug; fence-stripping uses octal \140 instead of literal backticks.)
  • Functional — fence-stripping → valid JSON; bare JSON parses; allowlist intersect drops a hallucinated admin, keeps canonical casing, dedups → bug,enhancement; malformed JSON → fallback; canon_label case-insensitive returns repo casing.
  • Engine — tsc --noEmit --strict zero type errors; apply idempotent (41 skipped on 2nd run, 0 created/updated); config validates against the schema; local replica of CI's "templates exist" + "no secrets" both pass.

Runtime smoke (a real issue opened on GitHub triggering the live model call) is the maintainer's to run after merge — it needs the deployed workflow + models:read on the repo.

New issues are classified by GitHub Models (free in Actions over the
GITHUB_TOKEN, no secret) and get advisory labels plus a summary comment,
applied deterministically. Triage never closes, assigns, or merges — a human
stays in the merge path.

Safe by construction: the issue-triage workflow holds issues:write +
models:read + contents:read and nothing else, so a prompt injection in an issue
body cannot reach code, a secret, or a merge. The body goes to the model as
untrusted data in a separate user message wrapped in <issue_body>; the model's
label suggestions are intersected with the repo's live label set (gh label
list), so a hallucinated label is dropped — the workflow never creates labels.
Malformed output or no GitHub Models access falls back to needs:human-triage.
The advisory comment is keyed by an HTML marker and updated in place, so
re-triggers never spam the issue.

Ships with issue templates (bug_report, feature_request, config), a
seed-labels.sh run once at setup to create the keepwright-specific labels
(deterministic, kept out of the triage workflow's blast radius), rule
09-issue-triage.md wired into the CLAUDE.md equalization table, and an optional
config block: "issues": { "triage": "off" | "github-models", "model": "..." }.

Engine: placeholders.ts adds ISSUES_TRIAGE + TRIAGE_MODEL and the issues config
type; the schema gains the issues block; apply.ts walks templates/.github/
ISSUE_TEMPLATE. Bump 2.1.0.

Autor: Leonardo Candiani
@leonardocandiani
leonardocandiani merged commit 413b018 into main Jun 5, 2026
1 check passed
@leonardocandiani
leonardocandiani deleted the feat/v2.1.0-issue-triage branch June 5, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant