From 72e478a1ec9637c1361532a645b0fe08111669dd Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 19 Jul 2026 23:53:22 +0000 Subject: [PATCH 1/3] =?UTF-8?q?feat(docs):=20forge=20docs=20impact=20?= =?UTF-8?q?=E2=80=94=20reusable=20documentation-impact=20graph?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Answers the question: "I changed X — which documented surfaces mention X and are now stale?" Implemented as a three-stage pipeline: 1. A typed-entity extractor registry pulls named surfaces (commands, flags, env vars, config keys, etc.) out of the source tree. 2. Those entities are compiled into an inverted doc index mapping each entity to the documentation locations that mention it. 3. A diff-to-impact query takes a set of changed files/entities and looks up the doc index to report which documented surfaces are now potentially stale. The command is advisory by default (prints findings, exits 0) and gains a --strict flag for CI enforcement (nonzero exit on stale docs). 'forge docs check' now prints a one-line pointer to 'forge docs impact' so users discover the deeper analysis. Adds src/docs_impact.js (extractor registry, inverted index, diff query) with test/docs_impact.test.js, wires the new command into src/cli.js and src/commands.js, and updates README.md, docs/GUIDE.md, and CHANGELOG.md. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 18 ++ README.md | 2 +- docs/GUIDE.md | 41 +++ src/cli.js | 26 ++ src/commands.js | 30 +- src/docs_check.js | 4 +- src/docs_impact.js | 600 +++++++++++++++++++++++++++++++++++++++ test/docs_impact.test.js | 257 +++++++++++++++++ 8 files changed, 975 insertions(+), 3 deletions(-) create mode 100644 src/docs_impact.js create mode 100644 test/docs_impact.test.js diff --git a/CHANGELOG.md b/CHANGELOG.md index 4277082..31d8ce9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,24 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +### Added + +- **`forge docs impact` — a reusable documentation-impact graph.** Where `docs check` + reconciles a fixed list of registries and `docs sync` scans a diff for raw identifiers, + `docs impact` answers the general question the project kept forgetting: _"I changed X — + which documented surfaces mention X and are now potentially stale?"_ It works in three + data-driven stages: (1) a pluggable extractor registry derives the **typed** entities + the project documents (command names, CLI flags, `FORGE_*` env vars, MCP tool names, + exported symbols, brand tokens, the version, `package.json` fields) from their canonical + sources — reusing `docs_check`'s `envVarsRead`/`srcFiles` and the `COMMANDS`/`TOOLS` + registries; (2) a word-boundary- and code-fence-aware scan of every doc surface (all + tracked `*.md`, `CITATION.cff`, the plugin manifests, the landing page, `package.json`) + builds an inverted index entity → `file:line`; (3) an impact query maps the entities a + git diff changed to every doc location that references them, ranked by confidence. + Advisory by default (`--strict` exits non-zero for CI); `--since `, `--staged`, + `--min-confidence `, and `--json` flags; `docs check` prints a one-line advisory + pointing here when the working tree touched a documented entity. New `src/docs_impact.js`. + ## [0.23.1] - 2026-07-19 ### Fixed diff --git a/README.md b/README.md index c9fe2b3..5bfdcda 100644 --- a/README.md +++ b/README.md @@ -217,7 +217,7 @@ that never clobbers your existing settings (skip it with `install.sh --no-settin | | `forge tools` | primary-tool config — gitignore secondary-tool artifacts (`.cursor/`, `.gemini/`, …) for tools this repo doesn't use; `forge tools ` sets it, `--reset` clears | | | `forge doctor` | pass/fail health check: tools, guards, MCP, drift, update | | | `forge update` | self-update — `--check` reports if a newer version exists, bare applies it, `--to ` pins/downgrades | -| | `forge docs` | docs↔code drift — `check` reconciles commands/env/MCP/CHANGELOG; `sync` sweeps the diff for stale doc mentions | +| | `forge docs` | docs↔code drift — `check` reconciles commands/env/MCP/CHANGELOG; `sync` sweeps the diff for stale doc mentions; `impact` maps which docs reference the entities THIS diff changed | | | `forge config` | provider setup — show / switch / add providers, set the default model | | | `forge integrations` | opt-in third-party MCP servers (e.g. context7) — `add` records the managed set and writes only with `--yes` (`--adopt` claims a same-name entry you already had); `remove` reverses it | | | `forge harden` | wire the pre-commit gate (gitleaks + commit gate) + sandbox settings | diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 7ce123f..e08021b 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -410,6 +410,46 @@ touched in the same diff still answers for **removed** symbols (updated-for-one- isn't updated-for-the-rename); and `.forge/state.md` is never scanned — handoff writes the changed-file list into it by design, so the gate's mtime check covers it instead. +### `forge docs impact` — change X, which docs mention X? + +`docs sync` sweeps a diff for raw identifiers; `docs impact` is the general, reusable +version of the same idea, built as a **documentation-reference graph** rather than a +per-file rule. It works in three data-driven stages: + +1. **Entity extraction.** A pluggable extractor registry derives the _typed_ set of + things the project documents — command names, CLI flags, `FORGE_*` env vars, MCP tool + names, exported symbols, brand tokens, the version, and `package.json` fields — from + their canonical sources (the same `COMMANDS`/`TOOLS` registries and `envVarsRead` + scanner `docs check` already uses). Adding a new entity type is one record in the + registry, not a new special case. +2. **Reference index.** A word-boundary- and code-fence-aware scan of every doc surface + (all tracked `*.md`, `CITATION.cff`, the plugin manifests, the landing page, + `package.json`) builds an inverted index: entity → every `file:line` that names it. + English-word-shaped entities (bare symbols like `build`) only count inside backticks + or fenced blocks, so prose doesn't false-positive. +3. **Impact query.** From a git diff of the canonical sources it computes which entities + _changed_ (a command renamed, a flag added, an env var removed, an export touched, the + version bumped) and returns every doc location that references them, ranked by + confidence. + +```console +$ forge docs impact --since main +docs impact — diff vs main: 2 changed entities, 1 doc file(s) potentially stale + env `APP_CACHE_MODE` (REMOVED) — 2 reference(s): + README.md:88 (code, 0.95) [Environment] Set `APP_CACHE_MODE` to tune the cache. + version `0.23.0` (REMOVED) — 1 reference(s): + README.md:3 (prose, 0.75) install @scope/pkg@0.23.0 + → review each surface: update it, or confirm the mention is still correct. +``` + +Advisory by default (matching the fail-open ethos) — `--strict` exits non-zero when any +doc surface is impacted, for CI. `--since ` sets the diff base (default: the session +baseline, then `HEAD`), `--staged` reads the index, `--min-confidence ` drops low-signal +hits, `--json` for tooling. `docs check` also prints a one-line advisory pointing here when +the working tree touched a documented entity. **Honest limits:** a token index catches a +doc that _names_ a changed entity; it cannot see a paraphrase that never uses the name, a +screenshot or diagram, or a design/wording choice with no textual anchor. + ### `forge verify` — did it actually work? The independent check: runs the real test suite and flags edited symbols that aren't in @@ -997,6 +1037,7 @@ Plain `forge cost` remains the per-day spend view via `ccusage`. | `forge doctor` | Health check: layers, install, drift, cortex. `forge doctor --fix` auto-repairs the safely fixable findings (missing settings hooks/permissions, ledger union-merge rule, stale `AGENTS.md`, non-executable guards) by reusing the same idempotent functions `forge init`/`forge sync` use, then re-checks. | | `forge docs check` | Docs↔code drift: commands, env vars, MCP tools, CHANGELOG reconciled against the code (CI-gated on the forge repo itself). | | `forge docs sync` | Diff-driven stale-docs sweep: UPDATED / STALE (file:line hits) / VERIFIED-UNAFFECTED per artifact (see the full section above). | +| `forge docs impact` | Documentation-reference graph: extract typed entities (commands/flags/env/MCP/symbols/version), index every doc surface, and report which docs reference the entities THIS diff changed — ranked by confidence (see the full section above). | | `forge catalog` | Start-Here index of every tool / crew / guard. | | `forge integrations` | Opt-in third-party MCP servers (e.g. `context7`, no longer installed by default). `integrations add ` prints the package, its network behaviour, and the files it touches, then writes only with `--yes`; the install is recorded in `.forge/forge.config.json` (`mcp.integrations`), so every later sync keeps emitting it. A same-name server you configured yourself is never overwritten unless you pass `--adopt` (recorded under `mcp.adopted`). `integrations remove ` reverses the add — it deletes only forge-owned entries/blocks/files and is a no-op when nothing is installed. | | `forge brain` / `forge remember` | Portable project memory inlined into `AGENTS.md`. | diff --git a/src/cli.js b/src/cli.js index 4754714..a69cc54 100755 --- a/src/cli.js +++ b/src/cli.js @@ -472,6 +472,22 @@ async function run(argv) { if (r.error || (argv.includes("--strict") && r.stale.length)) process.exitCode = 1; return; } + // `impact` — reusable documentation-impact graph: which documented surfaces + // reference the entities THIS diff changed (advisory; --strict exits 1). + if (sub === "impact") { + const { docsImpact, renderDocsImpact } = await import("./docs_impact.js"); + const { newestBaseline } = await import("./docs_sync.js"); + const staged = argv.includes("--staged"); + const sinceIdx = argv.indexOf("--since"); + const base = sinceIdx >= 0 ? argv[sinceIdx + 1] : newestBaseline(process.cwd()) || undefined; + const mcIdx = argv.indexOf("--min-confidence"); + const minConfidence = mcIdx >= 0 ? Number(argv[mcIdx + 1]) : 0; + const r = docsImpact(process.cwd(), { base, staged, minConfidence }); + if (json) console.log(JSON.stringify(r, null, 2)); + else console.log(renderDocsImpact(r)); + if (argv.includes("--strict") && r.impacted.length) process.exitCode = 1; + return; + } // `check` — self-check of the forge package's own docs against its code (commands // table, env reads, MCP registry, CHANGELOG). const { docsCheck } = await import("./docs_check.js"); @@ -489,6 +505,16 @@ async function run(argv) { console.log(`\n${r.issues.filter((i) => i.severity === "error").length} problem(s)`); process.exitCode = 1; } + // Advisory: if the working tree changed a documented entity, point at the impacted + // prose. Never fails the check (fail-open); best-effort, so a git-less tree is silent. + try { + const { docsImpact } = await import("./docs_impact.js"); + const imp = docsImpact(process.cwd()); + if (imp.impacted.length) + console.log( + `\n ! ${imp.summary.impactedFiles} doc file(s) may be stale from this diff — run \`${BRAND.cli} docs impact\` to review`, + ); + } catch {} return; } if (cmd === "integrations") { diff --git a/src/commands.js b/src/commands.js index 859b66f..f482128 100644 --- a/src/commands.js +++ b/src/commands.js @@ -109,7 +109,35 @@ export const COMMANDS = { dash: "live dashboard: ledger, metrics trends, radar, memory browser, timeline, blast radius", report: "emit a static, self-contained HTML snapshot of .forge/ — opens offline, no server", brand: "print the active brand token map", - docs: "docs↔code drift — check (registry reconcile) / sync (diff-driven stale-docs sweep)", + docs: { + summary: + "docs↔code drift — check (registry reconcile) / sync (diff-driven stale-docs sweep) / impact (reusable doc-reference graph: which docs mention what THIS diff changed)", + usage: "forge docs [check | sync | impact] [--since | --staged] [--strict] [--json]", + flags: [ + { + flag: "--since ", + desc: "impact: diff against this git ref (default: session baseline, then HEAD)", + }, + { + flag: "--staged", + desc: "impact/sync: diff the staged index instead of the working tree", + }, + { + flag: "--min-confidence ", + desc: "impact: drop reference hits below this confidence (0..1)", + }, + { + flag: "--strict", + desc: "exit non-zero when stale/impacted docs are found (for CI; advisory otherwise)", + }, + ], + examples: [ + "forge docs check", + "forge docs sync", + "forge docs impact", + "forge docs impact --since main", + ], + }, integrations: "opt-in third-party MCP servers (e.g. context7) — add records the managed set and writes only with --yes (--adopt claims a same-name entry); remove reverses it", }; diff --git a/src/docs_check.js b/src/docs_check.js index ef80e5b..01519d1 100644 --- a/src/docs_check.js +++ b/src/docs_check.js @@ -37,7 +37,9 @@ function readDoc(root, rel) { return existsSync(p) ? readFileSync(p, "utf8") : ""; } -function srcFiles(root) { +// Exported so docs_impact.js reuses the SAME src-file inventory (recursive, .js only) +// the env-var extractor scans — one definition of "what counts as a source file". +export function srcFiles(root) { const dir = join(root, "src"); if (!existsSync(dir)) return []; // recursive: src/emit/*.js reads are part of the same env contract. diff --git a/src/docs_impact.js b/src/docs_impact.js new file mode 100644 index 0000000..a71b3b0 --- /dev/null +++ b/src/docs_impact.js @@ -0,0 +1,600 @@ +// forge docs impact — a REUSABLE documentation-impact graph. Where `docs check` +// reconciles a fixed list of registries and `docs sync` scans a git diff for raw +// identifiers, this module answers the general question the project keeps forgetting: +// "I changed X — which documented surfaces mention X and are now potentially stale?" +// +// It works in three data-driven stages, none of them hardcoded per file: +// +// 1. ENTITY EXTRACTION. A data-driven EXTRACTORS registry derives the TYPED set of +// entities the project documents (commands, flags, env vars, MCP tools, exported +// symbols, brand tokens, the version, package.json fields) from their CANONICAL +// sources. Adding a new entity type = pushing one entry onto EXTRACTORS, never +// editing N call sites. The heavy extractors are REUSED from docs_check.js +// (envVarsRead, srcFiles) and the existing registries (COMMANDS, TOOLS, BRAND). +// +// 2. REFERENCE INDEX. A generic token scan over every prose/doc SURFACE (all tracked +// *.md, CITATION.cff, the plugin manifests, the landing page, package.json's +// description/keywords) builds an inverted index entity → [{file,line,section, +// context,confidence}]. The scan is word-boundary aware and code-fence aware: +// English-word-shaped entities (bare symbols) are only counted inside code spans, +// exactly like docs_sync's "soft" identifiers, so prose doesn't false-positive. +// +// 3. IMPACT QUERY. Given the entities a git diff CHANGED (a command renamed/removed, +// a flag added, an env var deleted, an export touched, the version bumped), it +// returns every indexed doc location that references them, ranked by confidence. +// This is the reusable core: change → index lookup → the exact stale-risk surfaces. +// +// Honest about limits (see the CLI help + README): a token index catches a doc that +// NAMES a changed entity. It cannot catch a paraphrase that never uses the name, a +// screenshot/diagram/image, or a design/wording choice with no textual anchor. It is +// advisory by construction — a high-recall "look here", not a proof of staleness. + +import { execFileSync } from "node:child_process"; +import { existsSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { BRAND } from "./brand.js"; +import { COMMANDS, HIDDEN_COMMANDS } from "./commands.js"; +import { envVarsRead, srcFiles } from "./docs_check.js"; +import { TOOLS } from "./mcp_tools.js"; + +const git = (root, args) => { + try { + return execFileSync("git", args, { + cwd: root, + encoding: "utf8", + stdio: ["ignore", "pipe", "ignore"], + }); + } catch { + return ""; + } +}; + +const esc = (s) => String(s).replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + +/** A tracked file is a canonical SOURCE (never scanned as a doc — that would flag every + * registry as a reference to itself). */ +const isSrcJs = (rel) => rel.startsWith("src/") && rel.endsWith(".js"); + +// History files are append-only and correct-by-design: their mentions of a changed +// entity (a renamed command in an old changelog line) are NOT staleness. Same exemption +// docs_sync applies. Kept out of both the reference index and the impacted output. +const HISTORY_RE = /(^|\/)(CHANGELOG\.md|\.forge\/decisions\.md)$/i; + +// --------------------------------------------------------------------------- +// Stage 1 — the data-driven extractor registry. +// --------------------------------------------------------------------------- +// Each extractor is a plain record: +// type stable entity-type label (also the confidence bucket) +// weight base confidence a plain-prose reference of this type earns (0..1) +// codeOnly entity names shaped like English words → only count inside code spans +// extract() the canonical current entities: {name, meta?}[] +// ref(names) build the doc-scan RegExp (group 1 = matched entity name) +// isSource (rel) → is this file a canonical source that DEFINES this type? +// discover (text) → entity names found in arbitrary source text (used on a diff's +// REMOVED lines to detect entities that were DELETED, and on package.json +// to recover the OLD version). Optional. +// +// Adding "config keys" or any new type is one more record here — nothing else changes. + +const boundaryRe = (names) => + new RegExp(`(?:^|[^\\w$/.-])(${names.map(esc).join("|")})(?![\\w$])`, "g"); + +/** @type {Array<{type:string,weight:number,codeOnly?:boolean,extract:(root:string)=>{name:string,meta?:object}[],ref:(names:string[])=>RegExp,isSource:(rel:string)=>boolean,discover?:(text:string)=>string[]}>} */ +export const EXTRACTORS = [ + { + // Command names — reference form is ` ` (BRAND.cli, never a literal), + // which keeps a common word like "docs" from matching bare prose. + type: "command", + weight: 0.9, + extract: () => [ + ...Object.keys(COMMANDS).map((name) => ({ name })), + ...HIDDEN_COMMANDS.map((name) => ({ name, meta: { hidden: true } })), + ], + ref: (names) => + new RegExp(`\\b${esc(BRAND.cli)}\\s+(${names.map(esc).join("|")})(?![\\w-])`, "g"), + isSource: (rel) => rel === "src/commands.js", + // Table keys in commands.js: ` init: {` / ` taste: "…"`. + discover: (text) => [...text.matchAll(/^\s{2}([a-z][a-z-]+):/gm)].map((m) => m[1]), + }, + { + // CLI flags, canonically declared in the COMMANDS table's `flags`/`usage`. + type: "flag", + weight: 0.8, + extract: () => { + const flags = new Set(); + for (const entry of Object.values(COMMANDS)) { + if (typeof entry !== "object") continue; + for (const f of entry.flags ?? []) + for (const m of String(f.flag).matchAll(/--[a-z][\w-]*/g)) flags.add(m[0]); + for (const m of String(entry.usage ?? "").matchAll(/--[a-z][\w-]*/g)) flags.add(m[0]); + } + return [...flags].map((name) => ({ name })); + }, + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "src/commands.js", + discover: (text) => [...text.matchAll(/--[a-z][\w-]*/g)].map((m) => m[0]), + }, + { + // Env vars — REUSES docs_check.envVarsRead (the one scanner of process.env + guards). + type: "env", + weight: 0.9, + extract: (root) => [...envVarsRead(root)].map((name) => ({ name })), + ref: (names) => boundaryRe(names), + isSource: (rel) => isSrcJs(rel) || /^global\/guards\/.*\.sh$/.test(rel), + discover: (text) => [ + ...[...text.matchAll(/process\.env\.([A-Z_][A-Z0-9_]*)/g)].map((m) => m[1]), + ...[ + ...text.matchAll( + /\b((?:FORGE|ANTHROPIC|LITELLM|OPENROUTER|ENABLE_CORTEX|CLAUDE)_[A-Z0-9_]+)\b/g, + ), + ].map((m) => m[1]), + ], + }, + { + // MCP tool names — from the same TOOLS registry docs_check reconciles. + type: "mcp-tool", + weight: 0.9, + extract: () => TOOLS.map((t) => ({ name: t.name })), + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "src/mcp_tools.js", + discover: (text) => [...text.matchAll(/name:\s*["']([a-z][a-z0-9_]+)["']/gi)].map((m) => m[1]), + }, + { + // Exported public symbols — the JS API surface. Word-shaped (build, load, has), so + // codeOnly: a symbol counts only inside a code span, never in running prose. + type: "symbol", + weight: 0.6, + codeOnly: true, + extract: (root) => { + const names = new Set(); + for (const file of srcFiles(root)) { + let text = ""; + try { + text = readFileSync(file, "utf8"); + } catch { + continue; + } + for (const m of text.matchAll( + /export\s+(?:async\s+)?(?:function|const|let|class)\s+([A-Za-z_$][\w$]*)/g, + )) + names.add(m[1]); + } + return [...names].map((name) => ({ name })); + }, + ref: (names) => boundaryRe(names), + isSource: isSrcJs, + discover: (text) => + [ + ...text.matchAll( + /export\s+(?:async\s+)?(?:function|const|let|class)\s+([A-Za-z_$][\w$]*)/g, + ), + ].map((m) => m[1]), + }, + { + // Brand tokens — from src/brand.js. Ubiquitous, so low confidence, but a rebrand + // genuinely touches every surface, which is exactly what this should surface. + type: "brand", + weight: 0.35, + extract: () => + [...new Set([BRAND.brand, BRAND.cli, BRAND.pkg, BRAND.home].filter(Boolean))].map((name) => ({ + name, + })), + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "brand.json", + }, + { + // The version string — a bump makes every hardcoded version reference stale. + // `v?` so `v0.23.1` and `0.23.1` both match; group 1 is the bare version. + type: "version", + weight: 0.75, + // Read the TARGET repo's package.json (root), not forge's own BRAND.version, so a + // version bump is detected in whatever repo `docs impact` runs against. + extract: (root) => { + try { + const v = JSON.parse(readFileSync(join(root, "package.json"), "utf8")).version; + return v ? [{ name: v }] : []; + } catch { + return []; + } + }, + ref: (names) => new RegExp(`(? rel === "package.json", + discover: (text) => [...text.matchAll(/"version":\s*"([^"]+)"/g)].map((m) => m[1]), + }, + { + // package.json identity fields — name + keywords. Low weight (keywords are common + // words) but a rename or keyword churn should still point at the docs that echo them. + type: "pkg-field", + weight: 0.4, + extract: (root) => { + try { + const pkg = JSON.parse(readFileSync(join(root, "package.json"), "utf8")); + const names = new Set(); + if (pkg.name) names.add(pkg.name); + for (const k of pkg.keywords ?? []) if (String(k).length >= 3) names.add(String(k)); + return [...names].map((name) => ({ name })); + } catch { + return []; + } + }, + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "package.json", + }, +]; + +/** + * The full current TYPED entity set, one flat list. Fail-open per extractor so a single + * broken source can't blank the whole graph. + * @param {string} [root] + * @returns {{type:string, name:string, weight:number, codeOnly:boolean, meta:object}[]} + */ +export function extractEntities(root = BRAND.root) { + const out = []; + const seen = new Set(); + for (const ex of EXTRACTORS) { + let items = []; + try { + items = ex.extract(root) ?? []; + } catch { + items = []; + } + for (const it of items) { + // A single-character name (e.g. a stray `process.env.X` in a comment) is + // indistinguishable from prose and never a real documented entity — drop it. + if (!it.name || String(it.name).length < 2) continue; + const key = `${ex.type} ${it.name}`; + if (seen.has(key)) continue; + seen.add(key); + out.push({ + type: ex.type, + name: it.name, + weight: ex.weight, + codeOnly: !!ex.codeOnly, + meta: it.meta ?? {}, + }); + } + } + return out; +} + +// --------------------------------------------------------------------------- +// Stage 2 — the generic, code-fence-aware reference index. +// --------------------------------------------------------------------------- + +/** Byte ranges of inline `code` spans on a single line (matched backtick runs). */ +function inlineCodeRanges(line) { + const ranges = []; + const re = /(`+)/g; + let m; + let openIdx = -1; + let openLen = 0; + while ((m = re.exec(line))) { + if (openIdx < 0) { + openIdx = m.index; + openLen = m[1].length; + } else if (m[1].length === openLen) { + ranges.push([openIdx, m.index + m[1].length]); + openIdx = -1; + openLen = 0; + } + } + return ranges; +} + +/** Every prose/doc SURFACE the index owes an answer for — discoverable, not a per-file + * allow-list: all tracked *.md and *.cff, the plugin manifests, the landing page, and + * package.json (its description/keywords are prose). Canonical .js sources and history + * files are excluded. Falls back to a fixed set when git is unavailable (tmp fixtures). */ +export function docSurfaces(root = BRAND.root) { + const set = new Set(); + const tracked = git(root, ["ls-files"]); + const list = tracked + ? tracked.split("\n").filter(Boolean) + : ["README.md", "docs/GUIDE.md", "ARCHITECTURE.md", "ROADMAP.md", "package.json"]; + for (const rel of list) { + if (HISTORY_RE.test(rel)) continue; + if (/\.(md|cff)$/i.test(rel)) set.add(rel); + } + for (const rel of [ + ".claude-plugin/plugin.json", + ".claude-plugin/marketplace.json", + ".codex-plugin/plugin.json", + "landing/index.html", + "package.json", + ]) + if (existsSync(join(root, rel))) set.add(rel); + return [...set].sort(); +} + +// GitHub-style heading slug is overkill here; the raw heading text is a good enough +// "section" label for a human reviewer. +const headingOf = (line) => { + const m = /^#{1,6}\s+(.+?)\s*#*\s*$/.exec(line); + return m ? m[1].trim() : null; +}; + +/** + * Build the inverted index: entityKey → occurrences. Generic over ANY entity list, so it + * indexes the current entities AND the reconstructed removed ones (old version, deleted + * command) the impact query needs. + * @param {string} root + * @param {{type:string,name:string,weight:number,codeOnly?:boolean}[]} entities + * @param {string[]} [surfaces] + * @returns {{index: Map, surfaces: string[]}} + */ +export function buildReferenceIndex(root, entities, surfaces = docSurfaces(root)) { + // One RegExp per (type) — an alternation of every name of that type — so a doc line is + // scanned once per type, not once per entity. group 1 is the matched name. + const byType = new Map(); + for (const e of entities) { + if (!byType.has(e.type)) byType.set(e.type, { spec: e, names: [] }); + byType.get(e.type).names.push(e.name); + } + const scanners = []; + for (const [type, { spec, names }] of byType) { + const ex = EXTRACTORS.find((x) => x.type === type); + if (!ex || !names.length) continue; + scanners.push({ type, spec, re: ex.ref(names) }); + } + + const index = new Map(); + const record = (type, name, occ) => { + const key = `${type} ${name}`; + if (!index.has(key)) index.set(key, []); + index.get(key).push(occ); + }; + + for (const rel of surfaces) { + let text = ""; + try { + text = readFileSync(join(root, rel), "utf8"); + } catch { + continue; + } + const lines = text.split("\n"); + let inFence = false; + let section = null; + for (let i = 0; i < lines.length; i += 1) { + const line = lines[i]; + if (/^\s*(```|~~~)/.test(line)) { + inFence = !inFence; + continue; // the fence marker itself is not content + } + if (!inFence) { + const h = headingOf(line); + if (h) section = h; + } + const codeRanges = inFence ? null : inlineCodeRanges(line); + const inCode = (pos) => + inFence || (codeRanges?.some(([a, b]) => pos >= a && pos < b) ?? false); + for (const { type, spec, re } of scanners) { + re.lastIndex = 0; + let m; + while ((m = re.exec(line))) { + const name = m[1]; + const pos = m.index + m[0].indexOf(name); + const code = inCode(pos); + if (spec.codeOnly && !code) continue; // word-shaped entity in plain prose — skip + record(type, name, { + file: rel, + line: i + 1, + section, + context: inFence ? "fence" : code ? "code" : "prose", + confidence: Math.min(1, spec.weight + (code ? 0.05 : 0)), + snippet: line.trim().slice(0, 160), + }); + } + } + } + } + return { index, surfaces }; +} + +// --------------------------------------------------------------------------- +// Stage 3 — change detection over canonical sources, then the impact query. +// --------------------------------------------------------------------------- + +/** Parse a unified diff into per-file added/removed line arrays. Handles new files + * (untracked, when not --staged) as all-added. */ +function diffHunks(root, { base, staged }) { + const args = staged + ? ["diff", "--unified=0", "--cached"] + : ["diff", "--unified=0", base || "HEAD"]; + const diff = git(root, args); + const perFile = new Map(); + const ensure = (rel) => { + if (!perFile.has(rel)) perFile.set(rel, { added: [], removed: [] }); + return perFile.get(rel); + }; + let current = null; + for (const line of diff.split("\n")) { + const add = /^\+\+\+ b\/(.*)$/.exec(line); + if (add) { + current = add[1] === "/dev/null" ? null : add[1]; + if (current) ensure(current); + continue; + } + const del = /^--- a\/(.*)$/.exec(line); + if (del && del[1] !== "/dev/null") ensure(del[1]); + if (!current) continue; + if (line.startsWith("+") && !line.startsWith("+++")) ensure(current).added.push(line.slice(1)); + else if (line.startsWith("-") && !line.startsWith("---")) + ensure(current).removed.push(line.slice(1)); + } + if (!staged) { + for (const f of git(root, ["ls-files", "--others", "--exclude-standard"]) + .split("\n") + .filter(Boolean)) { + if (perFile.has(f)) continue; + let text = ""; + try { + text = readFileSync(join(root, f), "utf8"); + } catch {} + if (text) perFile.set(f, { added: text.split("\n"), removed: [] }); + } + } + return perFile; +} + +/** + * The entities a diff changed, TYPED. For each type: (a) a current entity is CHANGED if + * its name-token appears on a changed line of one of that type's canonical sources; (b) a + * name the extractor can DISCOVER in that type's REMOVED lines but which is absent from + * the current set is a REMOVED entity (rename/deletion) — its old doc mentions are stale. + * Version bumps additionally recover the OLD version from package.json's removed lines. + * @param {string} root + * @param {{base?:string, staged?:boolean}} [opts] + * @returns {{type:string,name:string,reason:string,removed:boolean}[]} + */ +export function changedEntities(root, { base, staged } = {}) { + const hunks = diffHunks(root, { base, staged }); + const current = extractEntities(root); + const currentByType = new Map(); + for (const e of current) { + if (!currentByType.has(e.type)) currentByType.set(e.type, new Set()); + currentByType.get(e.type).add(e.name); + } + + const changed = []; + const emitted = new Set(); + const emit = (type, name, reason, removed) => { + if (!name || String(name).length < 2) return; + const key = `${type} ${name}`; + if (emitted.has(key)) return; + emitted.add(key); + changed.push({ type, name, reason, removed }); + }; + + for (const ex of EXTRACTORS) { + // Gather this type's changed source hunks. + let addedText = ""; + let removedText = ""; + let touched = false; + for (const [rel, { added, removed }] of hunks) { + if (!ex.isSource(rel)) continue; + touched = true; + addedText += `${added.join("\n")}\n`; + removedText += `${removed.join("\n")}\n`; + } + if (!touched) continue; + const changedText = `${addedText}\n${removedText}`; + const currentNames = currentByType.get(ex.type) ?? new Set(); + + // (a) current entities whose token is on a changed line. + for (const name of currentNames) { + const re = new RegExp(`(?:^|[^\\w$-])${esc(name)}(?![\\w$])`); + if (re.test(changedText)) emit(ex.type, name, "modified", false); + } + // (b) discovered names in removed/added text that no longer exist → renamed/removed. + if (ex.discover) { + const seen = new Set([...ex.discover(removedText), ...ex.discover(addedText)]); + for (const name of seen) if (!currentNames.has(name)) emit(ex.type, name, "removed", true); + } + } + return changed; +} + +/** + * The reusable core. Compute the changed entities, index every doc surface for BOTH the + * current and the removed entities, and return each changed entity with the doc locations + * that reference it — ranked by confidence. + * @param {string} root + * @param {{base?:string, staged?:boolean, minConfidence?:number, surfaces?:string[]}} [opts] + */ +export function docsImpact(root = BRAND.root, opts = {}) { + const { base, staged, minConfidence = 0, surfaces } = opts; + const changed = changedEntities(root, { base, staged }); + + // Index the CURRENT entities plus the reconstructed removed ones (so an old version or a + // deleted command still has doc occurrences to point at). + const entities = extractEntities(root); + const known = new Set(entities.map((e) => `${e.type} ${e.name}`)); + for (const c of changed) { + const key = `${c.type} ${c.name}`; + if (known.has(key)) continue; + known.add(key); + const ex = EXTRACTORS.find((x) => x.type === c.type); + entities.push({ + type: c.type, + name: c.name, + weight: ex?.weight ?? 0.5, + codeOnly: !!ex?.codeOnly, + meta: { removed: true }, + }); + } + + const { index, surfaces: scanned } = buildReferenceIndex( + root, + entities, + surfaces ?? docSurfaces(root), + ); + + const impacted = []; + for (const c of changed) { + const occ = (index.get(`${c.type} ${c.name}`) ?? []) + .filter((o) => o.confidence >= minConfidence) + .sort( + (a, b) => b.confidence - a.confidence || a.file.localeCompare(b.file) || a.line - b.line, + ); + if (occ.length) impacted.push({ ...c, occurrences: occ }); + } + // Entities with the most / highest-confidence hits first. + impacted.sort( + (a, b) => + (b.occurrences[0]?.confidence ?? 0) - (a.occurrences[0]?.confidence ?? 0) || + b.occurrences.length - a.occurrences.length, + ); + + const files = new Set(); + for (const e of impacted) for (const o of e.occurrences) files.add(o.file); + + return { + base: staged ? "--staged" : base || "HEAD", + changed, + impacted, + surfaces: scanned, + summary: { + changedEntities: changed.length, + impactedEntities: impacted.length, + impactedFiles: files.size, + occurrences: impacted.reduce((n, e) => n + e.occurrences.length, 0), + }, + }; +} + +/** Human rendering — grouped by changed entity, each hit cited file:line with confidence. */ +export function renderDocsImpact(report, { maxPerEntity = 8 } = {}) { + const out = [ + `docs impact — diff vs ${report.base}: ${report.summary.changedEntities} changed entit${ + report.summary.changedEntities === 1 ? "y" : "ies" + }, ${report.summary.impactedFiles} doc file(s) potentially stale`, + ]; + if (!report.changed.length) { + out.push(" clean — no documented entity changed in this diff."); + return out.join("\n"); + } + if (!report.impacted.length) { + out.push( + ` ${report.summary.changedEntities} entit${ + report.summary.changedEntities === 1 ? "y" : "ies" + } changed, but no doc surface references them — nothing to review.`, + ); + return out.join("\n"); + } + for (const e of report.impacted) { + const tag = e.removed ? "REMOVED" : "changed"; + out.push(` ${e.type} \`${e.name}\` (${tag}) — ${e.occurrences.length} reference(s):`); + for (const o of e.occurrences.slice(0, maxPerEntity)) { + const where = o.section ? ` [${o.section}]` : ""; + out.push( + ` ${o.file}:${o.line} (${o.context}, ${o.confidence.toFixed(2)})${where} ${o.snippet}`, + ); + } + if (e.occurrences.length > maxPerEntity) + out.push(` (+${e.occurrences.length - maxPerEntity} more)`); + } + out.push( + " → review each surface: update it, or confirm the mention is still correct. Advisory — image/design/paraphrase impact is not detectable by a token scan.", + ); + return out.join("\n"); +} diff --git a/test/docs_impact.test.js b/test/docs_impact.test.js new file mode 100644 index 0000000..4ed8e14 --- /dev/null +++ b/test/docs_impact.test.js @@ -0,0 +1,257 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { COMMANDS } from "../src/commands.js"; +import { + buildReferenceIndex, + changedEntities, + docSurfaces, + docsImpact, + EXTRACTORS, + extractEntities, +} from "../src/docs_impact.js"; +import { TOOLS } from "../src/mcp_tools.js"; + +// A real git repo — docs impact reads git diffs, so the pipeline can't be exercised +// against a bare directory. Writes `files`, commits them as the baseline, and returns +// helpers to mutate + inspect. +function repo(files = {}) { + const root = mkdtempSync(join(tmpdir(), "forge-impact-")); + const g = (...args) => execFileSync("git", args, { cwd: root, encoding: "utf8" }); + const write = (rel, content) => { + const p = join(root, rel); + mkdirSync(join(p, ".."), { recursive: true }); + writeFileSync(p, content); + }; + for (const [rel, content] of Object.entries(files)) write(rel, content); + g("init", "-q"); + g("config", "user.email", "t@t.t"); + g("config", "user.name", "t"); + g("add", "-A"); + g("commit", "-qm", "baseline"); + return { root, g, write }; +} + +// A directory (no git) for pure reference-index unit tests where we pass surfaces explicitly. +function scratch(files = {}) { + const root = mkdtempSync(join(tmpdir(), "forge-idx-")); + for (const [rel, content] of Object.entries(files)) { + const p = join(root, rel); + mkdirSync(join(p, ".."), { recursive: true }); + writeFileSync(p, content); + } + return root; +} + +// --- Stage 1: entity extraction ------------------------------------------------------ + +test("extractEntities: the registry yields the typed entity set from canonical sources", () => { + const { root } = repo({ + "package.json": JSON.stringify({ + name: "widget", + version: "3.1.0", + keywords: ["forge-test"], + }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function runWidget() {}\n", + }); + const ents = extractEntities(root); + const by = (type) => ents.filter((e) => e.type === type).map((e) => e.name); + + // Root-parametrized types read the TARGET repo. + assert.ok(by("env").includes("FORGE_TEST_KNOB"), "env var from process.env scan"); + assert.ok(by("symbol").includes("runWidget"), "exported symbol"); + assert.ok(by("version").includes("3.1.0"), "version from the repo's package.json"); + assert.ok(by("pkg-field").includes("widget"), "package name"); + + // Registry-backed types come from forge's own COMMANDS / TOOLS (self-check semantics). + assert.ok(by("command").includes(Object.keys(COMMANDS)[0]), "a command name"); + assert.ok(by("mcp-tool").includes(TOOLS[0].name), "an MCP tool name"); +}); + +test("extractEntities: single-character names are dropped (no `process.env.X` prose noise)", () => { + const root = scratch({ "src/a.js": "const x = process.env.X;\n" }); + const ents = extractEntities(root); + assert.ok(!ents.some((e) => e.name.length < 2)); +}); + +test("EXTRACTORS is data-driven — a new entity type is one record, no call-site edits", () => { + // Every extractor exposes the same shape, so the pipeline never special-cases a type. + for (const ex of EXTRACTORS) { + assert.equal(typeof ex.type, "string"); + assert.equal(typeof ex.extract, "function"); + assert.equal(typeof ex.ref, "function"); + assert.equal(typeof ex.isSource, "function"); + } +}); + +// --- Stage 2: reference index (word boundaries + code fences) ------------------------- + +test("buildReferenceIndex: word-boundary aware — a longer token is not a match", () => { + const root = scratch({ + "README.md": [ + "# t", + "Run `forge docs` to check. But forge docsxyz is unrelated.", + "Set FORGE_ABC now; FORGE_ABCDEF is a different var.", + ].join("\n"), + }); + const entities = [ + { type: "command", name: "docs", weight: 0.9 }, + { type: "env", name: "FORGE_ABC", weight: 0.9 }, + ]; + const { index } = buildReferenceIndex(root, entities, ["README.md"]); + assert.equal(index.get("command docs")?.length, 1, "matches `forge docs`, not `forge docsxyz`"); + assert.equal(index.get("env FORGE_ABC")?.length, 1, "matches FORGE_ABC, not FORGE_ABCDEF"); +}); + +test("buildReferenceIndex: code-fence aware — a word-shaped symbol only counts in code", () => { + const root = scratch({ + "README.md": [ + "# t", + "The word build appears in plain prose here.", // prose → must NOT count + "Call `build` to compile.", // inline code → counts + "```js", + "build();", // fenced code → counts + "```", + ].join("\n"), + }); + const entities = [{ type: "symbol", name: "build", weight: 0.6, codeOnly: true }]; + const { index } = buildReferenceIndex(root, entities, ["README.md"]); + const hits = index.get("symbol build") ?? []; + assert.equal(hits.length, 2, "only the inline-code and fenced occurrences"); + assert.ok(hits.every((h) => h.context !== "prose")); +}); + +test("buildReferenceIndex: records section, line, and a confidence bonus inside code", () => { + const root = scratch({ + "README.md": ["## Setup", "Set FORGE_ABC in your shell.", "Or `FORGE_ABC` inline."].join("\n"), + }); + const { index } = buildReferenceIndex( + root, + [{ type: "env", name: "FORGE_ABC", weight: 0.9 }], + ["README.md"], + ); + const hits = index.get("env FORGE_ABC"); + assert.equal(hits.length, 2); + assert.equal(hits[0].section, "Setup"); + assert.equal(hits[0].line, 2); + const code = hits.find((h) => h.context === "code"); + const prose = hits.find((h) => h.context === "prose"); + assert.ok(code.confidence > prose.confidence, "inline-code hit is more confident"); +}); + +// --- docSurfaces discovery ----------------------------------------------------------- + +test("docSurfaces: discovers markdown but exempts CHANGELOG (append-only history)", () => { + const { root } = repo({ + "README.md": "# r\n", + "docs/GUIDE.md": "# g\n", + "CHANGELOG.md": "# c\n", + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + }); + const surfaces = docSurfaces(root); + assert.ok(surfaces.includes("README.md")); + assert.ok(surfaces.includes("docs/GUIDE.md")); + assert.ok(surfaces.includes("package.json")); + assert.ok(!surfaces.includes("CHANGELOG.md"), "history is exempt"); +}); + +// --- Stage 3: change detection + impact query ---------------------------------------- + +test("impact: a removed env var flags the README that documents it", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function go() {}\n", + "README.md": "# w\n\nSet `FORGE_TEST_KNOB` to configure the widget.\n", + }); + write("src/app.js", "export function go() {}\n"); // remove the env read + const changed = changedEntities(root, { base: "HEAD" }); + assert.ok( + changed.some((c) => c.type === "env" && c.name === "FORGE_TEST_KNOB" && c.removed), + "env var detected as removed", + ); + const r = docsImpact(root, { base: "HEAD" }); + const env = r.impacted.find((e) => e.type === "env" && e.name === "FORGE_TEST_KNOB"); + assert.ok(env, "env var is in the impacted set"); + assert.ok(env.occurrences.some((o) => o.file === "README.md")); +}); + +test("impact: a version bump flags docs that hardcode the OLD version", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }, null, 2), + "src/app.js": "export function go() {}\n", + "README.md": "# w — v1.0.0\n\nUpgrade guide for v1.0.0.\n", + }); + write("package.json", JSON.stringify({ name: "w", version: "2.0.0" }, null, 2)); + const r = docsImpact(root, { base: "HEAD" }); + const oldV = r.impacted.find((e) => e.type === "version" && e.name === "1.0.0"); + assert.ok(oldV, "the OLD version is a changed (removed) entity"); + assert.ok( + oldV.occurrences.some((o) => o.file === "README.md"), + "README's v1.0.0 flagged", + ); + assert.ok( + r.impacted.some((e) => e.type === "version" && e.name === "2.0.0"), + "new version too", + ); +}); + +test("impact: a renamed command (commands.js diff) flags its old doc mentions", () => { + // The command extractor discovers table keys in a changed src/commands.js. A key that + // no longer exists in forge's COMMANDS registry is treated as removed/renamed. + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/commands.js": "export const COMMANDS = {\n frobnicate: 'do a thing',\n};\n", + "README.md": "# w\n\nRun `forge frobnicate` to do the thing.\n", + }); + write("src/commands.js", "export const COMMANDS = {\n frobulate: 'do a thing',\n};\n"); + const changed = changedEntities(root, { base: "HEAD" }); + assert.ok( + changed.some((c) => c.type === "command" && c.name === "frobnicate" && c.removed), + "the removed command key is detected", + ); + const r = docsImpact(root, { base: "HEAD" }); + const cmd = r.impacted.find((e) => e.type === "command" && e.name === "frobnicate"); + assert.ok(cmd?.occurrences.some((o) => o.file === "README.md")); +}); + +test("impact: a diff that changes no documented entity yields nothing to review", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "export function go() {}\n", + "README.md": "# w\n\nHello.\n", + }); + write("notes.txt", "just some scratch prose, no entities\n"); + const r = docsImpact(root, { base: "HEAD" }); + assert.equal(r.impacted.length, 0); +}); + +test("impact: a changed entity mentioned ONLY in CHANGELOG is not reported (history)", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function go() {}\n", + "CHANGELOG.md": "# Changelog\n\n## [1.0.0]\n\n- added FORGE_TEST_KNOB\n", + }); + write("src/app.js", "export function go() {}\n"); + const r = docsImpact(root, { base: "HEAD" }); + assert.ok( + !r.impacted.some((e) => e.occurrences.some((o) => o.file === "CHANGELOG.md")), + "CHANGELOG mentions are exempt", + ); +}); + +test("docsImpact: --staged reads the index, and the report shape is stable", () => { + const { root, g, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function go() {}\n", + "README.md": "# w\n\nSet `FORGE_TEST_KNOB`.\n", + }); + write("src/app.js", "export function go() {}\n"); + g("add", "src/app.js"); + const r = docsImpact(root, { staged: true }); + assert.equal(r.base, "--staged"); + assert.ok("summary" in r && "surfaces" in r && Array.isArray(r.changed)); + assert.ok(r.impacted.some((e) => e.type === "env" && e.name === "FORGE_TEST_KNOB")); +}); From c999c13a6ff52ac6f2faa859b803e21cffab6852 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 20 Jul 2026 00:24:18 +0000 Subject: [PATCH 2/3] ci(security): run gitleaks binary directly instead of the licensed action MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit dependabot #91 bumped gitleaks/gitleaks-action v2→v3, whose breaking update requires a paid GITLEAKS_LICENSE for org-associated accounts and HARD-FAILS when its license-validation server is unreachable ("License key validation will be enforced") — turning the secret-scan gate red on master and every PR for reasons unrelated to the code. Replace the action with the pinned gitleaks binary (8.21.2). `gitleaks detect` walks the full commit history (fetch-depth:0) exactly as before, with no network license dependency, so the gate stays deterministic and blocking. set -euo pipefail keeps it fail-closed. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/security.yml | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml index 9af0901..143ce0d 100644 --- a/.github/workflows/security.yml +++ b/.github/workflows/security.yml @@ -2,6 +2,13 @@ # Deliberately blocking — a real credential in the tree or history should fail the build, # not warn. (Rotate first, then rewrite/land; a red X is the cheap part of a leak.) # +# We run the gitleaks BINARY directly rather than gitleaks/gitleaks-action@v3: that action's +# breaking update requires a paid GITLEAKS_LICENSE for org-associated accounts and hard-fails +# the job when its license-validation server is unreachable ("License key validation will be +# enforced"), turning a green gate red for reasons unrelated to the code. The pinned binary +# performs the identical scan (`gitleaks detect` walks every commit) with no network license +# dependency, so the gate stays deterministic and blocking. +# # Why this passes on our own redaction tests: test/_fixtures.js assembles secret-LOOKING # strings at RUNTIME via .join() precisely so no secret-format literal exists in any blob # for a scanner to match. Verified clean with gitleaks 8.16 across the whole history and @@ -24,7 +31,13 @@ jobs: - uses: actions/checkout@v7 with: fetch-depth: 0 # gitleaks scans commit history, not just the checkout - - uses: gitleaks/gitleaks-action@v3 + - name: Secret scan (gitleaks binary — no license, scans full history) env: - GITHUB_TOKEN: ${{ github.token }} # PR comments / job summary - # GITLEAKS_LICENSE is only required for ORG-owned repos; this repo is user-owned. + GITLEAKS_VERSION: "8.21.2" + run: | + set -euo pipefail + curl -sSfL \ + "https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz" \ + | tar -xz -C /tmp gitleaks + # `detect` walks the whole git log (fetch-depth:0 above); non-zero exit on any find. + /tmp/gitleaks detect --source . --no-banner --redact --exit-code 1 From c26e14150cfea0074c00aa8d2c9c4a0ca8ce5fb6 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 20 Jul 2026 00:33:59 +0000 Subject: [PATCH 3/3] ci(security): allowlist synthetic PEM fixtures in the redaction test suite MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Running the gitleaks binary (prior commit) surfaced 2 findings the licensed action never reached — it was erroring on license validation before scanning, so the gate was effectively blind. Both are false positives: synthetic PEM private-key literals in test/secrets.test.js, the suite whose whole job is to assert forge redacts secret-shaped strings. They are committed, so a full-history scan always matches them. Add test/secrets.test.js to the .gitleaks.toml path allowlist (alongside the existing bibliography-key exceptions); every other path stays fully scanned. Verified locally: gitleaks 8.21.2 → 'no leaks found', exit 0. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/security.yml | 9 +++++---- .gitleaks.toml | 6 +++++- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml index 143ce0d..6c08fc9 100644 --- a/.github/workflows/security.yml +++ b/.github/workflows/security.yml @@ -9,10 +9,11 @@ # performs the identical scan (`gitleaks detect` walks every commit) with no network license # dependency, so the gate stays deterministic and blocking. # -# Why this passes on our own redaction tests: test/_fixtures.js assembles secret-LOOKING -# strings at RUNTIME via .join() precisely so no secret-format literal exists in any blob -# for a scanner to match. Verified clean with gitleaks 8.16 across the whole history and -# working tree before this workflow was added — no .gitleaks.toml allowlist is needed. +# Why this passes on our own redaction tests: most secret-LOOKING fixtures are assembled at +# RUNTIME (test/_fixtures.js .join()) so no secret-format literal exists in a blob to match. +# The few committed synthetic fixtures that a scanner can't tell from the real thing — the +# PEM/private-key literals in test/secrets.test.js — are allowlisted by path in .gitleaks.toml +# (which also carries the bibliography-key exceptions). Every other path is fully scanned. name: Security on: diff --git a/.gitleaks.toml b/.gitleaks.toml index ca1c7e7..56391d6 100644 --- a/.gitleaks.toml +++ b/.gitleaks.toml @@ -8,10 +8,14 @@ useDefault = true # The literal field name "key" next to a quoted alphanumeric value trips the default # generic-api-key rule even though there is no secret here — just citation metadata. [allowlist] -description = "Bibliography citation-key fields, not secrets" +description = "Bibliography citation-key fields and synthetic redaction-test fixtures, not secrets" paths = [ '''^research/formal-synthesis/graded_reference_set\.json$''', '''^research/formal-synthesis/merged_references\.json$''', + # test/secrets.test.js is the redaction test suite: it holds synthetic PEM private-key and + # high-entropy-token fixtures on purpose and asserts forge redacts them. Those literals are + # committed, so a full-history scan always matches them — they are NOT real credentials. + '''^test/secrets\.test\.js$''', ] # Obvious placeholder credentials used as test fixtures to exercise the secret # redactor/scanner — sequential-hex tokens that are unmistakably fake, not real secrets.