diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 1e92608d2..be7663305 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -78,6 +78,15 @@ "metadata": { "recommended": false } + }, + { + "name": "aidd-qa", + "source": "./plugins/aidd-qa", + "description": "Acceptance QA: validates observable behavior against acceptance criteria and records reviewer evidence. Browser is the only supported interface today.", + "strict": true, + "metadata": { + "recommended": false + } } ] } diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6b30bef6c..0ee0bfacd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -232,6 +232,7 @@ jobs: aidd-refine, aidd-ui, aidd-telemetry, + aidd-qa, ] steps: - name: Check if this plugin was released diff --git a/.release-please-manifest.json b/.release-please-manifest.json index c54203832..5727ea153 100644 --- a/.release-please-manifest.json +++ b/.release-please-manifest.json @@ -8,5 +8,6 @@ "plugins/aidd-refine": "3.0.1", "plugins/aidd-ui": "0.2.1-alpha.0", "plugins/aidd-telemetry": "0.2.0", + "plugins/aidd-qa": "0.1.0", "cli": "5.3.0" } diff --git a/README.md b/README.md index 72715cdc3..760238e8d 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Unify **engineering teams** around **standardized workflows** and **shared best ๐Ÿงฑ **IDE agnostic** ยท ๐Ÿ—๏ธ **Legacy systems** ยท ๐ŸŒฑ **Token-optimized** ยท ๐Ÿ‡ซ๐Ÿ‡ท **Made in France**

- 8 plugins ยท 51 skills ยท 2 agents + 9 plugins ยท 51 skills ยท 2 agents

[![Open Source](https://img.shields.io/badge/Open_Source-Yes-yellow?logo=open-source-initiative&logoColor=white)](https://opensource.org/) @@ -58,7 +58,7 @@ Why not just write your own commands? โ†’ [FAQ](docs/FAQ.md#-why-aidd-instead-of ### Claude Code -Installs the 6 stable plugins (`aidd-ui` is ๐Ÿšง alpha and `aidd-telemetry` ๐Ÿงช beta, install separately โ€” see [Plugins](#-plugins)). +Installs the 6 stable plugins (`aidd-ui` is ๐Ÿšง alpha, `aidd-telemetry` ๐Ÿงช beta, and `aidd-qa` ๐Ÿ†• new, install those separately โ€” see [Plugins](#-plugins)). **In the session** (slash commands) @@ -240,7 +240,7 @@ learning only when it is durable enough to improve the next feature. ## ๐Ÿงฉ Plugins -Eight plugins covering the whole SDLC โ€” **install all of them**; they work together. (`aidd-ui` is ๐Ÿšง **alpha** and `aidd-telemetry` ๐Ÿงช **beta** โ€” both off the curated path.) +Nine plugins covering the whole SDLC โ€” **install the six stable ones**; they work together. (`aidd-ui` is ๐Ÿšง **alpha**, `aidd-telemetry` ๐Ÿงช **beta**, and `aidd-qa` ๐Ÿ†• **new** โ€” all three off the curated path.) @@ -257,9 +257,9 @@ Project init, memory bank, context-artifact generation, diagrams, learning, expl ### โš™๏ธ [aidd-dev](plugins/aidd-dev/README.md) -`11 skills` ยท stable +`10 skills` ยท stable -Code transformation: plan, implement, assert, audit, review, test, refactor, debug. Standalone Browser QA records short web evidence. +Code transformation: plan, implement, assert, audit, review, test, refactor, debug. - +
@@ -320,7 +320,15 @@ UI / UX design โ€” smoke-test only, not ready for use. Answers what a piece of work cost โ€” tokens, models, and which skill spent them. The switch is git-tracked, so it applies to everyone who clones; opt out per person with `AIDD_TELEMETRY=0`. Nothing leaves your machine. + +### ๐ŸŽฌ [aidd-qa](plugins/aidd-qa/README.md) ๐Ÿ†• + +`1 skill` ยท **new** + +Acceptance QA โ€” locks browser scenarios from acceptance criteria and records reviewer evidence. + +
diff --git a/aidd_docs/memory/architecture.md b/aidd_docs/memory/architecture.md index 7390570bf..7ba45d316 100644 --- a/aidd_docs/memory/architecture.md +++ b/aidd_docs/memory/architecture.md @@ -10,13 +10,13 @@ The macro technical shape: the stack, how the pieces fit, and the decisions behi | --- | --- | | Product | markdown โ€” skills, agents, rules, templates. No framework runtime; an LLM interprets them. | | Delivery | Node `>=22.12`, pnpm. `cli/` is the `aidd` binary; `kanban/` is a private package. | -| Manifest | `.claude-plugin/marketplace.json`, the plugin manifest: 8 plugins, no version among them. Versions are release-please's, in `deployment.md`. | +| Manifest | `.claude-plugin/marketplace.json`, the plugin manifest: 9 plugins, no version among them. Versions are release-please's, in `deployment.md`. | ## How it fits together ```mermaid flowchart LR - Manifest[".claude-plugin/marketplace.json"] -->|lists| Plugins["plugins/ ยท 8"] + Manifest[".claude-plugin/marketplace.json"] -->|lists| Plugins["plugins/ ยท 9"] Plugins -->|ships| Surfaces["skills ยท agents ยท commands ยท hooks ยท rules"] CLI["cli/ ยท aidd"] -->|reads| Manifest CLI -->|installs| Target["a project's AI tool dir"] @@ -40,6 +40,6 @@ The concern-to-plugin taxonomy is canonical in [`docs/ARCHITECTURE.md`](../../do ## Gotchas -- 8 plugins ship, 2 off the curated install path: `aidd-ui` is alpha, `aidd-telemetry` beta and opt-in. +- 9 plugins ship, 3 off the curated install path: `aidd-ui` is alpha, `aidd-telemetry` beta and opt-in, `aidd-qa` new and unproven outside this repository. - A skill never links outside itself: the tree ships both flat and as a marketplace, so no relative path survives both. - Bundled hooks run Node. No `node` on `PATH`, no memory refresh and no run journal. diff --git a/aidd_docs/memory/deployment.md b/aidd_docs/memory/deployment.md index 47fa1ab55..901f9cdd9 100644 --- a/aidd_docs/memory/deployment.md +++ b/aidd_docs/memory/deployment.md @@ -50,7 +50,7 @@ Branch model in `vcs.md`, cadence and safety rules in [`RELEASE.md`](../../RELEA 4. Archives are staged outside the repo tree, uploaded with `gh release upload --clobber`. 5. `back-merge.yml` folds `main` into `next`. -Config: `release-please-config.json`, ten packages. Manifest: `.release-please-manifest.json`. +Config: `release-please-config.json`, eleven packages. Manifest: `.release-please-manifest.json`. ## Gotchas diff --git a/aidd_docs/memory/project-brief.md b/aidd_docs/memory/project-brief.md index 781087fc7..80099b5b7 100644 --- a/aidd_docs/memory/project-brief.md +++ b/aidd_docs/memory/project-brief.md @@ -40,6 +40,7 @@ What this project is, the problem it solves, and its domain language. The non-de | Generate context artifacts | `aidd-context:03-context-generate` and its per-kind generators | | Development loop | `aidd-dev` โ€” plan, implement, assert, audit, review, test, refactor, debug | | Typed product backlog | `aidd-pm` โ€” brief, epic, story, spec, spike, defect | +| Acceptance QA evidence | `aidd-qa:01-acceptance-qa` | | Refine input and output | `aidd-refine` โ€” brainstorm, challenge, blind spots | | End-to-end orchestration | `aidd-orchestrator:01-sdlc` | | Measure what a session cost | `aidd-telemetry`, opt-in, plus `aidd telemetry` | diff --git a/aidd_docs/memory/testing.md b/aidd_docs/memory/testing.md index 39acfa3e6..eb1b17575 100644 --- a/aidd_docs/memory/testing.md +++ b/aidd_docs/memory/testing.md @@ -13,7 +13,7 @@ How the project is tested: the layers, the tools, and the conventions. Where tes | `cli/` | vitest, four projects โ€” see the CLI bank | | `kanban/` | its own vitest suite. It shares no code with `cli/` | | Per-tool distributions | golden snapshots in `cli/tests/golden/`, mirrored by the `build-per-tool` CI matrix; Claude Code's own `plugin validate` over a fresh claude build, in `cli-ci.yml` | -| Browser journeys | `aidd-dev:11-browser-qa`, see below | +| Browser journeys | `aidd-qa:01-acceptance-qa`, see below | ## Tools @@ -46,4 +46,4 @@ CI runs more: `validate.yml` re-runs the whole pre-commit over the whole tree on - Runner: `npx --yes @playwright/cli@0.1.17`, the framework pin. Never `latest` during QA. - Also required: `ffmpeg` and `ffprobe`. Output is WebM evidence per scenario. -- Owned by `aidd-dev:11-browser-qa`; this repository ships the capability, it has no browser journey of its own. +- Owned by `aidd-qa:01-acceptance-qa`; this repository ships the capability, it has no browser journey of its own. diff --git a/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/backlog-link.json b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/backlog-link.json new file mode 100644 index 000000000..53c06a37d --- /dev/null +++ b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/backlog-link.json @@ -0,0 +1,5 @@ +{ + "backlog": "ai-driven-dev/framework#908", + "written_at": "2026-09-24T00:00:00Z", + "written_by": "aidd-dev:01-plan" +} diff --git a/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-1.md b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-1.md new file mode 100644 index 000000000..7a3cd3cbe --- /dev/null +++ b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-1.md @@ -0,0 +1,91 @@ +--- +status: pending +--- + +# Instruction: Scaffold `aidd-qa` with the acceptance QA skill + +## Architecture projection + +> Tree of the final files. โœ… create ยท โœ๏ธ modify ยท โŒ delete + +```txt +plugins/aidd-qa/ +โ”œโ”€โ”€ .claude-plugin/plugin.json โœ… name, version 0.1.0, description, skills[] +โ”œโ”€โ”€ README.md โœ… concern, skill table, browser-only scope, install line +โ””โ”€โ”€ skills/01-acceptance-qa/ + โ”œโ”€โ”€ SKILL.md โœ… router: prerequisites โ†’ load-scope โ†’ prepare-run โ†’ run-scenarios + โ”œโ”€โ”€ actions/00-prerequisites.md โœ… moved from aidd-dev, unchanged behavior + โ”œโ”€โ”€ actions/01-load-scope.md โœ… rewritten: scenarios derive from acceptance criteria only + โ”œโ”€โ”€ actions/02-prepare-run.md โœ… moved, deterministic setup and teardown kept + โ”œโ”€โ”€ actions/03-run-scenarios.md โœ… moved, report fields expected/actual/verdict/evidence + โ”œโ”€โ”€ assets/qa-report-template.md โœ… one row per scenario: expected, actual, verdict, evidence + โ””โ”€โ”€ references/interface-browser-playwright-cli.md โœ… moved recording contract, names browser as the supported interface +``` + +## User Journey + +```mermaid +flowchart TD + A[Acceptance criteria + reviewed candidate] --> B[prerequisites] + B --> C[load-scope: one scenario per browser-observable criterion] + C --> D[prepare-run: auth, fixtures, teardown] + D --> E[run-scenarios: record, verdict, reset] + E --> F[qa.md + qa/*.webm] +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Setup + copy the plugin into a fresh marketplace checkout => plugin directory present: 5: system + section Happy path + run check-architecture-rules on plugins/aidd-qa => no violation: 5: cli + section Edge case - criterion not browser-observable + load-scope given a criterion with no browser outcome => criterion listed as out of interface, no scenario: 5: system +``` + +## Tasks to do + +### `1)` Manifest and README + +> The plugin declares itself and one skill. + +1. `plugin.json` from `aidd-ui`'s shape: `name: aidd-qa`, `version: 0.1.0`, description "Acceptance QA: validates observable behavior against acceptance criteria and records reviewer evidence. Use when โ€ฆ Do NOT use for โ€ฆ", `skills: ["./skills/01-acceptance-qa"]`, keywords. +2. `README.md`: concern, skill table, browser as the only interface today, `/plugin install aidd-qa@aidd-framework`. No sibling-plugin address. + +### `2)` Move the skill + +> Browser QA lives under `aidd-qa` with history kept. + +1. `git mv plugins/aidd-dev/skills/11-browser-qa plugins/aidd-qa/skills/01-acceptance-qa` (phase 2 recreates the redirect in `aidd-dev`). +2. Rename `references/run-scope-playwright-cli.md` to `references/interface-browser-playwright-cli.md`; update the link in `03-run-scenarios.md`. +3. `SKILL.md`: frontmatter `name: 01-acceptance-qa`, description stating input = acceptance criteria + reviewed candidate, browser interface; router table and transversal rules kept; add rule "never derive a scenario from the diff or the source code". + +### `3)` Acceptance-derived scope + +> Scenarios come from acceptance criteria, not from implementation. + +1. `01-load-scope` Input: acceptance criteria (issue, spec, plan) + reference to the reviewed candidate (branch, commit, or running URL). +2. Process: one happy path from the criteria's primary journey, edge cases only from criteria or the plan's browser Test Scope; each scenario keeps the criterion it proves; a criterion with no browser-observable outcome is listed as out of interface, never tested by reading code. +3. Remove steps that source edge cases from the implementation artifact or related tests. +4. Add a `## Test` section to every action that lacks one. + +### `4)` Report per scenario + +> Each scenario states expected, actual, verdict, evidence. + +1. `qa-report-template.md`: header verdict, source (acceptance criteria path), candidate, run date; table `Scenario | Criterion | Expected | Actual | Verdict | Evidence`. +2. `03-run-scenarios` Report step and Test reference those fields; video validation steps kept. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| --- | --- | +| 1 | `plugin.json` validates against its schema; no `aidd-:` of another plugin in README | +| 2 | `plugins/aidd-qa/skills/01-acceptance-qa/` holds 4 actions, 1 asset, 1 reference; `check-architecture-rules.js` passes | +| 3 | `01-load-scope` names acceptance criteria as its only scenario source and forbids diff or source-derived scenarios | +| 4 | the report template has Expected, Actual, Verdict, Evidence columns; video checks (codec, dimension, duration, frames) still present | diff --git a/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-2.md b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-2.md new file mode 100644 index 000000000..9c22452b1 --- /dev/null +++ b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-2.md @@ -0,0 +1,66 @@ +--- +status: pending +--- + +# Instruction: Retire `aidd-dev:11-browser-qa` to a redirect + +## Architecture projection + +> Tree of the final files. โœ… create ยท โœ๏ธ modify ยท โŒ delete + +```txt +plugins/aidd-dev/ +โ”œโ”€โ”€ .claude-plugin/plugin.json โœ๏ธ description drops Browser QA; skills[] keeps ./skills/11-browser-qa +โ”œโ”€โ”€ README.md โœ๏ธ Browser QA row becomes a moved note +โ””โ”€โ”€ skills/11-browser-qa/ + โ”œโ”€โ”€ SKILL.md โœ… retired redirect router, one action + โ””โ”€โ”€ actions/01-redirect.md โœ… prints the migration message and stops +aidd_docs/memory/testing.md โœ๏ธ Browser QA owner is aidd-qa +``` + +## User Journey + +```mermaid +flowchart TD + A[User invokes the old Browser QA skill] --> B[redirect action] + B --> C[Message: moved to the aidd-qa plugin + install command] + C --> D[Stop, no QA run] +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Happy path + invoke the retired skill => migration message naming aidd-qa and its install command: 5: system + section Edge case - description matching + ask for browser QA with both plugins installed => description declares itself retired and not for running QA: 5: system +``` + +## Tasks to do + +### `1)` Redirect skill + +> The old invocation answers with a migration message. + +1. `SKILL.md`: `name: 11-browser-qa`, description "Retired. Explains where browser QA moved. Use only when this skill is invoked by name. Do NOT use to run QA or record evidence." Actions table with `redirect`. +2. `actions/01-redirect.md`: Input none; Output the message "Browser QA moved to the `aidd-qa` plugin. Install it with `/plugin install aidd-qa@aidd-framework` (or `aidd plugin install aidd-qa`) and run its acceptance QA skill."; Process print and stop, never run QA; `## Test`. +3. No `aidd-:` token anywhere in the redirect. + +### `2)` aidd-dev surface + +> aidd-dev no longer claims Browser QA. + +1. `plugin.json` description: remove "plus short standalone Browser QA evidence". +2. `README.md`: remove Browser QA from the covers sentence; row 2.11 says retired, moved to the `aidd-qa` plugin. +3. `aidd_docs/memory/testing.md`: owner `aidd-qa`, skill `01-acceptance-qa`. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| --- | --- | +| 1 | the redirect holds no QA process; `check-architecture-rules.js` passes on it | +| 2 | `grep -i "browser qa" plugins/aidd-dev/.claude-plugin/plugin.json` finds nothing; memory names `aidd-qa` as owner | diff --git a/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-3.md b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-3.md new file mode 100644 index 000000000..cb03aef56 --- /dev/null +++ b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-3.md @@ -0,0 +1,70 @@ +--- +status: pending +--- + +# Instruction: Register the plugin and update the docs + +## Architecture projection + +> Tree of the final files. โœ… create ยท โœ๏ธ modify ยท โŒ delete + +```txt +.claude-plugin/marketplace.json โœ๏ธ aidd-qa entry, strict, metadata.recommended false +release-please-config.json โœ๏ธ plugins/aidd-qa package +.release-please-manifest.json โœ๏ธ "plugins/aidd-qa": "0.1.0" +.github/workflows/ci.yml โœ๏ธ build-plugin matrix gains aidd-qa +commitlint.config.cjs โœ๏ธ scope-enum gains aidd-qa, qa +docs/ARCHITECTURE.md โœ๏ธ concerns table row: aidd-qa, Acceptance QA, Execution + status note +docs/CATALOG.md โœ๏ธ aidd-qa section; Browser QA row removed from aidd-dev +README.md โœ๏ธ plugin counts, aidd-qa section, aidd-dev line drops Browser QA +aidd_docs/memory/architecture.md โœ๏ธ 9 plugins, 3 off the curated path +aidd_docs/memory/project-brief.md โœ๏ธ key feature row for acceptance QA +``` + +## User Journey + +```mermaid +flowchart TD + A[marketplace.json lists aidd-qa] --> B[release-please versions it] + B --> C[ci.yml builds its archive] + A --> D[docs and taxonomy name it] +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Happy path + run the scripts suite => release-covers-every-plugin and architecture-doc tests pass: 5: cli + section Edge case - curated install + read marketplace.json => aidd-qa has recommended false: 5: system +``` + +## Tasks to do + +### `1)` Registration + +> Every release and CI guard sees the plugin. + +1. Marketplace entry after `aidd-telemetry`, description by concern, `recommended: false`. +2. release-please package block copied from `plugins/aidd-ui`; manifest `0.1.0`. +3. `ci.yml` matrix and commitlint scopes. + +### `2)` Docs and memory + +> Every place that counts or lists plugins stays true. + +1. `docs/ARCHITECTURE.md` row + a one-line status note (off the curated path until proven). +2. `docs/CATALOG.md` and `README.md`: add aidd-qa, fix counts and the aidd-dev description, keep the README badge/status style used for `aidd-ui`/`aidd-telemetry`. +3. Memory `architecture.md` gotcha count, `project-brief.md` feature row. +4. Let `pnpm exec lefthook run pre-commit` regenerate catalogs and counts; never hand-edit generated files. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| --- | --- | +| 1 | `release-covers-every-plugin.test.js` passes; marketplace JSON validates | +| 2 | `architecture-doc-matches-the-tree.test.js` passes; no doc still says 8 plugins or attributes Browser QA to aidd-dev | diff --git a/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-4.md b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-4.md new file mode 100644 index 000000000..0aec832d1 --- /dev/null +++ b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/phase-4.md @@ -0,0 +1,62 @@ +--- +status: pending +--- + +# Instruction: Prove it installs and translates to a second host + +## Architecture projection + +> Tree of the final files. โœ… create ยท โœ๏ธ modify ยท โŒ delete + +```txt +aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/validation.md โœ… commands run and their decisive output +``` + +## User Journey + +```mermaid +flowchart TD + A[Gates] --> B[claude plugin validate plugins/aidd-qa] + B --> C[cli build + aidd translate to Codex] + C --> D[validation.md] +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Setup + build the CLI => dist/cli.js present: 5: cli + section Happy path + translate the marketplace to codex into the scratchpad => aidd-qa skill, actions, asset, reference present: 5: cli + section Edge case - Claude Code manifest + claude plugin validate plugins/aidd-qa => valid: 5: cli +``` + +## Tasks to do + +### `1)` Gates + +> Every repository gate is green. + +1. `pnpm exec lefthook run pre-commit`. +2. `node scripts/check-tests-leave-git-alone.js -- node --test 'scripts/__tests__/**/*.test.js'`. +3. `pnpm test:changed`. + +### `2)` Host proof + +> The plugin works in Claude Code and Codex. + +1. `claude plugin validate plugins/aidd-qa` (and the marketplace root). +2. `cd cli && pnpm install && pnpm build`; read `node cli/dist/cli.js translate --help`; translate to Codex into the scratchpad; list the aidd-qa output. +3. Record commands and decisive lines in `validation.md`. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| --- | --- | +| 1 | all three commands exit 0 | +| 2 | Claude Code validation passes and the Codex output contains the acceptance QA skill with its four actions | diff --git a/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/plan.md b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/plan.md new file mode 100644 index 000000000..19987b003 --- /dev/null +++ b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/plan.md @@ -0,0 +1,46 @@ +--- +objective: "An installable, off-curated-path aidd-qa plugin owns browser acceptance QA derived from acceptance criteria, and aidd-dev no longer carries browser QA." +status: implemented +--- + +# Plan: aidd-qa plugin + +## Overview + +| Field | Value | +| --- | --- | +| **Goal** | Create `aidd-qa`, move Browser QA into it as acceptance QA, remove the `aidd-dev` skill, register the plugin everywhere a plugin is registered | +| **Source** | https://github.com/ai-driven-dev/framework/issues/908 | + +## Phases + +| # | Phase | File | +| --- | --- | --- | +| 1 | Scaffold `aidd-qa` with the acceptance QA skill | [`phase-1.md`](./phase-1.md) | +| 2 | Retire `aidd-dev:11-browser-qa` to a redirect | [`phase-2.md`](./phase-2.md) | +| 3 | Register the plugin and update the docs | [`phase-3.md`](./phase-3.md) | +| 4 | Prove it installs and translates to a second host | [`phase-4.md`](./phase-4.md) | + +## Resources + +| Source | Verified | +| --- | --- | +| `docs/CREATE_PLUGIN.md` | registration = marketplace entry with `metadata.recommended: false`, release-please config and manifest; no cross-plugin reference in descriptions or READMEs | +| `docs/ARCHITECTURE.md` + `scripts/__tests__/architecture-doc-matches-the-tree.test.js` | the concerns table must hold one row per plugin in the tree | +| `scripts/__tests__/release-covers-every-plugin.test.js` | `ci.yml` `build-plugin` matrix and `release-please-config.json` must list every marketplace plugin | +| `scripts/lib/architecture-rules.js` (`PLUGIN_ADDRESS`) | any `aidd-:` token (optional `/` or `@`) in a skill, action, reference or agent of another plugin is an orthogonality violation; a bare plugin name or `aidd-qa@aidd-framework` is not | +| commit 627408fb (aidd-telemetry added) | touchpoints for a new plugin: marketplace, release manifest, README, memory; `.claude/settings.json` enables only curated plugins | +| PR #512 (Browser QA landed) | `aidd-vcs` pull-request draft links `**/qa/*.webm`; keeping the `qa/` evidence folder name keeps that link working | + +## Decisions + +| Decision | Why | +| --- | --- | +| Skill `aidd-qa:01-acceptance-qa`, browser as its only interface, declared in an interface reference | the entry point is named by intention (acceptance validation), so API or CLI interfaces can be added later without renaming; none is claimed now | +| Layer Execution in the taxonomy | it drives the running application, which the Knowledge firewall forbids | +| `aidd-dev:11-browser-qa` is removed after a first redirect iteration | `aidd-dev` owns no QA surface; `aidd-qa` is the single owner | +| `aidd-dev:06-test` `test-journey` stays in `aidd-dev` | it is developer-side validation the SDLC Deliver zone runs before commit, not independent acceptance evidence; moving it is outside #908 | +| Evidence folder stays `qa/` with `happy-path.webm` and `edge-case-.webm` | the pull-request draft already links `**/qa/*.webm` | +| First release tagged `aidd-qa-v1.0.0` via `release-as` on the package, manifest at `0.1.0` | a `Release-As` footer on the squash commit would also re-version every other path it touches; drop `release-as` once `v1.0.0` ships | +| Not added to `.claude/settings.json` `enabledPlugins` | that list holds only curated plugins; `aidd-ui` and `aidd-telemetry` are absent too | +| Commits split by path | release-please bumps per path from the commit type | diff --git a/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/validation.md b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/validation.md new file mode 100644 index 000000000..2f361f8ee --- /dev/null +++ b/aidd_docs/tasks/2026_09/2026_09_24_aidd-qa-plugin/validation.md @@ -0,0 +1,152 @@ +--- +status: done +--- + +# Validation: aidd-qa plugin + +Commands run from the repository root unless noted, with their decisive output line. The original "Gates", "Host proof", and "Architecture conformance" tables below were all run against that implementation's own final working tree, before it was split into its first five commits โ€” not re-run per commit. Each commit's own `pre-commit` hook run exited 0 (a non-zero exit would have aborted the commit), but that hook reads the working tree at commit time, not a diff scoped to that commit's own files; see "Commits" below for what that means for the two intermediate trees. "Repair re-run", further down, records a separate, later run against this repair's own final tree โ€” read its own heading for which tree it covers, not this paragraph. + +Review #908 found five defects after the push recorded in the original "Push" section: a missing run-verdict rule in `03-run-scenarios.md`, a `load-scope` that never stopped early when no criterion is browser-observable, a duplicate `## Test` bullet, two hand-written docs still describing `06-test` as generic test coverage after `ba2a59a5` narrowed it, and this record's own commit table and push section going stale as later commits (`ba2a59a5`, `e8d3538e`, `cff70e66`) landed without an update. Fixing the second defect had a second-pass bug: the first attempt at a `load-scope`-skips-`prerequisites` rule left the router's own action order (`00-prerequisites` before `01-load-scope`) unchanged, so the rule was aspirational rather than true; a second pass moved the criteria check ahead of `00` in `SKILL.md` itself. The docs-wording fix for defect four also had a first-pass bug: it copied this task's own writing constraint ("no sibling-plugin address") into the README and CATALOG rows as if it described the skill, corrected once caught. Both are visible as separate commits below rather than folded into the commits that introduced them, since those commits were already pushed. + +## Gates (phase 4, task 1) + +| # | Command | Exit | Decisive output | +| --- | --- | --- | --- | +| 1 | `pnpm exec lefthook run pre-commit` (final clean sweep, all task files staged) | 0 | `summary: (done in 48.22 seconds)` โ€” `check-skill-argument-hints`, `cli-architecture`, `doc-duplication`, `markdown-links`, `referenced-paths`, `scripts-tests`, `summarize-plugin-catalogs`, `summarize-telemetry-prompts-doc`, `sync-readme-counts` all `โœ”๏ธ`; embedded `scripts-tests` run: `โ„น tests 503` / `โ„น pass 503` / `โ„น fail 0` | +| 2 | `node scripts/check-tests-leave-git-alone.js -- node --test 'scripts/__tests__/**/*.test.js'` | 0 | `โ„น tests 503` / `โ„น pass 501` / `โ„น fail 0` / `โ„น skipped 2` โ€” the 2 skips are named `hands a broken Biome rule back to the agent, naming it` and `formats a file in place and stays silent on what would not block a commit` (`# SKIP`), both pre-existing Biome-gated CLI tests, unrelated to this change | +| 3 | `pnpm test:changed` | 0 | every project block reports `โ„น fail 0`; final combined run `โ„น fail 0` | + +`pre-commit`'s glob-scoped jobs (`architecture-rules`, `json-validity`, `yaml-validity`, `skill-frontmatter`) reported "no files for inspection" on the staged-file glob lefthook resolved in this checkout, so they did not execute inside that run. Ran explicitly instead: `pnpm exec lefthook run pre-commit --all-files --job architecture-rules --job json-validity --job yaml-validity --job skill-frontmatter` โ†’ exit 0, `โœ… Architecture rules: 431 governed file(s) checked, no violation`, `JSON validation passed for 79 file(s).`, `YAML validation passed for 22 file(s).`, `skill-frontmatter` โœ”๏ธ with no breach printed. + +Root `pnpm install` and `cd cli && pnpm install` were run first โ€” neither `node_modules` existed in this worktree, so `js-yaml` (root) and `vitest` (`cli`) were missing and `cli-architecture` failed with `vitest: command not found` (exit 127) until installed. Not a regression from this change; recorded because it would otherwise have looked like an empty-gate false pass. + +One scripts-suite regression was found and fixed as part of this change: `scripts/__tests__/architecture-rules.test.js` pins `skillsWithActions` to the number of skills with an `actions/` dir, swept from the real `plugins/` tree. Adding `aidd-qa:01-acceptance-qa` raises that from 48 to 49; the assertion was updated to `49` (test intent unchanged โ€” it still fails if a sweep silently misses a skill). + +### Repair re-run (review #908 follow-up) + +Each command below was run with its exit code captured directly (`cmd > log 2>&1; echo EXIT=$?`), not through a wrapper that could mask it. This is the last of two re-runs during this repair; the first (after the three initial fixes, before the `SKILL.md` routing and docs-wording corrections) produced the same decisive numbers, so only this final one, against this repair's actual final tree (`docs/CATALOG.md`, `plugins/aidd-dev/README.md`, and `plugins/aidd-qa/skills/01-acceptance-qa/{SKILL.md,actions/01-load-scope.md,actions/03-run-scenarios.md}` staged together), is recorded: + +| # | Command | Exit | Decisive output | +| --- | --- | --- | --- | +| 1 | `pnpm exec lefthook run pre-commit` | 0 | `summary: (done in 48.73 seconds)` โ€” `check-skill-argument-hints`, `doc-duplication`, `markdown-links`, `referenced-paths`, `scripts-tests`, `summarize-plugin-catalogs`, `summarize-telemetry-prompts-doc`, `sync-readme-counts` all `โœ”๏ธ`; embedded `scripts-tests` run: `โ„น tests 503` / `โ„น pass 503` / `โ„น fail 0` / `โ„น skipped 0` | +| 2 | `node scripts/check-architecture-rules.js` (no args, whole governed tree) | 0 | `โœ… Architecture rules: 352 governed file(s) checked, no violation` | +| 3 | `node scripts/check-tests-leave-git-alone.js -- node --test 'scripts/__tests__/**/*.test.js'` | 0 | `โ„น tests 503` / `โ„น suites 22` / `โ„น pass 503` / `โ„น fail 0` / `โ„น skipped 0` | +| 4 | `claude plugin validate plugins/aidd-qa` | 0 | `โœ” Validation passed` | + +Both scripts-suite entries in this re-run agree on one number, 503 tests / 503 pass / 0 fail / 0 skipped โ€” the earlier 503-pass-vs-501-pass-plus-2-skip split recorded above (gates 1 and 2, phase 4) no longer reproduces on this tree. `architecture-rules`, `json-validity`, `skill-frontmatter`, and `yaml-validity` again reported "no files for inspection" against lefthook's staged-file glob in this checkout, the same quirk noted above; `check-architecture-rules.js` was run explicitly instead, as this task's dispatch required, rather than via `--all-files --job`. `summarize-plugin-catalogs` and `sync-readme-counts` ran but changed nothing โ€” `git status` after each pre-commit run showed no file beyond the ones already staged. + +### Second repair re-run (review #908, second follow-up) + +Commands below were run with their exit code captured directly (`cmd > log 2>&1; echo EXIT=$?`), against this round's tree with commit 15 (`SKILL.md`, `01-load-scope.md`, `02-prepare-run.md`, `03-run-scenarios.md`, `qa-report-template.md`) already landed and this file staged alone for commit 16: + +| # | Command | Exit | Decisive output | +| --- | --- | --- | --- | +| 1 | `pnpm exec lefthook run pre-commit` | 0 | `summary: (done in 3.27 seconds)` โ€” `doc-duplication`, `markdown-links`, `referenced-paths`, `summarize-plugin-catalogs`, `summarize-telemetry-prompts-doc`, `sync-readme-counts` all `โœ”๏ธ`; `architecture-rules`, `check-skill-argument-hints`, `json-validity`, `scripts-tests`, `skill-frontmatter`, `yaml-validity` skipped โ€” this round's only staged file for this run is this task doc, which matches none of their globs | +| 2 | `node scripts/check-architecture-rules.js` (no args, whole governed tree) | 0 | `โœ… Architecture rules: 352 governed file(s) checked, no violation` | +| 3 | `node scripts/check-tests-leave-git-alone.js -- node --test 'scripts/__tests__/**/*.test.js'` | 0 | `โ„น tests 503` / `โ„น suites 22` / `โ„น pass 503` / `โ„น fail 0` / `โ„น skipped 0` | +| 4 | `claude plugin validate plugins/aidd-qa` | 0 | `โœ” Validation passed` | + +`git status` after the pre-commit run showed no file beyond `validation.md`, already staged โ€” `summarize-plugin-catalogs` and `sync-readme-counts` changed nothing, since commit 15 touched no `CATALOG.md`-governed surface. + +## Host proof (phase 4, task 2) + +| # | Command | Exit | Decisive output | +| --- | --- | --- | --- | +| 1 | `claude plugin validate plugins/aidd-qa` | 0 | `โœ” Validation passed` | +| 2 | `claude plugin validate plugins/aidd-dev` | 0 | `โœ” Validation passed` (redirect skill included) | +| 3 | `claude plugin validate .` | 0 | `โœ” Validation passed` (marketplace root, 9 plugins) | +| 4 | `cd cli && pnpm install && pnpm build` | 0 | `Bundle size: 727.1 KB / budget: 734 KB` / `OK: within budget` | +| 5 | `node cli/dist/cli.js translate --help` | 0 | `--to Conversion target (claude, cursor, copilot, codex, opencode, kilo)` | +| 6 | `node cli/dist/cli.js translate . --to codex --out /translate-codex --as marketplace` | 0 | `Built 9 plugins, 477 files written to /translate-codex` | +| 7 | `HOME=/install-sandbox/home node cli/dist/cli.js setup --source local --path --ai claude,codex --plugins none --yes --scope project` (run from an empty `/install-sandbox/project`, criterion 10) | 0 | `Installed claude, codex (2 files)` | +| 8 | `HOME=/install-sandbox/home node cli/dist/cli.js plugin install aidd-qa --tool claude --scope project --yes` then the same with `--tool codex` (criterion 10) | 0 (both) | `Warning: Native plugin activation โ€” upgrade marketplace 'aidd-framework' skipped: codex marketplace upgrade aidd-framework failed: Error: marketplace aidd-framework is not configured as a Git marketplace` then `Installed 'aidd-qa'.` (both tools; the warning is expected โ€” the sandbox's marketplace source is local, not Git) | + +Gate 8's install landed the full skill surface under both tools' caches โ€” `find /install-sandbox/home/.claude/plugins/cache/aidd-framework/aidd-qa/0.1.0/skills/01-acceptance-qa -name '*.md'` lists `SKILL.md` plus its 4 actions, 1 asset, 1 reference, matching the phase-1 projection and gate 6's Codex translate listing; `diff` against the working tree's own `SKILL.md` at commit 15's tree reported no difference, so the installed copy carries this round's Rejected-section and skip-destination fix, not a stale cached one. + +Translated output confirms the full skill surface reached Codex: + +``` +plugins/aidd-qa/.codex-plugin/plugin.json +plugins/aidd-qa/skills/01-acceptance-qa/SKILL.md +plugins/aidd-qa/skills/01-acceptance-qa/actions/00-prerequisites.md +plugins/aidd-qa/skills/01-acceptance-qa/actions/01-load-scope.md +plugins/aidd-qa/skills/01-acceptance-qa/actions/02-prepare-run.md +plugins/aidd-qa/skills/01-acceptance-qa/actions/03-run-scenarios.md +plugins/aidd-qa/skills/01-acceptance-qa/assets/qa-report-template.md +plugins/aidd-qa/skills/01-acceptance-qa/references/interface-browser-playwright-cli.md +``` + +4 actions, 1 asset, 1 reference โ€” matches the phase-1 architecture projection. + +## Architecture conformance + +| Command | Exit | Decisive output | +| --- | --- | --- | +| `node scripts/check-architecture-rules.js $(find plugins/*/skills plugins/*/agents -name '*.md')` | 0 | `โœ… Architecture rules: 433 governed file(s) checked, no violation` | +| `node --test scripts/__tests__/architecture-doc-matches-the-tree.test.js` | 0 | `the plugin concerns table has one row per plugin in the tree` passes | +| `node --test scripts/__tests__/release-covers-every-plugin.test.js` | 0 | `build-plugin builds an archive for every plugin the marketplace lists` and `release-please versions every plugin the marketplace lists` both pass | +| `node scripts/check-doc-duplication.js` | 0 | `โœ… Doc duplication: 0 duplicated sentence(s) in 48 files` | +| `node scripts/check-referenced-paths.js` | 0 | `โœ… Referenced paths: 0 dead in 34 files` | +| `node scripts/check-skill-argument-hints.mjs` | 0 | `Every skill names what the user brings.` | +| `node scripts/check-markdown-links.js --ignore cli/tests/fixtures --ignore cli/aidd_docs/tasks` | 0 | `โœ… Links: 0 broken in 791 files` | + +`/aidd-dev:03-assert`'s `assert-architecture` facet (report-only): no macro violation โ€” `plugins/aidd-qa/` matches the documented plugin anatomy (`.claude-plugin/plugin.json` + `skills/01-acceptance-qa/{SKILL.md, actions/, assets/, references/}`, no unused optional surfaces); no micro violation โ€” the skill's action files carry no cross-plugin address. `assert-frontend` was skipped: this change ships markdown only, no running UI to drive. + +No `aidd-:` token for another plugin appears in `plugins/aidd-qa/**` or `plugins/aidd-dev/skills/11-browser-qa/**`; the redirect names only the bare `aidd-qa` plugin and `/plugin install aidd-qa@aidd-framework` (no colon after `aidd-qa`, so `PLUGIN_ADDRESS` does not match it). + +## Commits + +All commits on this branch (`git log origin/next..HEAD`), oldest first. Rows 1-8 were pushed before the first repair started (`cff70e66` confirmed reaching `origin` at that push, original "Push" section below). Rows 9-14 are that first repair's, each carrying its real local SHA `git rev-parse --short=8 HEAD` printed right after its own `git commit`, captured into this table before the next commit ran; row 14's SHA (`3c81dfb7`) is filled in now that it is known, though the commit that first wrote this row could not yet state it. Rows 15-16 are a second round, driven by this review's follow-up findings on load-scope's skip destination, run-scenarios' verdict ordering for a rejected scenario set, and this record's own defect count and commit table; row 16, this file's own commit, cannot state its own SHA โ€” read it off `git log` or `git ls-remote` on the pushed branch. + +| # | SHA | Subject | +| --- | --- | --- | +| 1 | `3ef728a7` | `feat(aidd-qa): scaffold acceptance QA plugin from browser QA` | +| 2 | `ffb9c9c5` | `feat(aidd-dev): retire browser-qa to a redirect` | +| 3 | `50c13400` | `chore(marketplace): register aidd-qa plugin` | +| 4 | `f5c3dfcb` | `docs(aidd-qa): add the plan and its validation record` | +| 5 | `cd048e6b` | `docs(aidd-qa): correct the validation record` | +| 6 | `ba2a59a5` | `fix(aidd-dev): scope 06-test to developer-side validation` | +| 7 | `e8d3538e` | `fix(aidd-qa): restrict criteria sourcing and complete the report contract` | +| 8 | `cff70e66` | `docs: correct plugin count and the pushed validation record` | +| 9 | `89243332` | `fix(aidd-qa): add run-verdict rule and stop load-scope early` โ€” `03-run-scenarios.md`, `01-load-scope.md`, `SKILL.md` | +| 10 | `db49deaa` | `fix(aidd-dev): describe 06-test as developer-side in its README` | +| 11 | `b642829f` | `docs: correct 06-test's catalog description and the validation record` โ€” `docs/CATALOG.md`, this file (an earlier draft of it) | +| 12 | `911bcab5` | `fix(aidd-qa): route the zero-criterion skip before prerequisites` โ€” `SKILL.md`, correcting the routing bug commit 9's transversal rule left in place, and correcting commit 9's own commit-body claim about where `load-scope` stops | +| 13 | `a6e0cda4` | `fix(aidd-dev): drop the addressing note from the 06-test README row` โ€” correcting a writing-constraint phrase commit 10 copied into product-facing text | +| 14 | `3c81dfb7` | `docs: correct 06-test's catalog description and the validation record` โ€” `docs/CATALOG.md`, this file | +| 15 | `ec1cf16f` | `fix(aidd-qa): report a fully rejected scenario set as blocked, not skipped` โ€” `SKILL.md`, `01-load-scope.md`, `02-prepare-run.md`, `03-run-scenarios.md`, `qa-report-template.md`, correcting review #908's follow-up warning about a scenario set emptied by rejection | +| 16 | (this round's final docs commit โ€” this record) | `docs: correct the validation record's defect count and commit table, add install proof` โ€” this file | + +`summarize-plugin-catalogs` and `sync-readme-counts` regenerate `plugins/*/CATALOG.md` and README's counts block from the live working tree, not from the commit's own staged diff โ€” the working tree already held the final content when commit 1 ran, so `plugins/aidd-dev/CATALOG.md` in commit 1 already describes the redirect that only lands in commit 2. That is a known, accepted side effect; it does not change what either commit's hand-authored content says. It also means the two intermediate trees are not independently clean against the gates in this file: + +- **At commit 1:** `plugins/aidd-dev/.claude-plugin/plugin.json` still lists `"./skills/11-browser-qa"` in `skills[]`, but that tree has no `plugins/aidd-dev/skills/11-browser-qa/` directory (it moved to `aidd-qa` in this same commit, and the redirect is not added until commit 2). `scripts/__tests__/architecture-rules.test.js`'s `skillsWithActions` sweep would read `48` on this tree, not the `49` the pinned assertion (also changed in commit 1) expects โ€” the pin only becomes true at commit 2, once the redirect's own `actions/` directory exists. This was a mistake in how the pin's commit placement was chosen, caught only while writing this correction, not fixed by rewriting unpushed history. +- **At commit 1 and 2:** no `aidd-qa` entry exists yet in `.claude-plugin/marketplace.json`, so `release-covers-every-plugin.test.js` and `architecture-doc-matches-the-tree.test.js`'s concerns-table check would fail on those trees in isolation (9 plugin directories, 8-row concerns table / 8-plugin marketplace). + +The commits were not restructured to fix this: the tree every gate in the original "Gates", "Host proof", and "Architecture conformance" tables above was actually run against is that implementation's final one (after commit 3), and splitting further would move the same CATALOG-regeneration mismatch somewhere else rather than remove it. The same applies to the first repair's own six commits (9-14): "Repair re-run" above was run once, against that repair's final tree, not per commit. The second round's own two commits (15-16) were checked the same way; see "Second repair re-run" below. + +## Push + +`git push -u origin feat/aidd-qa-plugin` first failed on `pnpm exec lefthook run pre-push` (glob `cli/**` โ€” see below for why it ran), at `cli-test`, before any network call: + +| Job | Result | +| --- | --- | +| `cli-knip` | โœ”๏ธ | +| `cli-test` (`pnpm --dir cli test`) | โœ– `Test Files 1 failed \| 528 passed \| 1 skipped (530)` / `Tests 1 failed \| 6794 passed \| 1 skipped (6796)`, decisive line: `tests/e2e/sandbox-reaches-no-tool-binary.e2e.test.ts:51 AssertionError: expected '' not to be ''` | + +The failing assertion is `E2E: the sandbox a test spawns into > still reaches node and git, which the code under test genuinely needs`; under this test's synthetic sandboxed `PATH`, `which node` returns nothing. Isolated it and confirmed: + +- Root cause, verified rather than guessed: `ls "$(dirname "$(node -p 'process.execPath')")" | grep -xE 'opencode|claude|codex|copilot|cursor-agent'` printed `codex` โ€” this machine's `node` (via nvm) shares a `bin/` directory with a `codex` binary. `pathWithoutAidd()` in `cli/tests/e2e/helpers.ts` builds the sandbox `PATH` from `dirname(process.execPath)` among others, then runs `.filter(withoutDrivableToolBinary)`, which drops any directory holding an AI-tool binary โ€” dropping node's own directory along with it because `codex` sits next to it. Machine-specific: `git diff c3a3355f..HEAD --stat -- cli/` is empty (none of this branch's commits touch `cli/`), and the failing test file was last changed in `95bdbbc3` (2026-09-09), weeks before this task โ€” a pre-existing local gap, not a regression. + +No workaround that bypasses or weakens the gate was used: no `--no-verify`, no excluding the job, no editing the test. Instead, the collision itself was fixed for this shell: the `node` binary was copied โ€” not symlinked, since `process.execPath` resolves a symlink back to the original, `codex`-sharing directory โ€” into an isolated directory holding no AI-tool binary, which was then prepended to `PATH` for the push. With that `PATH`, `pnpm exec lefthook run pre-push` passed in full (`cli-knip` โœ”๏ธ, `cli-test` all passing, no failing file), and `git push -u origin feat/aidd-qa-plugin` completed without `--no-verify`. `cd048e6b` and every commit before it on this branch reached `origin` at that push; confirmed with `git ls-remote origin refs/heads/feat/aidd-qa-plugin` printing that SHA. + +This record does not have first-hand detail on how `ba2a59a5`, `e8d3538e`, and `cff70e66` individually reached `origin` โ€” they were not pushed in this session. What is directly known: the session that made this repair started with `cff70e66` already at `origin/feat/aidd-qa-plugin` (its own environment snapshot reported the branch as pushed at that SHA before any repair commit existed), and this repair's own push used the same mechanism as the one detailed above โ€” an isolated directory holding only a copied `node` binary, prepended to `PATH` for `git push`, never `--no-verify` โ€” because the local `node`/`codex` collision this shell sits on has not changed. That push's `pnpm exec lefthook run pre-push` ran `cli-knip` and `cli-test` again (this repair changed no file under `cli/`); its own exit code and `git ls-remote` output are the evidence, not a claim repeated here. This file does not track a single frozen "pushed tip" SHA: the branch's tip is whatever `HEAD` is when the pull request is opened, i.e. the last row of the "Commits" table above at that time. `git ls-remote origin refs/heads/feat/aidd-qa-plugin` is the way to read it, not this paragraph. + +## Deviations from the plan + +- Updated `scripts/__tests__/architecture-rules.test.js`'s pinned `skillsWithActions` count (48 โ†’ 49) โ€” not named in any phase file, required because the suite hardcodes a measured count that a new skill-with-actions legitimately changes. +- Updated `docs/MAINTAINERS.md`'s package count line (`10 packages (root + 8 plugins + cli)` โ†’ `11 packages (root + 9 plugins + cli)`) โ€” not named in phase-3, but it is the same fact `deployment.md` states and would otherwise go stale. +- `plugins/aidd-qa/README.md` and `docs/CATALOG.md`'s new `aidd-qa` section do not use the `[N.x]` "Bracket ID" numbering the curated plugins use: `aidd-telemetry`, the other off-curated-path plugin, never adopted that convention either (confirmed by grep โ€” no `[8.x]` rows exist in its README), so `aidd-qa` follows the same off-curated precedent rather than inventing a `[9.x]` series nobody else has used since telemetry landed. +- Root `README.md`'s "Plugins" intro changed from "install all of them" to "install the six stable ones", and the Claude Code install line's off-curated parenthetical grew a third name (`aidd-qa`). The plan asked only for a new tile and corrected counts; this wording change was made because "install all of them" was already inaccurate before this change (it excluded `aidd-ui` and `aidd-telemetry`, both already off the curated path) and adding a third off-curated plugin made the inaccuracy harder to ignore. Flagging it as a judgment call beyond the plan's literal scope rather than reverting it silently. +- `/aidd-dev:02-implement` was not invoked as a skill; the phases were implemented directly and validated against each phase's own "Test acceptance criteria" table by hand. `/aidd-dev:03-assert` was invoked and its two applicable facets (`01-assert`, `02-assert-architecture`) run as reported above; `03-assert-frontend` was skipped with a stated reason. +- `phase-1.md` through `phase-4.md` are committed with their original `status: pending` frontmatter unchanged. Only `plan.md`'s `status` was set to `implemented`, per this dispatch's explicit instruction; no instruction named a phase-file status convention, and none was invented. +- `/aidd-vcs:01-commit` was invoked through the Skill tool for commit 1 only, which surfaced its `01-collect` / `02-message` / `03-commit` process. Commits 2-5 followed that same process by hand (stage the concern's files, message from the imposed text, `git commit`, verify with `git show --stat`) without re-invoking the skill each time. The review #908 repair (commits 9-14) invoked `/aidd-vcs:01-commit` through the Skill tool once for commit 9, then followed the same by-hand process for commits 10-14. The second round (commits 15-16) invoked `/aidd-vcs:01-commit` through the Skill tool for commit 15, then followed the same by-hand process for commit 16. diff --git a/commitlint.config.cjs b/commitlint.config.cjs index 2a6a85071..7014c848c 100644 --- a/commitlint.config.cjs +++ b/commitlint.config.cjs @@ -14,6 +14,7 @@ module.exports = { "aidd-refine", "aidd-orchestrator", "aidd-ui", + "aidd-qa", "context", "dev", "vcs", @@ -23,6 +24,7 @@ module.exports = { "ui", "aidd-telemetry", "telemetry", + "qa", "cli", "kanban", "framework", diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index cf063a76f..486617d92 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -96,9 +96,12 @@ Every capability lives in exactly one plugin, chosen by **concern**. This taxono | `aidd-orchestrator` | Orchestration | Coordination | | `aidd-ui` ๐Ÿšง | UI/UX design | Execution | | `aidd-telemetry` ๐Ÿงช | Measurement | Observation | +| `aidd-qa` ๐Ÿ†• | Acceptance QA | Execution | `aidd-ui` is alpha: smoke-test only, off the curated install path. +`aidd-qa` is new, off the curated install path until it is proven outside this repository. It validates observable behavior against acceptance criteria and drives a browser to record evidence, so it sits in the Execution layer alongside `aidd-dev`. + `aidd-telemetry` is beta, off the curated install path: opt-in only โ€” a repository must commit `.aidd/config.json` with `telemetry.enabled: true`. Each session appends observations, one JSON object per line, to its own `aidd_docs/runs/__.jsonl`, created on demand and git-ignored; that directory's presence is a location, not a permission. A line is never rewritten, only appended โ€” `session_start`, `turn_end`, `file_written`, `step_start`, `step_end`, `task_declared` and `unrecognised_payload` (a path is repository-relative, never a task_id: task identity is a derivation, and belongs to whatever reads the log). Never a measurement; tokens and cost are joined afterwards from the provider's telemetry. **Observation** writes only *about* the other layers, never the artifact it describes, and nothing may depend on it. diff --git a/docs/CATALOG.md b/docs/CATALOG.md index 56798519b..5bc2852a4 100644 --- a/docs/CATALOG.md +++ b/docs/CATALOG.md @@ -10,6 +10,7 @@ The exhaustive list of AIDD plugins, skills, and actions. Skills are invoked thr - [aidd-orchestrator](#-aidd-orchestrator) - async orchestration (optional) - [aidd-ui](#-aidd-ui) - UI / UX (๐Ÿšง alpha, not ready) - [aidd-telemetry](#-aidd-telemetry) - measurement, hooks and skills (๐Ÿงช beta, off the curated path) +- [aidd-qa](#-aidd-qa) - acceptance QA (๐Ÿ†• new, off the curated path) --- @@ -35,7 +36,7 @@ Bootstrap, project init, context-artifact generation, diagrams, learning, and ex ## ๐Ÿ’ป aidd-dev -Code transformation: plan, implement, assert, audit, review, test, refactor, debug, for-sure, todo. Standalone Browser QA records short web evidence. +Code transformation: plan, implement, assert, audit, review, test, refactor, debug, for-sure, todo. | Skill | Role | Actions | | --------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | @@ -44,12 +45,11 @@ Code transformation: plan, implement, assert, audit, review, test, refactor, deb | `03-assert` | Assert features work - general, architecture, frontend UI | `01-assert`, `02-assert-architecture`, `03-assert-frontend` | | `04-audit` | Read-only codebase audit across quality pillars | `01-code-quality`, `02-architecture`, `03-security`, `04-dependencies`, `05-performance`, `06-tests`, `07-ui` | | `05-review` | Read-only review of a diff - code quality and feature behavior | `01-review-code`, `02-review-functional` | -| `06-test` | Write and iterate tests, validate user journeys in the browser | `01-test`, `02-test-journey` | +| `06-test` | Write and iterate developer-side tests, validate journeys in the browser during implementation - not independent acceptance QA or reviewer evidence | `01-test`, `02-test-journey` | | `07-refactor` | Improve code without changing behavior across four axes | `01-performance`, `02-security`, `03-cleanup`, `04-architecture` | | `08-debug` | Reproduce and fix bugs with a test-driven workflow | `01-reproduce`, `02-debug`, `03-reflect-issue` | | `09-for-sure` | Iterative loop that retries until a success condition is met | `01-init-tracking`, `02-auto-accept`, `03-autonomous-loop` | | `10-todo` | Split the prompt into independent todos, run one implementer agent per todo in parallel | `01-todo` | -| `11-browser-qa` | Record short reviewer videos for browser-scoped happy and edge cases | `00-prerequisites`, `01-load-scope`, `02-prepare-run`, `03-run-scenarios` | ## ๐Ÿ“‹ aidd-pm @@ -121,3 +121,11 @@ CLI, and each skill says so before doing anything else if it is missing. | `00-init` | Turn measurement on for a project and prove it is recording | `01-check`, `02-enable`, `03-verify` | | `01-cost` | Answer what a period or one task cost, by step, model and tool | `01-locate`, `02-collect`, `03-report` | | `02-check` | Answer whether measurement is actually recording, line by line | `01-locate`, `02-diagnose` | + +## ๐ŸŽฌ aidd-qa + +Locks a scenario scope from acceptance criteria, runs it against a reviewed candidate, and hands back a per-scenario verdict with recorded evidence. Browser only, for now. + +| Skill | Role | Actions | +| ------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------- | +| `01-acceptance-qa` | Lock scenarios from acceptance criteria, run them, and report a verdict per scenario | `01-load-scope`, `02-prerequisites`, `03-prepare-run`, `04-run-scenarios` | diff --git a/docs/MAINTAINERS.md b/docs/MAINTAINERS.md index 51d66ae0b..982af57d2 100644 --- a/docs/MAINTAINERS.md +++ b/docs/MAINTAINERS.md @@ -13,7 +13,7 @@ How to operate this repository day to day. This file is the **Maintainer** playb | Live backlog & roadmap | [Project board #8](https://github.com/orgs/ai-driven-dev/projects/8) | single source of truth | | Roles โ†’ access | GitHub teams `trusted-partners` / `certified-members` / `core-team` | mapped to the role ladder | | Branch protection | ruleset "main protection" + `.github/rulesets/main.json` | `main` is PR-only | -| Releases | release-please (`ci.yml`) + `release-please-config.json` | 10 packages (root + 8 plugins + `cli`), auto | +| Releases | release-please (`ci.yml`) + `release-please-config.json` | 11 packages (root + 9 plugins + `cli`), auto | | Pre-commit checks | `lefthook.yml` + `scripts/` | json/yaml/schema/frontmatter/catalogs/counts | ## ๐Ÿ“… Daily diff --git a/plugins/aidd-dev/.claude-plugin/plugin.json b/plugins/aidd-dev/.claude-plugin/plugin.json index 917597521..ef4e06046 100644 --- a/plugins/aidd-dev/.claude-plugin/plugin.json +++ b/plugins/aidd-dev/.claude-plugin/plugin.json @@ -2,7 +2,7 @@ "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "aidd-dev", "version": "2.5.0", - "description": "Code transformation: plan, implement, assert, audit, review, test, refactor, debug, for-sure, plus short standalone Browser QA evidence. Hosts engineering agents.", + "description": "Code transformation: plan, implement, assert, audit, review, test, refactor, debug, for-sure. Hosts engineering agents.", "author": { "name": "AI-Driven Dev", "url": "https://github.com/ai-driven-dev" @@ -17,8 +17,7 @@ "./skills/07-refactor", "./skills/08-debug", "./skills/09-for-sure", - "./skills/10-todo", - "./skills/11-browser-qa" + "./skills/10-todo" ], "agents": [ "./agents/executor.md", diff --git a/plugins/aidd-dev/CATALOG.md b/plugins/aidd-dev/CATALOG.md index 671f54194..c54c4de54 100644 --- a/plugins/aidd-dev/CATALOG.md +++ b/plugins/aidd-dev/CATALOG.md @@ -19,7 +19,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai - [`skills/08-debug`](#skills08-debug) - [`skills/09-for-sure`](#skills09-for-sure) - [`skills/10-todo`](#skills10-todo) - - [`skills/11-browser-qa`](#skills11-browser-qa) --- @@ -103,7 +102,7 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai |-------|------|---| | `actions` | [01-test.md](skills/06-test/actions/01-test.md) | - | | `actions` | [02-test-journey.md](skills/06-test/actions/02-test-journey.md) | - | -| `-` | [SKILL.md](skills/06-test/SKILL.md) | `Write and iterate tests until they pass, or validate a user journey end to end in the browser. Use when the user wants to add coverage, find what's untested, or walk a flow. Not for auditing test health or debugging a failure.` | +| `-` | [SKILL.md](skills/06-test/SKILL.md) | `Write and iterate developer-side tests until they pass, or validate a user journey end to end in the browser during implementation. Use when the user wants to add coverage, find what's untested, or walk a flow while building it. Do NOT use for independent acceptance QA or reviewer evidence, auditing test health, or debugging a failure.` | #### `skills/07-refactor` @@ -145,15 +144,3 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai | `actions` | [01-todo.md](skills/10-todo/actions/01-todo.md) | - | | `-` | [SKILL.md](skills/10-todo/SKILL.md) | `Split the user prompt into independent todos and run one executor agent per todo in parallel, then report a minimal table. Use when the user says "todo" or asks to fan out a multi-part request into parallel implementations.` | -#### `skills/11-browser-qa` - -| Group | File | Description | -|-------|------|---| -| `actions` | [00-prerequisites.md](skills/11-browser-qa/actions/00-prerequisites.md) | - | -| `actions` | [01-load-scope.md](skills/11-browser-qa/actions/01-load-scope.md) | - | -| `actions` | [02-prepare-run.md](skills/11-browser-qa/actions/02-prepare-run.md) | - | -| `actions` | [03-run-scenarios.md](skills/11-browser-qa/actions/03-run-scenarios.md) | - | -| `assets` | [qa-report-template.md](skills/11-browser-qa/assets/qa-report-template.md) | - | -| `references` | [run-scope-playwright-cli.md](skills/11-browser-qa/references/run-scope-playwright-cli.md) | - | -| `-` | [SKILL.md](skills/11-browser-qa/SKILL.md) | `Run post-review browser QA and produce short named videos for a locked happy path and sourced browser edge cases. Use when the user wants concise reviewer evidence for a web journey. Not for API, CLI, automated tests, diff review, or application fixes.` | - diff --git a/plugins/aidd-dev/README.md b/plugins/aidd-dev/README.md index a23bc830b..4e0ddb2cf 100644 --- a/plugins/aidd-dev/README.md +++ b/plugins/aidd-dev/README.md @@ -8,7 +8,7 @@ Code transformation plugin for the AI-Driven Development framework. First time? Install with `/plugin install aidd-dev@aidd-framework`, then run `aidd-dev:01-plan`. -Covers code transformation: planning, implementation, assertions, audits, code review, testing, refactoring, debugging, for-sure, and parallel todo fan-out. Standalone Browser QA records short web evidence. Also hosts AI agents. +Covers code transformation: planning, implementation, assertions, audits, code review, testing, refactoring, debugging, for-sure, and parallel todo fan-out. Also hosts AI agents. ## Skills @@ -19,12 +19,11 @@ Covers code transformation: planning, implementation, assertions, audits, code r | [2.3] | [assert](skills/03-assert/SKILL.md) | Assert features work as intended - general assertions, architecture conformance, and frontend UI validation. | | [2.4] | [audit](skills/04-audit/SKILL.md) | Perform deep codebase analysis to identify technical debt, dead code, and improvement opportunities. | | [2.5] | [review](skills/05-review/SKILL.md) | Review a diff along three axes: code quality, feature behavior against the plan, and relevancy (fit to the need, declared-rule conformance, no rot). | -| [2.6] | [test](skills/06-test/SKILL.md) | Write and iterate on tests until they pass, and validate user journeys end-to-end in the browser. | +| [2.6] | [test](skills/06-test/SKILL.md) | Write and iterate on developer-side tests until they pass, and validate user journeys end-to-end in the browser during implementation - not independent acceptance QA or reviewer evidence. | | [2.7] | [refactor](skills/07-refactor/SKILL.md) | Optimize code for performance and fix security vulnerabilities following OWASP guidelines. | | [2.8] | [debug](skills/08-debug/SKILL.md) | Reproduce and fix bugs systematically using test-driven workflow, root cause analysis, and hypothesis validation. | | [2.9] | [for-sure](skills/09-for-sure/SKILL.md) | Iterative agent loop that tracks attempts and retries until a success condition is met. | | [2.10] | [todo](skills/10-todo/SKILL.md) | Split the prompt into independent todos, run one executor agent per todo in parallel, then report a minimal table. | -| [2.11] | [browser-qa](skills/11-browser-qa/SKILL.md) | Record one short named video for a locked browser happy path and each sourced browser edge case. | ## Agents diff --git a/plugins/aidd-dev/skills/06-test/SKILL.md b/plugins/aidd-dev/skills/06-test/SKILL.md index 8500963e4..5645ef9f8 100644 --- a/plugins/aidd-dev/skills/06-test/SKILL.md +++ b/plugins/aidd-dev/skills/06-test/SKILL.md @@ -1,6 +1,6 @@ --- name: 06-test -description: Write and iterate tests until they pass, or validate a user journey end to end in the browser. Use when the user wants to add coverage, find what's untested, or walk a flow. Not for auditing test health or debugging a failure. +description: Write and iterate developer-side tests until they pass, or validate a user journey end to end in the browser during implementation. Use when the user wants to add coverage, find what's untested, or walk a flow while building it. Do NOT use for independent acceptance QA or reviewer evidence, auditing test health, or debugging a failure. argument-hint: scope | journey model: sonnet --- diff --git a/plugins/aidd-dev/skills/11-browser-qa/SKILL.md b/plugins/aidd-dev/skills/11-browser-qa/SKILL.md deleted file mode 100644 index d886556e5..000000000 --- a/plugins/aidd-dev/skills/11-browser-qa/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: 11-browser-qa -description: Run post-review browser QA and produce short named videos for a locked happy path and sourced browser edge cases. Use when the user wants concise reviewer evidence for a web journey. Not for API, CLI, automated tests, diff review, or application fixes. -argument-hint: plan | artifact ---- - -# Browser QA - -```mermaid -flowchart LR - prerequisites["prerequisites"] --> scope["load-scope"] --> prepare["prepare-run"] --> run["run-scenarios"] -``` - -## Actions - -Read only the next action's file before running it. - -| # | Action | Does | -| --- | --------------- | ---------------------------------------------------------- | -| 00 | `prerequisites` | Verify the browser runner and media dependencies | -| 01 | `load-scope` | Lock one happy path and a bounded set of sourced edge cases | -| 02 | `prepare-run` | Resolve the shortest deterministic path to executable runs | -| 03 | `run-scenarios` | Record, normalize, verify, reset, and report every scenario | - -## Transversal rules - -- Run against a reviewed change and never patch the application. -- Never spawn agents. Batch independent reads and tool checks, but keep state-changing browser work sequential. -- Do not narrate action transitions, searches, fixtures, selectors, or successful checks. Report only a blocker, a required decision, or the final verdict and paths. diff --git a/plugins/aidd-dev/skills/11-browser-qa/actions/01-load-scope.md b/plugins/aidd-dev/skills/11-browser-qa/actions/01-load-scope.md deleted file mode 100644 index 6191ee5c6..000000000 --- a/plugins/aidd-dev/skills/11-browser-qa/actions/01-load-scope.md +++ /dev/null @@ -1,28 +0,0 @@ -# 01 - Load Scope - -Lock the smallest defensible browser QA scope before execution. - -## Input - -Plan path or implementation artifact. - -## Output - -- 1 locked browser happy path, -- a bounded set of sourced browser edge cases, -- a source label, -- a resolved evidence folder. - -## Process - -1. **Resolve.** the requested feature from its plan. -2. **Filter.** Keep only Happy path and Edge case sections where every task's Mermaid actor is `browser` (Ignore non-browser, mixed-channel, and unmarked scenario sections without displaying them). -3. **Lock.** Lock 1 browser happy path from the explicit user journey, then the filtered plan Test Scope, then browser-observable acceptance criteria in the implementation artifact. - - Ask one concise question only when those sources conflict or expose multiple browser journeys. -4. **Collect.** Include every filtered planned edge case + candidates from explicit browser validation, error, empty-state, permission, boundary, or recovery branches already visible in the implementation artifact. - - Search directly related browser tests only when the filtered plan contains no edge case. -5. **Bound.** Deduplicate candidates against planned edges. Keep at least 3 proposed edges, ranked by user impact, browser observability, determinism, and proximity to the requested journey. -6. **Decide.** Automatically include a proposed edge only when it is deterministic, browser-observable, in scope, and non-destructive. Require a decision only for an external or destructive action. -7. **Validate.** Reject a scenario without a source, trigger, browser-observable outcome, or executable teardown when it changes state. -8. **Locate.** Use the existing AIDD feature folder when the source belongs to one. Otherwise use `aidd_docs/tasks//_/`. -9. **Show.** Emit `Happy path: locked ()` and one compact `Edge case | Source | Decision` table. Do not repeat scenario steps. diff --git a/plugins/aidd-dev/skills/11-browser-qa/actions/02-prepare-run.md b/plugins/aidd-dev/skills/11-browser-qa/actions/02-prepare-run.md deleted file mode 100644 index f2e0a185e..000000000 --- a/plugins/aidd-dev/skills/11-browser-qa/actions/02-prepare-run.md +++ /dev/null @@ -1,27 +0,0 @@ -# 02 - Prepare Run - -Resolve the application state and scenario paths before retained recording begins. - -## Input - -Verified prerequisites and the earlier defined scope. - -## Output - -A successful prepared run with a reachable application, authenticated sessions, deterministic fixtures, executable scenario steps, and proven teardown. - -## Process - -1. **Preflight.** Check the application and fixed `1280ร—720` viewport. -2. **Reuse.** Read `aidd_docs/memory/testing.md` first when it exists. - - Resolve Browser QA entry, auth, fixtures, and reset from its `Browser QA` section, then a directly related browser test, then one targeted browser snapshot. - - Stop searching as soon as the run is executable. -3. **Authenticate.** Establish the required role before recording. - - Never include login discovery or secret lookup in evidence. -4. **Fixture.** Use deterministic data satisfying each setup. - - Never choose a live record by guesswork. -5. **Rehearse** only non-mutating steps and selectors. - - Never execute the final state-changing action merely to rehearse it. -6. **Reset.** Resolve an executable teardown for every state-changing scenario. - - If preparation changed state, execute the teardown and verify the baseline now; a future restart is not proof. -7. **Return.** Keep only the fixture, initial URL, minimal steps, expected outcome, teardown, and isolated session id per scenario. diff --git a/plugins/aidd-dev/skills/11-browser-qa/actions/03-run-scenarios.md b/plugins/aidd-dev/skills/11-browser-qa/actions/03-run-scenarios.md deleted file mode 100644 index 53b6deb75..000000000 --- a/plugins/aidd-dev/skills/11-browser-qa/actions/03-run-scenarios.md +++ /dev/null @@ -1,29 +0,0 @@ -# 03 - Run Scenarios - -Execute, save, and report one clean browser QA take per scenario. - -## Input - -The prepared run, source label, and resolved evidence folder. - -## Output - -`/qa.md` + 1 final WebM per scenario. - -## Process - -1. **Group.** Run at most two read-only scenarios concurrently in isolated sessions. - - Run every state-changing scenario sequentially. -2. **Record.** Apply setup before recording, then follow the recording contract in [run-scope-playwright-cli.md](../references/run-scope-playwright-cli.md). -3. **Verdict.** Compare actual with expected. - - Retain a product failure and mark the run failed. -4. **Recover.** Discard a setup or tooling failure, reset, and retry once. - - A second operational failure blocks the scenario. -5. **Reset.** Execute teardown after every state-changing take, verify the baseline, then close the session. -6. **Normalize.** Normalize at most two independent raw files concurrently. - - Save only `qa/happy-path.webm` and `qa/edge-case-.webm` after `ffprobe` and chronological frame inspection pass. -7. **Clean.** Delete raw takes and temporary validation frames only after every final file passes codec, dimension, duration, path, cut-point, and frame checks. - - Never retain screenshots or alternate media. -8. **Report.** Fill [qa-report-template.md](../assets/qa-report-template.md) with the source label. - - Keep one result row per scenario and add Findings only for a failure or blocker. -9. **Return.** Output the verdict and evidence paths, then ask `Open happy-path.webm in the browser for review?`; open the final file there when confirmed. diff --git a/plugins/aidd-dev/skills/11-browser-qa/assets/qa-report-template.md b/plugins/aidd-dev/skills/11-browser-qa/assets/qa-report-template.md deleted file mode 100644 index 464dede59..000000000 --- a/plugins/aidd-dev/skills/11-browser-qa/assets/qa-report-template.md +++ /dev/null @@ -1,12 +0,0 @@ -# Browser QA: {{feature}} - -- **Verdict**: {{pass | fail | skipped}} -- **Source**: {{plan path | implementation path | user request}} -- **Run**: {{yyyy_mm_dd}} - -## Scenarios - -| Scenario | Result | Verdict | Evidence | Duration | -| -------- | ------ | ------- | -------- | -------- | - -{{findings-section-when-needed}} diff --git a/plugins/aidd-qa/.claude-plugin/plugin.json b/plugins/aidd-qa/.claude-plugin/plugin.json new file mode 100644 index 000000000..188824f79 --- /dev/null +++ b/plugins/aidd-qa/.claude-plugin/plugin.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", + "name": "aidd-qa", + "version": "0.1.0", + "description": "Acceptance QA: validates observable behavior against acceptance criteria and records reviewer evidence. Use when a reviewed candidate needs evidence that it meets its acceptance criteria. Do NOT use for diff review, unit or integration tests, or fixing the application.", + "author": { + "name": "AI-Driven Dev", + "url": "https://github.com/ai-driven-dev" + }, + "skills": [ + "./skills/01-acceptance-qa" + ], + "keywords": [ + "qa", + "acceptance", + "browser", + "evidence" + ], + "repository": "https://github.com/ai-driven-dev/framework", + "homepage": "https://ai-driven.dev", + "license": "MIT" +} diff --git a/plugins/aidd-qa/CATALOG.md b/plugins/aidd-qa/CATALOG.md new file mode 100644 index 000000000..189426589 --- /dev/null +++ b/plugins/aidd-qa/CATALOG.md @@ -0,0 +1,34 @@ +# aidd-qa catalog + +Auto-generated index of skills, agents, references and assets shipped by the `aidd-qa` plugin. + +> This file is automatically updated by the `scripts/summarize-markdown.js` script. + +## Table of Contents + +- [`.claude-plugin`](#claude-plugin) +- [`skills`](#skills) + - [`skills/01-acceptance-qa`](#skills01-acceptance-qa) + +--- + +### `.claude-plugin` + +| File | +|------| +| [plugin.json](.claude-plugin/plugin.json) | + +### `skills` + +#### `skills/01-acceptance-qa` + +| Group | File | Description | +|-------|------|---| +| `actions` | [01-load-scope.md](skills/01-acceptance-qa/actions/01-load-scope.md) | - | +| `actions` | [02-prerequisites.md](skills/01-acceptance-qa/actions/02-prerequisites.md) | - | +| `actions` | [03-prepare-run.md](skills/01-acceptance-qa/actions/03-prepare-run.md) | - | +| `actions` | [04-run-scenarios.md](skills/01-acceptance-qa/actions/04-run-scenarios.md) | - | +| `assets` | [qa-report-template.md](skills/01-acceptance-qa/assets/qa-report-template.md) | - | +| `references` | [interface-browser-playwright-cli.md](skills/01-acceptance-qa/references/interface-browser-playwright-cli.md) | - | +| `-` | [SKILL.md](skills/01-acceptance-qa/SKILL.md) | `Validate a reviewed candidate's observable behavior against its acceptance criteria and record short named videos as reviewer evidence. Use when acceptance criteria exist and a browser-observable journey needs proof it holds. Do NOT use for diff review, unit or integration tests, or application fixes.` | + diff --git a/plugins/aidd-qa/README.md b/plugins/aidd-qa/README.md new file mode 100644 index 000000000..575ce2064 --- /dev/null +++ b/plugins/aidd-qa/README.md @@ -0,0 +1,17 @@ +โ† [aidd-framework](../../README.md) + +# aidd-qa + +Acceptance QA concern for the AI-Driven Development framework. + +> Status: new, off the curated install path until proven. + +First time? Install with `/plugin install aidd-qa@aidd-framework`, then run `aidd-qa:01-acceptance-qa`. + +Validates a reviewed candidate's observable behavior against its acceptance criteria and records reviewer evidence. Scenarios come only from acceptance criteria, never from the diff or the source code. Browser is the only supported interface today; a scenario without a browser-observable outcome is listed out of interface rather than tested. + +## Skills + +| Skill | Description | +|---|---| +| [acceptance-qa](skills/01-acceptance-qa/SKILL.md) | Validate a reviewed candidate against its acceptance criteria and record one short named video per locked scenario. | diff --git a/plugins/aidd-qa/skills/01-acceptance-qa/SKILL.md b/plugins/aidd-qa/skills/01-acceptance-qa/SKILL.md new file mode 100644 index 000000000..6d4f83e31 --- /dev/null +++ b/plugins/aidd-qa/skills/01-acceptance-qa/SKILL.md @@ -0,0 +1,32 @@ +--- +name: 01-acceptance-qa +description: Validate a reviewed candidate's observable behavior against its acceptance criteria and record short named videos as reviewer evidence. Use when acceptance criteria exist and a browser-observable journey needs proof it holds. Do NOT use for diff review, unit or integration tests, or application fixes. +argument-hint: acceptance criteria | reviewed candidate +--- + +# Acceptance QA + +```mermaid +flowchart LR + scope["load-scope"] --> prerequisites["prerequisites"] --> prepare["prepare-run"] --> run["run-scenarios"] --> report["qa.md"] + scope -- skipped --> report +``` + +## Actions + +Read only the next action's file before running it. + +| # | Action | Does | +| --- | --------------- | ---------------------------------------------------------- | +| 01 | `load-scope` | Lock at most one happy path and the edge cases its acceptance criteria name | +| 02 | `prerequisites` | Verify the browser runner and media dependencies | +| 03 | `prepare-run` | Resolve the shortest deterministic path to executable runs | +| 04 | `run-scenarios` | Record, normalize, verify, reset, and report every scenario | + +## Transversal rules + +- Run against a reviewed change and never patch the application. +- Never delete data not proven test-only. +- Never derive a scenario from the diff or the source code; every scenario traces to an acceptance criterion. +- Never spawn agents. Batch independent reads and tool checks, but keep state-changing browser work sequential. +- Do not narrate action transitions, searches, fixtures, selectors, or successful checks. Report only a blocker, a required decision, or the final verdict and paths. diff --git a/plugins/aidd-qa/skills/01-acceptance-qa/actions/01-load-scope.md b/plugins/aidd-qa/skills/01-acceptance-qa/actions/01-load-scope.md new file mode 100644 index 000000000..94f0fad70 --- /dev/null +++ b/plugins/aidd-qa/skills/01-acceptance-qa/actions/01-load-scope.md @@ -0,0 +1,31 @@ +# 01 - Load Scope + +Lock the smallest defensible acceptance QA scope before execution. + +## Input + +Acceptance criteria (issue, spec, user story, or the user) and the reviewed candidate reference (branch, commit, or running URL). + +## Output + +- Scope: at most 1 happy path and its edge cases, each with the criteria it proves. +- Out of interface: criteria, or quoted parts, with no browser-observable outcome. +- Rejected: scenario, criterion, reason. +- Evidence folder, source label (criteria path), and candidate reference. + +## Process + +1. **Resolve.** Take criteria from the issue, spec, story, or user; never a plan. Note the candidate reference. +2. **Filter.** Keep browser-observable criteria; split a partial one and quote the rest out of interface. Never test by reading code. +3. **Locate.** Use the source's feature folder, else `aidd_docs/tasks//_/`. +4. **Skip.** Nothing kept: write `qa.md` from [qa-report-template.md](../assets/qa-report-template.md) with verdict `skipped`, then stop. +5. **Lock.** 1 happy path from the primary journey; edge cases only when a criterion names them. An edge from the diff, code, tests, or an unmapped plan edge is never a candidate. Ask once when journeys conflict or an edge is external or destructive. +6. **Validate.** Reject a scenario missing a trigger or an observable outcome; keep the reason. +7. **Show.** Evidence folder, happy path, `Edge case | Criteria | Decision` table, Out of interface and Rejected when non-empty. Never repeat steps. + +## Test + +- Every scenario traces to a criterion; none comes from a plan, the diff, code, or tests. +- An unobservable criterion or part is quoted out of interface, never scoped. +- Nothing kept => `qa.md` with `skipped`, and no later action runs. +- A rejection keeps its reason; an unmapped plan edge is never listed as rejected. diff --git a/plugins/aidd-dev/skills/11-browser-qa/actions/00-prerequisites.md b/plugins/aidd-qa/skills/01-acceptance-qa/actions/02-prerequisites.md similarity index 91% rename from plugins/aidd-dev/skills/11-browser-qa/actions/00-prerequisites.md rename to plugins/aidd-qa/skills/01-acceptance-qa/actions/02-prerequisites.md index 9090966a1..dee1bf07f 100644 --- a/plugins/aidd-dev/skills/11-browser-qa/actions/00-prerequisites.md +++ b/plugins/aidd-qa/skills/01-acceptance-qa/actions/02-prerequisites.md @@ -1,10 +1,10 @@ -# 00 - Prerequisites +# 02 - Prerequisites -Verify the runner dependencies before resolving the QA scope. +Verify the runner dependencies once the scope is locked. ## Input -None. +The locked scope. ## Output diff --git a/plugins/aidd-qa/skills/01-acceptance-qa/actions/03-prepare-run.md b/plugins/aidd-qa/skills/01-acceptance-qa/actions/03-prepare-run.md new file mode 100644 index 000000000..f2f9f495d --- /dev/null +++ b/plugins/aidd-qa/skills/01-acceptance-qa/actions/03-prepare-run.md @@ -0,0 +1,28 @@ +# 03 - Prepare Run + +Make every scenario executable before recording. + +## Input + +Verified prerequisites and the locked scope. + +## Output + +Per scenario: fixture, initial URL, minimal steps, expected outcome, proven teardown, and isolated session id. Plus every scenario rejected here, with its reason. + +## Process + +1. **Reuse.** Resolve entry, auth, fixtures, and reset from the `Browser QA` section of `aidd_docs/memory/testing.md` when it exists, then a related browser test, then one targeted snapshot. Stop once executable. + - Before starting the entry, prove the storage it mounts is test-only when its reset deletes data. Unproven: ask once, never guess. +2. **Preflight.** Check the application and fixed `1280ร—720` viewport. +3. **Authenticate.** Establish the role before recording; never show login discovery or secrets in evidence. +4. **Fixture.** Use deterministic data per setup; never pick a live record by guesswork. +5. **Rehearse.** Only non-mutating steps and selectors; never the final state-changing action. +6. **Reset.** Resolve a verified, executable teardown per state-changing scenario; without one, reject the scenario and carry it forward with its reason. If preparation changed state, run the teardown and verify the baseline now, not at a restart. + +## Test + +- An entry on storage not proven test-only is never started when its reset deletes data; it produces one question. +- A state-changing scenario without a verified teardown is rejected with its reason. +- No login discovery, secret, or guessed live record appears in evidence. +- Preparation that changed state is torn down and verified before the run is ready. diff --git a/plugins/aidd-qa/skills/01-acceptance-qa/actions/04-run-scenarios.md b/plugins/aidd-qa/skills/01-acceptance-qa/actions/04-run-scenarios.md new file mode 100644 index 000000000..99867f18d --- /dev/null +++ b/plugins/aidd-qa/skills/01-acceptance-qa/actions/04-run-scenarios.md @@ -0,0 +1,31 @@ +# 04 - Run Scenarios + +Record, verify, and report one clean take per scenario. + +## Input + +The prepared run and its rejections; from load-scope, the source label, candidate, evidence folder, Out of interface, and Rejected. + +## Output + +`/qa.md` and one final WebM per scenario that ran. + +## Process + +1. **Group.** Steps 1-7 cover executable scenarios only. At most two read-only ones concurrently, in isolated sessions; state-changing ones sequentially. +2. **Record.** Apply setup, then follow [interface-browser-playwright-cli.md](../references/interface-browser-playwright-cli.md). +3. **Verdict.** From each step's `expected`, `actual`, `ok`: any `ok: false` => `fail`, else `pass`. Keep the take of every `fail`. +4. **Recover.** A setup failure, throw, or non-zero exit is operational: discard the take, reset, retry once; a second one => `blocked`. +5. **Reset.** After each state-changing take, run its verified teardown, verify the baseline, then close the session. +6. **Normalize.** At most two independent raw files concurrently. Save only `qa/happy-path.webm` and `qa/edge-case-.webm` once `ffprobe` and frame inspection pass; record each final file's duration from `ffprobe`. +7. **Clean.** Delete raw takes and validation frames only after every final file passes codec, dimension, duration, path, cut-point, and frame checks. Never retain screenshots or alternate media. +8. **Report.** Fill [qa-report-template.md](../assets/qa-report-template.md), even when every scenario was rejected: one row per scenario (evidence `none` when `blocked`), Out of interface and Rejected (`none` when empty), Findings only for `fail` or `blocked`. Run verdict, in order: any `fail` => `fail`; any `blocked` or rejected => `blocked`; else `pass`. `skipped` belongs to load-scope only. +9. **Return.** Output the verdict and evidence paths; when `qa/happy-path.webm` exists, ask `Open happy-path.webm in the browser for review?` and open it when confirmed. + +## Test + +- Each row names its criteria, expected, actual, verdict (`pass`, `fail`, or `blocked`), duration, and evidence. +- Run verdict: `fail` over `blocked` or rejected over `pass`; a `pass` is never reported without the Out of interface list. +- A product mismatch yields `fail` from the returned results, never a thrown take; a second operational failure yields `blocked`. +- Raw takes and frames survive until every final file passes; then only `qa/happy-path.webm` and `qa/edge-case-.webm` remain beside `qa.md`. +- Every earlier rejection appears in the report with its reason, even when it empties the executable set, and the report still runs. diff --git a/plugins/aidd-qa/skills/01-acceptance-qa/assets/qa-report-template.md b/plugins/aidd-qa/skills/01-acceptance-qa/assets/qa-report-template.md new file mode 100644 index 000000000..a36355ceb --- /dev/null +++ b/plugins/aidd-qa/skills/01-acceptance-qa/assets/qa-report-template.md @@ -0,0 +1,21 @@ +# Acceptance QA: {{feature}} + +- **Verdict**: {{pass | fail | blocked | skipped}} +- **Source**: {{acceptance criteria path}} +- **Candidate**: {{branch | commit | running URL}} +- **Run**: {{yyyy_mm_dd}} + +## Out of interface + +{{criteria, or quoted parts, with no browser-observable outcome, or "none"}} + +## Scenarios + +| Scenario | Criteria | Expected | Actual | Verdict | Duration | Evidence | +| -------- | --------- | -------- | ------ | ------- | -------- | -------- | + +## Rejected + +{{scenarios rejected in load-scope or prepare-run, criterion โ€” reason, or "none"}} + +{{findings-section-when-needed}} diff --git a/plugins/aidd-dev/skills/11-browser-qa/references/run-scope-playwright-cli.md b/plugins/aidd-qa/skills/01-acceptance-qa/references/interface-browser-playwright-cli.md similarity index 79% rename from plugins/aidd-dev/skills/11-browser-qa/references/run-scope-playwright-cli.md rename to plugins/aidd-qa/skills/01-acceptance-qa/references/interface-browser-playwright-cli.md index 1eb2f5ba3..0f4535309 100644 --- a/plugins/aidd-dev/skills/11-browser-qa/references/run-scope-playwright-cli.md +++ b/plugins/aidd-qa/skills/01-acceptance-qa/references/interface-browser-playwright-cli.md @@ -10,7 +10,9 @@ Support `@playwright/cli >= 0.1.17`; execute the framework pin `@playwright/cli@ npx --yes @playwright/cli@0.1.17 -s=qa- ``` -`run-code` takes one `page` argument: pass `async page => { ... }`, never bare statements. Keep stdout, stderr, and exit status visible; no redirects, pipes, command substitutions, or `|| true`. A `SyntaxError` or non-zero exit invalidates the take. +Run every command from a temporary directory outside the application repository. + +`run-code` takes one `page` argument: pass `async page => { ... }`, never bare statements. Never throw on a product mismatch: check each expected outcome with a bounded `waitFor`, and return `{ step, expected, actual, ok }` per step. Keep stdout, stderr, and exit status visible; no redirects, pipes, command substitutions, or `|| true` in runner commands. A throw, `SyntaxError`, or non-zero exit is a tooling failure and invalidates the take. ## Recording @@ -33,14 +35,17 @@ npx --yes @playwright/cli@0.1.17 -s=qa-- run-code 'async await page.mouse.wheel(0, 600); await pause(300); await page.getByRole("button", { name: "final action" }).click(); - await page.getByText("observable expected outcome").waitFor({ state: "visible" }); + const expected = "observable expected outcome"; + const ok = await page.getByText(expected).waitFor({ state: "visible", timeout: 5000 }).then(() => true, () => false); await pause(1000); + return [{ step: "final action", expected, actual: ok ? expected : "not visible", ok }]; }' npx --yes @playwright/cli@0.1.17 -s=qa-- video-stop +# Run the teardown and verify the baseline before closing. npx --yes @playwright/cli@0.1.17 -s=qa-- close ``` -`video-stop` writes to `.playwright-cli/` in the current directory. Name raw files `raw-happy-path.webm` or `raw-edge-case-.webm`. +`video-stop` writes the raw file to the current directory. Name raw files `raw-happy-path.webm` or `raw-edge-case-.webm`. ## Duration diff --git a/release-please-config.json b/release-please-config.json index 6d2808f58..7d7526731 100644 --- a/release-please-config.json +++ b/release-please-config.json @@ -104,6 +104,17 @@ } ] }, + "plugins/aidd-qa": { + "package-name": "aidd-qa", + "release-as": "1.0.0", + "extra-files": [ + { + "type": "json", + "path": ".claude-plugin/plugin.json", + "jsonpath": "$.version" + } + ] + }, "cli": { "release-type": "node", "package-name": "@ai-driven-dev/cli"