diff --git a/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/backlog-link.json b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/backlog-link.json new file mode 100644 index 000000000..fc797c5b8 --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/backlog-link.json @@ -0,0 +1,5 @@ +{ + "backlog": "ai-driven-dev/framework#626", + "written_at": "2026-10-07T12:00:00Z", + "written_by": "aidd-dev:01-plan" +} diff --git a/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/journey.md b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/journey.md new file mode 100644 index 000000000..b8adb85ee --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/journey.md @@ -0,0 +1,138 @@ +# Journey evidence + +Nothing here evaluates the validator, and no gate reads prose, so each case is run as fresh +`claude -p` contexts given the skill text and the task, outside the repository. Criteria +fixed before running, no run repeated, every count below mechanical. + +## Placement - pass + +22 drafts, three requests, both text versions. Markers outside the open questions section: +**0**. Heading exact: **22/22**, though only the template names it. + +## Driving to zero - not reached, and nothing now refuses a spec for it + +With the text pasted into one prompt, a request answering every required field produced 0, 0, +0 entries against 3, 2, 0 before the cut. Through the installed tree it produces 3, 2, 2. The +installed figure is the one to trust: it is the route a user runs. + +**No causal claim is made about either pair.** The cut changed several clauses at once, so +length is isolated in neither table, and the thin-request ranges overlap (2, 3, 2, 2, 4 +against 3, 4, 3, 4, 5). + +Because zero is not reached, no refusal rides on it. `open_questions_unresolved` is gone. +What invalidates a spec is a gap written outside the section, which is what #626 measured. + +## The measured case - pass + +Retention and the row cap left open, the two #626 measured: **2, 2, 2** entries, both versions. + +## Refine - pass + +| Finding | Result | +| --- | --- | +| answers one of two | the entry is gone, the other stays, 3/3 | +| answers one in part | narrowed to what remains, 3/3 | +| answers the last | `None`, 3/3 | +| spec carries no such section | the section is added, heading exact, 3/3 | + +The third row failed before this round: cutting `None` from step 4 produced an empty section +3/3. The fourth is an observation, not a mechanism. Its before-and-after prompts differ by +more than the text, since the later one inlines the template, so the link `02-refine.md` +gained was never exercised as a link. + +## The validator - pass + +Fresh context, the yml and one spec. Stands in for the gate #625 brings. + +| Spec | Verdict | +| --- | --- | +| one gap listed in its own section | valid 3/3 | +| `None`, with a gap written under `## Context` | invalid 3/3 | +| no open questions section at all | invalid 3/3 | +| `None`, every required section resolved | valid 3/3, and 3/3 again on a fuller spec | + +The first row read invalid 3/3 while a hard threshold refused an open gap. It refused +complete work, so it is gone. + +The fourth row failed while the yml carried no definition of a gap: an evaluator reading it +alone supplied its own, and refused a spec whose content was complete. + +## The installed tree, read from disk - placement passes, zero does not + +Eight sessions in eight sandboxes, each built by `aidd translate . --to claude`, each reading +`SKILL.md` and following it to the action and to whatever the action links. Real reads, real +routing, the branch's own files. + +| Measure | Result | +| --- | --- | +| the section exists | 8/8 | +| markers outside it | 0/8 | +| entries, thin request | 7, 7, 5, 5, 6 | +| entries, request fully answered | 3, 2, 2 | +| `None` on a fully answered request | **0/3** | + +Placement holds where it counts. Zero does not: a request answering every required field +still produces entries, and `open_questions_unresolved: invalid` then refuses the spec. Two +questions recur, the column set 3/3 and when the seven days start 3/3, and the first is +presentation detail no required section depends on. + +The pasted-text runs above measured 2, 3, 2, 2, 4 entries and `None` 3/3 on the same +requests. They were optimistic. The router reads more files, and asks more. + +### Why the earlier installed-skill attempt measured nothing + +`aidd-pm:04-spec` resolves to the user-scope install, `~/.claude/plugins/cache/aidd-framework/aidd-pm/2.5.0`, +whatever a project declares. Those runs drafted against the published skill and reproduced +#626 exactly: no section, seven to ten markers inline. A project marketplace does not +register in a headless session either, which `Unknown skill: spec-sandbox:04-spec` says +outright. So the Skill tool's own registration is still unmeasured, and only it is. + +## Not measured + +The Skill tool's registration, which no sandbox can exercise while a user-scope install +shadows it. One model. And no judge classified the entries, so which gaps a draft raises +rests on reading them. + +## After the last cut - pass + +The whole skill, cut to what changes a draft: 1379 words to 1050, with a section and a +reference added. Four sessions through the installed tree and six validator verdicts: + +| Measure | Result | +| --- | --- | +| every required section present | 4/4 | +| the open questions section exists | 4/4 | +| markers outside it | 0/4 | +| a gap listed in its own section | valid 2/2 | +| a gap written under `## Context` | invalid 2/2 | +| `None` | valid 2/2 | + +Entries stay high on a thin request, 8, 7 and 6, and 3 on a fully answered one. The cut +neither fixed nor worsened that; nothing refuses a spec for it. + +## The two paths nothing had exercised - pass + +Through the tree, after the declaration moved to a reference and its shape to an asset. + +| Case | Result | +| --- | --- | +| the request names a ticket | `backlog-link.json` written, all three fields, `written_by` the skill | +| it names none | no file | +| the file is already there, for another item | left untouched, the hand-written one survives | +| refine resolves one gap of two | the entry is gone, the other stays, 2/2 | +| refine resolves the last | `None` | + +The declaration had only ever been checked against the text pasted in one prompt, before it +moved. Refine had never been run through the tree at all. + +## The PRD path, the optional headings, a second model - pass + +| Case | Runs | Result | +| --- | --- | --- | +| drafted from a PRD | 3 | every required section present, the PRD's own open question carried as the one gap, nothing outside | +| `(optional)` reaching a drafted spec | 6 | 0, since the template now says it in the guidance and not in the heading | +| a second model | 3 | every required section present, the section exists, nothing outside | + +The second model lists 6, 7 and 7 entries on the thin request where the first lists 6 to 8. +So the over-asking is not one model's habit. Nothing refuses a spec for it. + diff --git a/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/phase-1.md b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/phase-1.md new file mode 100644 index 000000000..fc43aa6c5 --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/phase-1.md @@ -0,0 +1,89 @@ +--- +status: implemented +--- + +# Instruction: the marker has a home +## Architecture projection +> Tree of the final files. ✅ create · ✏️ modify · ❌ delete + +```txt +. +└── plugins/aidd-pm/skills/04-spec/ + ├── references/tbd-marker.md ✏️ the marker, its home, one entry per gap + ├── assets/spec-template.md ✏️ an `## Open Questions` section + ├── assets/spec-validator.yml ✏️ a residual question is invalid + └── actions/ + ├── 01-build.md ✏️ gaps are listed, never inlined; one test row + └── 02-refine.md ✏️ the same for a field still unanswered; one test row +``` +## User Journey +```mermaid +flowchart TD + A[a request leaves a decision pending] --> B[04-spec drafts the spec] + B --> C[the gap is listed under Open Questions, once] + C --> D{any question left?} + D -- yes --> E[the spec is invalid: the decision comes back to the user] + D -- no --> F[Open Questions reads None and the spec validates] +``` +## Test Scope +```mermaid +--- +title: Test scope +--- +journey + section Setup + draft a spec from a request that leaves one decision pending => a spec carrying one gap: 5: system + section Happy path + read the drafted spec => the gap appears once, under Open Questions: 5: system + read every other section => each carries resolved content only, Context and Stakeholders included: 5: system + draft the same spec twice => the gap lands in the same section both times: 5: system + section Edge case - no gap at all + every field is answered => draft the spec => Open Questions reads None and the spec validates: 1: system + section Edge case - a question survives validation + a gap is still listed => validate the spec => the validator calls it invalid: 1: system +``` +## Tasks to do +### `1)` state the marker's home +> `tbd-marker.md` carries the form, the home, and the one-entry rule. + +1. Keep the form `TBD: `. +2. Say the home is the template's open questions section, without naming the heading. +3. Define the unit: a gap is an unresolved decision a required section depends on. +4. Say each gap is one entry there, and that no other section carries a marker. + +### `2)` give the template the section +> A spec has a place to put a pending decision. + +1. Add `## Open Questions` to `spec-template.md`, after `Done-when`. +2. Guide it: one entry per unresolved decision, the marker's form, `None` when there is nothing left. +3. Say it drives to zero before the spec validates. + +### `3)` make a residual question invalid +> The validator refuses a spec that still asks something. + +1. Add `open_questions_unresolved: invalid` to `hard_thresholds`. +2. Leave every weight and `pass_threshold` untouched. +3. Add the `open_questions` criterion as `required: true`, `weight: 0`, describing the list's shape: required makes an absent or malformed section invalid, and a zero weight keeps the weighted total at 100. + +### `4)` route both actions to the home +> Neither action places a marker by judgement any more. + +1. `01-build.md` step 3: list every gap per `tbd-marker.md`. +2. `02-refine.md` step 4: the same, and remove an entry only once its decision is fully answered. +3. Add one test row to each: a gap is listed once, in the open questions section only. + +### `5)` prove it +> The change holds under the repository's own gates. + +1. Read every edited file back. +2. Run `pnpm exec lefthook run pre-commit`. +3. Commit through the commit skill. + +## Test acceptance criteria +| Task | Acceptance criteria | +| --- | --- | +| 1 | `tbd-marker.md` names the template's open questions section as the home, and allows one entry per gap | +| 2 | `spec-template.md` carries the section with its guidance, and `None` is the empty form | +| 3 | `spec-validator.yml` calls a spec with an unresolved question invalid, and no weight or threshold moved | +| 4 | each action's gap step points at the home, and each `## Test` table has the row for it | +| 5 | pre-commit is green and the work is committed | diff --git a/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/phase-2.md b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/phase-2.md new file mode 100644 index 000000000..50f27d526 --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/phase-2.md @@ -0,0 +1,77 @@ +--- +status: implemented +--- + +# Instruction: the rules are cut to what changes the draft +## Architecture projection +> Tree of the final files. ✅ create · ✏️ modify · ❌ delete + +```txt +. +└── plugins/aidd-pm/skills/04-spec/ + ├── references/tbd-marker.md ✏️ the unit and the home, nothing else + ├── assets/spec-template.md ✏️ the section owns its own heading + ├── assets/spec-validator.yml ✏️ placement refuses, an open gap does not + └── actions/ + ├── 01-build.md ✏️ the steps lose their restated prose + └── 02-refine.md ✏️ the gap step says each thing once +``` +## User Journey +```mermaid +flowchart TD + A[a rule is written in five files] --> B[the template owns the heading] + B --> C[the actions point at the reference] + C --> D{does a rule still bind?} + D -- no --> E[it is restored and measured] + D -- yes --> F[the words go] +``` +## Test Scope +```mermaid +--- +title: Test scope +--- +journey + section Setup + build the installed tree from this branch => a sandbox per run: 5: system + section Happy path + draft from the shortened text => every gap sits in the open questions section: 5: system + resolve the last gap through refine => the section reads None: 5: system + validate a spec whose gap sits elsewhere => invalid: 5: system + section Edge case - a complete request + every required field answered => draft the spec => no refusal rides on what it still asks: 1: system +``` +## Tasks to do +### `1)` take the heading out of four files +> The template owns its own section name. + +1. Name the section in `spec-template.md` and nowhere else. +2. Point the reference, both actions and the validator at it by description. + +### `2)` cut the prose that changes no draft +> On the user asking for the existing text to be cut too, past this issue. + +1. Drop the restated sub-bullet, the parenthetical the template already carries, and the example verbs. +2. Reduce the backlog declaration to its rules. + +### `3)` restore every rule the cut dropped +> A cut that loses a behaviour is a defect, not a saving. + +1. `None` when no entry remains, in refine's step 4 and its test row. +2. The definition of a gap in the validator, which an evaluator reads alone. +3. Refine links the template, so the heading resolves. +4. What `written_by` holds, and the trigger and one-support rule of the backlog declaration. + +### `4)` take the refusal off an open gap +> Measured: a complete request still lists entries, so the refusal refused complete work. + +1. Replace `open_questions_unresolved` with `open_questions_outside_their_section`. +2. Say in the criterion that a listed gap does not invalidate the spec. +3. Move zero-before-locking to the template's guidance, where nothing enforces it until #625. + +## Test acceptance criteria +| Task | Acceptance criteria | +| --- | --- | +| 1 | only `spec-template.md` names the heading, and the installed tree still places every gap there | +| 2 | no rule is lost: each one still names a behaviour a reader can apply | +| 3 | the four restored clauses each pass three runs out of three | +| 4 | a spec with one open gap is valid, and one with a gap elsewhere is invalid | diff --git a/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/plan.md b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/plan.md new file mode 100644 index 000000000..c4944c9f0 --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/plan.md @@ -0,0 +1,38 @@ +--- +objective: "Every gap a spec still carries is listed once under its own section, so two runs on the same feature place it identically." +status: implemented +--- + +# Plan: a spec declares its open questions +## Overview +| Field | Value | +| --- | --- | +| **Goal** | give the `TBD:` marker one deterministic home in a spec, and make a residual one invalidate the spec | +| **Source** | ai-driven-dev/framework#626 | + +## Phases +| # | Phase | File | +| --- | --- | --- | +| 1 | the marker has a home | [`phase-1.md`](./phase-1.md) | +| 2 | the rules are cut to what changes the draft | [`phase-2.md`](./phase-2.md) | + +## Resources +| Source | Verified | +| --- | --- | +| `plugins/aidd-pm/skills/04-spec/references/tbd-marker.md` | the marker's only definition is its form, `TBD: ` — no home is stated anywhere | +| `plugins/aidd-pm/skills/04-spec/actions/01-build.md` | step 3 replaces "any missing required field" with the marker, which is what leaves placement to the drafting agent | +| `plugins/aidd-pm/skills/04-spec/actions/02-refine.md` | step 4 says the same for a field still unanswered | +| `plugins/aidd-pm/skills/04-spec/SKILL.md` | its transversal rule says to mark every gap and names no home, so it needs no change | +| `plugins/aidd-pm/skills/04-spec/assets/spec-validator.yml` | required criteria weigh 90 of 100, so a new weighted criterion would move `pass_threshold` | +| `cli/tests/golden/` | holds no copy of the spec template, so no snapshot is coupled to this change | + +## Decisions +| Decision | Why | +| --- | --- | +| The home is the template's open questions section, required, and named in no other file | it mirrors `prd-template.md`, and the issue names it | +| Each gap is listed there once, and nowhere else | a gap written in both its eventual section and the list is the same non-determinism with an extra copy | +| Every section other than the open questions one carries resolved content only | a hard constraint that is undecided contradicts the validator's own definition of that section, and a copy under an optional section is the second of the two placements the issue measured | +| One noun, defined in the reference and repeated once in the validator | an evaluator reads the yml alone, measured: without the definition there it supplied its own and refused a complete spec. The one exception to one fact, one home, and the reason it is one | the template said `unresolved decision`, the reference said `gap` and the actions said `missing required field`. Three units in one skill, and the broadest of them now blocks validation | +| What no required section depends on is not a gap | it excludes over-asking without inventing a subjective category: `SKILL.md`'s `Hold intent, never implementation` and the validator's `contains_implementation_details` are boundaries a drafter already applies. Measured insufficient: the installed tree still asks for the column set 3 times out of 3, which is why no refusal rides on it | it excludes over-asking without inventing a subjective category: `SKILL.md`'s `Hold intent, never implementation` and the validator's `contains_implementation_details` are boundaries a drafter already applies | +| A gap written outside the section invalidates the spec through `hard_thresholds`; a gap merely still open does not | the required weights already total 90 against a `pass_threshold` of 90, so a weighted addition would need the threshold retuned; a zero weight checks the section's shape and leaves the arithmetic alone | +| `tbd-marker.md` states the home; the two actions link to it and `SKILL.md` is untouched | one fact, one home: a rule written in three places drifts in two of them. The guard would not have caught it, since `check-doc-duplication.js` scans only `docs`, the two memory banks and the READMEs | diff --git a/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/review.md b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/review.md new file mode 100644 index 000000000..8a675b606 --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/review.md @@ -0,0 +1,32 @@ +# Review: spec-open-questions +- **Verdict**: changes-requested +- **Diff**: `origin/next...HEAD` (`a4d185a5`, `3f785340`, `50d023f2`) +- **Axes run**: code, functional, relevancy +- **Date**: 2026_10_07 +- **Findings**: 0 critical, 4 warning, 2 minor + +## Phases +### Phase 1 — the marker has a home +- [x] `tbd-marker.md` names `## Open Questions` as the home and allows one entry per gap — `plugins/aidd-pm/skills/04-spec/references/tbd-marker.md:5,7` +- [x] `spec-template.md` carries `## Open Questions` with its guidance, and `None` is the empty form — `plugins/aidd-pm/skills/04-spec/assets/spec-template.md:32,34` +- [x] `spec-validator.yml` calls a spec with an unresolved question invalid, and no weight or threshold moved — `plugins/aidd-pm/skills/04-spec/assets/spec-validator.yml:45`; weights 25+20+20+25+0+5+5=100, required 90, `pass_threshold` 90, both edits are insertions +- [x] each action's gap step points at the home, and each `## Test` table has the row for it — `actions/01-build.md:19,44`, `actions/02-refine.md:18,28,29` +- [ ] pre-commit is green and the work is committed — partial: the `plan.md` correction is uncommitted (`M` in the working tree), and only part of pre-commit was replayed (`architecture-rules` 351 governed files no violation, `markdown-links` 0 broken in 815 files, `commitlint HEAD~3..HEAD` exit 0, validator YAML parses; no `skill-frontmatter`, `context-imports` or catalog regeneration check) + +## Findings +| Sev | Kind | Phase | Location | Issue | Fix | +| --- | --- | --- | --- | --- | --- | +| 🟡 | rot | 1 | `plugins/aidd-pm/skills/04-spec/actions/01-build.md:19`, `actions/02-refine.md:18`, `assets/spec-template.md:34`, `assets/spec-validator.yml:26`, `references/tbd-marker.md:7`, `SKILL.md:26` | One entry is defined by three units, and the two actions hold the narrowest. The actions trigger on a missing or unanswered **field**; the template and the validator, both new with this change, on an unresolved **decision**; `tbd-marker.md` and the transversal rule on a **gap**. A drafter reading all five picks the broadest. Measured: a request answering target, role, range, retention and the row cap and stating only no non-goals produced six entries (CSV column order, which date the range filters on, invoice scope, the refusal wording, post-window behaviour, who may download). Two of those are arguable `Hard constraints` ambiguities; column order, refusal wording and the post-window message are optional detail that no required section depends on. Under `open_questions_unresolved: invalid` each one now blocks validation, so the unspecified unit does not merely over-ask, it refuses a spec whose required content is complete. | Name one unit everywhere, and **widen** rather than narrow: "an unresolved decision a required section depends on", with optional detail explicitly not an open question. `01-build.md:19` and `02-refine.md:18` move from "field" to that unit; `spec-template.md:34`, `spec-validator.yml:26` and `tbd-marker.md:7` adopt the same words. Do not narrow to "field": #626's measured case is a pending decision inside a `Hard constraints` section that exists (`draft1.md`, `draft2.md`), which a field-level trigger would stop catching, forcing the drafter to drop or invent the value. | +| 🟡 | fit | 1 | `plugins/aidd-pm/skills/04-spec/assets/spec-validator.yml:25-28,40-45` | Ambiguous criterion, flagged rather than rated. `open_questions` reads "listing each unresolved decision once"; read spec-wide, a `TBD:` under `## Context` while `## Open Questions` reads `None` fails it, read section-only it passes, and nothing in the file says which. No hard threshold covers the case and `context` is `required: false`, so on the section-only reading no criterion fails that spec. `Context` is one of the two placements #626 measured, and #626's Expected behaviour names `spec-validator.yml` as needing the matching update ("TBD's home becomes `## Open Questions`"). The required sections are covered indirectly — a vague constraint fails `hard_constraints` — so only the optional ones leak. | Make the exclusivity explicit instead of leaving it to the evaluator: extend `open_questions`'s description with "and no other section carries a marker", or add `marker_outside_open_questions: invalid` to `hard_thresholds`. | +| 🟡 | functional | 1 | `aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/plan.md:35` | Criterion 5 ("pre-commit is green and the work is committed") is partial. The Decision 5 rationale correction exists only in the working tree; `git show HEAD:…/plan.md` line 35 still reads "`check-doc-duplication.js` fails a sentence written in two documents", which is false for these paths — that script's `SCANNED_DIRECTORIES` are `docs`, `aidd_docs/memory`, `cli/aidd_docs/memory` plus three README globs (`scripts/check-doc-duplication.js:24-26`), none of which reach `plugins/**`. The candidate diff therefore still ships the rationale the previous round rejected. | Commit the `plan.md` correction, then replay the full pre-commit so the catalog and frontmatter gates are covered too. | +| 🟡 | rot | 1 | `aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/plan.md:33`, `phase-1.md:51,72` | `50d023f2` widened the rule to "every other section", but the binding artifact was not followed: Decision 3 still reads "The template's **required** sections carry resolved content only", and `phase-1.md:51,72` still say "no **required** section carries a marker". The shipped text is stricter than the plan that authorised it, so the plan now understates the rule it is the authority for. | Align Decision 3 and `phase-1.md:51,72` with "every other section". | +| 🟢 | fit | 1 | `plugins/aidd-pm/skills/04-spec/actions/02-refine.md:18` | Step 4 removes "every entry the rewrite answered", then lists "each **field** still unanswered". A two-part entry whose first half a finding answered stops being an unanswered field once the section carries content, so nothing re-lists the surviving half: `None` is written and the decision is dropped. Unexercised by the journey — its finding answered both halves of the cap question at once (`refine2.md`). Same root as finding 1. | Scope step 4 to the entry, not the field: re-list every entry a finding did not fully answer, narrowed to what is still open. | +| 🟢 | conform | 1 | `plugins/aidd-pm/skills/04-spec/SKILL.md:26` | #626's Expected behaviour and Affected files both name `SKILL.md`'s Transversal rules; the diff leaves the file untouched. Plan Decision 5 makes the call deliberately and defensibly: the rule names no home, so there is nothing to correct, and one fact belongs in one place. A documented deviation, not a defect — but the issue reads as unmet unless the pull request body says so. | Cite the deviation and its reason in the PR body. | + +## Verification +| Metric | Value | +| --- | --- | +| Verified | 80% (4/5) | +| Files checked | `plugins/aidd-pm/skills/04-spec/SKILL.md`, `actions/01-build.md`, `actions/02-refine.md`, `assets/spec-template.md`, `assets/spec-validator.yml`, `references/tbd-marker.md`, `aidd_docs/tasks/2026_10/2026_10_07_spec-open-questions/plan.md`, `phase-1.md`, `backlog-link.json`, `plugins/aidd-pm/skills/03-prd/assets/prd-template.md`, `plugins/aidd-pm/CATALOG.md`, `docs/CATALOG.md`, `scripts/check-doc-duplication.js`, `cli/tests/golden/snapshots/framework-build/golden.json`, `cli/tests/fixtures/framework-real/plugins/aidd-pm/skills/05-spec/SKILL.md`, 8 journey outputs under the session scratchpad | +| Unchecked | criterion 5 — `fix` | +| Unplanned | `backlog-link.json` is in the diff but not in phase-1's architecture projection; it names `ai-driven-dev/framework#626` with `written_by: "aidd-dev:01-plan"`, which `01-frame.md` authorises. Evidence limits: placement is well evidenced (8 journey outputs, no marker outside `## Open Questions` in any, the same request drafted twice placed both entries in the same section), but `draft_all2.md`'s `None` does not isolate completeness — its request ends with "No other decision is pending.", an explicit suppressor, and it is n=1 against the n=1 that produced six entries. Phase-1's Test Scope case "a gap is still listed → the validator calls it invalid" and the stray-marker case were never executed; two validator runs would settle both (a spec with one residual entry; a spec whose `## Open Questions` reads `None` with a `TBD:` under `## Context`). The frozen fixture `cli/tests/fixtures/framework-real/.../05-spec/SKILL.md:51,53,72` still carries the old placement-free wording and is correctly untouched: it is hashed by `cli/tests/golden/snapshots/framework-build/golden.json`. | diff --git a/cli/tests/contexts/telemetry/infrastructure/task-backlog-skill-shape.integration.test.ts b/cli/tests/contexts/telemetry/infrastructure/task-backlog-skill-shape.integration.test.ts index d0e649d1f..eebc56a12 100644 --- a/cli/tests/contexts/telemetry/infrastructure/task-backlog-skill-shape.integration.test.ts +++ b/cli/tests/contexts/telemetry/infrastructure/task-backlog-skill-shape.integration.test.ts @@ -6,17 +6,16 @@ import { afterEach, describe, expect, it } from "vitest"; import { TaskBacklogAdapter } from "../../../../src/contexts/telemetry/infrastructure/task-backlog-adapter.js"; import { REPOSITORY_ROOT } from "../../../helpers/repository-root.js"; -/** Each skill's own fenced example is fed through the real adapter over a real temp folder, - * never a stand-in parser, so a field renamed on either side fails here. */ +/** Fed through the real adapter, never a stand-in parser, so a rename on either side fails. */ const REPO_ROOT = REPOSITORY_ROOT; -const SPEC_SKILL_MD = join( +const SPEC_EXAMPLE_JSON = join( REPO_ROOT, "plugins", "aidd-pm", "skills", "04-spec", - "actions", - "01-build.md" + "assets", + "backlog-link-template.json" ); const PLAN_SKILL_MD = join( REPO_ROOT, @@ -28,10 +27,10 @@ const PLAN_SKILL_MD = join( "04-plan.md" ); -/** The first fenced json block: the literal example a skill tells an agent to write, - * tolerant of a numbered-list item's own indentation. `null` when none is found. */ -function fencedJsonExample(markdown: string): string | null { - const match = /^[ \t]*```json\r?\n([\s\S]*?)\r?\n[ \t]*```/mu.exec(markdown); +function taughtExample(path: string): string | null { + const text = readFileSync(path, "utf8"); + if (path.endsWith(".json")) return text.trim(); + const match = /^[ \t]*```json\r?\n([\s\S]*?)\r?\n[ \t]*```/mu.exec(text); return match?.[1] ?? null; } @@ -51,18 +50,16 @@ async function projectWithLink(json: string): Promise<{ root: string; taskFolder } describe.each([ - ["aidd-pm:04-spec", SPEC_SKILL_MD], + ["aidd-pm:04-spec", SPEC_EXAMPLE_JSON], ["aidd-dev:01-plan", PLAN_SKILL_MD], ])("%s's own backlog-link.json example matches what the reader accepts", (_skill, path) => { - it("names a fenced JSON example at all (guards against a no-op extraction)", () => { - const markdown = readFileSync(path, "utf8"); - expect(fencedJsonExample(markdown)).not.toBeNull(); + it("names a JSON example at all (guards against a no-op extraction)", () => { + expect(taughtExample(path)).not.toBeNull(); }); it("parses through the real TaskBacklogAdapter as a declared item", async () => { - const markdown = readFileSync(path, "utf8"); - const example = fencedJsonExample(markdown); - if (example === null) throw new Error("no fenced json example to test"); + const example = taughtExample(path); + if (example === null) throw new Error("no json example to test"); const { root, taskFolder } = await projectWithLink(`${example}\n`); const adapter = new TaskBacklogAdapter(root); @@ -80,8 +77,8 @@ describe.each([ describe("both skills agree with each other, not only with the reader", () => { it("write the identical field names, so neither can drift from the other unnoticed", () => { - const specExample = fencedJsonExample(readFileSync(SPEC_SKILL_MD, "utf8")); - const planExample = fencedJsonExample(readFileSync(PLAN_SKILL_MD, "utf8")); + const specExample = taughtExample(SPEC_EXAMPLE_JSON); + const planExample = taughtExample(PLAN_SKILL_MD); expect(specExample).not.toBeNull(); expect(planExample).not.toBeNull(); diff --git a/plugins/aidd-pm/CATALOG.md b/plugins/aidd-pm/CATALOG.md index 027f408b7..8c2029194 100644 --- a/plugins/aidd-pm/CATALOG.md +++ b/plugins/aidd-pm/CATALOG.md @@ -76,7 +76,9 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai |-------|------|---| | `actions` | [01-build.md](skills/04-spec/actions/01-build.md) | - | | `actions` | [02-refine.md](skills/04-spec/actions/02-refine.md) | - | +| `assets` | [backlog-link-template.json](skills/04-spec/assets/backlog-link-template.json) | - | | `assets` | [spec-template.md](skills/04-spec/assets/spec-template.md) | - | +| `references` | [backlog-link.md](skills/04-spec/references/backlog-link.md) | - | | `references` | [tbd-marker.md](skills/04-spec/references/tbd-marker.md) | - | | `-` | [SKILL.md](skills/04-spec/SKILL.md) | `Generate or refine a spec, a feature's immutable contract, from a request, a PRD, or review findings. Use when the user wants to draft or refine a spec. Not for writing code, a full PRD, or changing a locked spec.` | diff --git a/plugins/aidd-pm/skills/04-spec/SKILL.md b/plugins/aidd-pm/skills/04-spec/SKILL.md index 10afd34c0..ec9127be2 100644 --- a/plugins/aidd-pm/skills/04-spec/SKILL.md +++ b/plugins/aidd-pm/skills/04-spec/SKILL.md @@ -26,5 +26,4 @@ Run the flow above. Read only the next action file. - Never invent; mark every gap instead of guessing. - Never explore the codebase. - Hold intent, never implementation: solution-agnostic, no how, few acceptance criteria. -- Keep it readable: clear headers, bulleted criteria, explicit non-goals. - Immutable once validated: never rewrite a locked spec. diff --git a/plugins/aidd-pm/skills/04-spec/actions/01-build.md b/plugins/aidd-pm/skills/04-spec/actions/01-build.md index 7ff9affa9..e12ef7caf 100644 --- a/plugins/aidd-pm/skills/04-spec/actions/01-build.md +++ b/plugins/aidd-pm/skills/04-spec/actions/01-build.md @@ -4,44 +4,28 @@ Draft a fresh spec from a free-form request, or by lifting fields from an existi ## Input -A free-form request, or a path to an existing PRD. A feature name for the folder, derived from the request when absent. +A free-form request, or a path to a PRD. ## Output -The path to `spec.md` in the feature folder, drafted from the template, with the ambiguities and assumptions noted, or no write when the request is too vague. +The path to `spec.md`, or no write. ## Process -1. **Qualify.** When the request is too vague to draft anything useful, stop and ask for a clearer one. -2. **Source.** Map the input onto [spec-template.md](../assets/spec-template.md), dropping any implementation detail. - - PRD path: lift its target, hard constraints, non-goals, and done-when. - - Free-form request: map it directly onto the template sections. -3. **Gaps.** Replace any missing required field per [tbd-marker.md](../references/tbd-marker.md). -4. **Check.** Confirm every section the validator requires is present. Omit an optional section (stakeholders, context) that has nothing to say rather than emit a placeholder. -5. **Write.** Resolve the feature folder: reuse an existing `aidd_docs/tasks//_/` match for this feature, or create one. Save it there. -6. **Declare.** When the request names a backlog item — a ticket already resolved (e.g. by `/aidd-pm:01-ticket-info`), or a path to its Markdown artefact — and the folder carries no `backlog-link.json` yet, write one there, naming that item on whatever support it lives: - - ```json - { - "backlog": "owner/repo#123", - "written_at": "2026-08-21T09:00:00Z", - "written_by": "aidd-pm:04-spec" - } - ``` - - `backlog` is the one field: a forge reference (`"owner/repo#123"`) or a project-relative Markdown path (`"aidd_docs/backlog/tasks/x.md"`), never both and never a second field for the other support. `written_at` is now, in ISO 8601 UTC; `written_by` is this skill's own name, verbatim. - - When the request names no backlog item, write nothing — a folder with no declaration is a normal state, never an error. When `backlog-link.json` already exists, leave it untouched: it is correctable by hand, and a later run must never overwrite a correction. -7. **Return.** Surface its path and the notes. +1. **Qualify.** Stop and ask for a clearer request when this one is too vague to draft. +2. **Source.** Map the input onto [spec-template.md](../assets/spec-template.md). From a PRD, lift its target, hard constraints, non-goals and done-when. +3. **Gaps.** List every gap per [tbd-marker.md](../references/tbd-marker.md). +4. **Check.** No required section missing. Omit an optional one with nothing to say, never a placeholder. +5. **Write.** Save it in `aidd_docs/tasks//_/`, reusing this feature's folder when one exists. +6. **Declare.** Declare the request's backlog item per [backlog-link.md](../references/backlog-link.md). +7. **Return.** Its path and its gaps. ## Test | Case | Pass | | --- | --- | -| The action completes | `spec.md` exists in the feature folder | -| The file is validated | every section required by [spec-validator.yml](../assets/spec-validator.yml) is present | +| The action completes | `spec.md` exists, with every section [spec-template.md](../assets/spec-template.md) requires | | The spec is read back | it carries no library name, framework pattern, or source-file layout | +| A gap exists | it is listed once, in the open questions section only | | Too vague | no write; one clarifying question returned | -| The request names a backlog item | the folder's `backlog-link.json` names it, with `written_at` and `written_by` | -| The request names none | no `backlog-link.json` is written, and nothing errors | -| `backlog-link.json` already exists | it is left unchanged, even when the request names a different item | +| A backlog item is named, or none is, or the file is already there | each handled per [backlog-link.md](../references/backlog-link.md) | diff --git a/plugins/aidd-pm/skills/04-spec/actions/02-refine.md b/plugins/aidd-pm/skills/04-spec/actions/02-refine.md index cf4e7f6f3..860bb0c62 100644 --- a/plugins/aidd-pm/skills/04-spec/actions/02-refine.md +++ b/plugins/aidd-pm/skills/04-spec/actions/02-refine.md @@ -4,27 +4,27 @@ Rewrite an existing spec in place to address review findings. ## Input -The path to the current spec, and the findings to address, a list or free text. +The spec's path, and the findings. ## Output -The refined spec at the same path, or no change if not written. +The refined spec at the same path, or no write. ## Process -1. **Load.** Read the spec and the findings. -2. **Map.** Pair each finding with the section it touches. -3. **Rewrite.** Apply each finding in place: clarify wording, add missing fields, remove invalid claims. Leave untouched sections as they are. -4. **Gaps.** Replace any field still unanswered per [tbd-marker.md](../references/tbd-marker.md). -5. **Check.** Confirm every section the validator requires is present, then overwrite the spec at its path. -6. **Verify.** Report what changed as `before -> after`, and the verification result. +1. **Map.** Pair each finding with the section it touches. +2. **Rewrite.** Apply each finding in place. Leave every other section as it is. +3. **Gaps.** Drop a resolved entry, narrow a half-answered one, list the rest per [tbd-marker.md](../references/tbd-marker.md). `None` when none remains. +4. **Check.** No required section missing, then overwrite the spec. +5. **Verify.** Report each change as `before -> after`, with the result. ## Test | Case | Pass | | --- | --- | -| The action completes | the spec still exists at its path with every section required by [spec-validator.yml](../assets/spec-validator.yml) | +| The action completes | the spec is still at its path, with every section [spec-template.md](../assets/spec-template.md) requires | | A finding is resolved | the spec changed at the section it names | -| A finding cannot be resolved | the marker from [tbd-marker.md](../references/tbd-marker.md) marks it in place | -| A write happened | the result reports the stable identity, `before -> after` fields, and verification result | -| No write happened | the result states that no persisted change occurred | +| A finding resolves a gap | its entry is gone, and the section reads `None` when it was the last | +| A finding cannot be resolved | it is listed once, in the open questions section only | +| A write happened | the result reports the path and each `before -> after` | +| No write | the result says nothing was written | diff --git a/plugins/aidd-pm/skills/04-spec/assets/backlog-link-template.json b/plugins/aidd-pm/skills/04-spec/assets/backlog-link-template.json new file mode 100644 index 000000000..3fd848d37 --- /dev/null +++ b/plugins/aidd-pm/skills/04-spec/assets/backlog-link-template.json @@ -0,0 +1,5 @@ +{ + "backlog": "owner/repo#123", + "written_at": "2026-08-21T09:00:00Z", + "written_by": "aidd-pm:04-spec" +} diff --git a/plugins/aidd-pm/skills/04-spec/assets/spec-template.md b/plugins/aidd-pm/skills/04-spec/assets/spec-template.md index c669e7551..88a0ae38e 100644 --- a/plugins/aidd-pm/skills/04-spec/assets/spec-template.md +++ b/plugins/aidd-pm/skills/04-spec/assets/spec-template.md @@ -4,37 +4,38 @@ ## Target - + > Example: "Allow authenticated users to export their account data as a downloadable JSON file." ## Hard constraints - + -- -- +- ## Non-goals - + -- -- +- ## Done-when - + -- -- +- -## Stakeholders (optional) +## Open Questions -- Decider: -- Owner: -- Consumer: + -## Context (optional) +## Stakeholders + - +- Decider: +- Owner: +- Consumer: + +## Context + diff --git a/plugins/aidd-pm/skills/04-spec/assets/spec-validator.yml b/plugins/aidd-pm/skills/04-spec/assets/spec-validator.yml index 5b1f55d8f..2cb016ef2 100644 --- a/plugins/aidd-pm/skills/04-spec/assets/spec-validator.yml +++ b/plugins/aidd-pm/skills/04-spec/assets/spec-validator.yml @@ -1,34 +1,38 @@ -# Checklist a spec is validated against. -# Each criterion is independently scored. Required criteria must be `fulfilled` for the spec to pass. +# Checklist a spec is validated against. Each criterion is scored on its own. criteria: - id: target - description: A single clear sentence stating what the system must achieve. Unambiguous, verifiable, single target (not multiple disguised as one). + description: One sentence, unambiguous and verifiable, naming a single target. required: true weight: 25 - id: hard_constraints - description: Non-negotiable constraints listed explicitly. Each constraint is concrete and testable, not vague. + description: Non-negotiable constraints, each concrete and testable. required: true weight: 20 - id: non_goals - description: Out-of-scope items listed explicitly. Reader knows what NOT to build. + description: Out-of-scope items listed, so a reader knows what not to build. required: true weight: 20 - id: done_when - description: Completion conditions listed, each an observable outcome a user or the system can be seen producing. Verifiable by observation, not a command and not a feeling. + description: Completion conditions, each an observable outcome, never a command. required: true weight: 25 + - id: open_questions + description: Every gap listed once in the open questions section as a precise question, or None. A gap is an unresolved decision a required section depends on. A listed gap is valid; a marker written in any other section is not. + required: true + weight: 0 + - id: stakeholders - description: Decider, owner, and consumer roles identified. Optional but encouraged for non-trivial specs. + description: Decider, owner and consumer named. required: false weight: 5 - id: context - description: Relevant background or links provided when needed for downstream decisions. + description: Background or links a downstream decision needs. required: false weight: 5 @@ -37,10 +41,8 @@ hard_thresholds: target_unclear_or_multiple_targets: invalid done_when_unverifiable: invalid contains_implementation_details: invalid + open_questions_outside_their_section: invalid scoring: pass_threshold: 90 - notes: | - A spec is valid when every required criterion is fulfilled AND the weighted score - is at or above the pass_threshold. Hard thresholds force `invalid` regardless of score. - Implementation details (libraries, patterns, file layout) belong to the plan, not the spec. + notes: Valid when every required criterion is fulfilled and the weighted score reaches pass_threshold. A hard threshold forces invalid whatever the score. diff --git a/plugins/aidd-pm/skills/04-spec/references/backlog-link.md b/plugins/aidd-pm/skills/04-spec/references/backlog-link.md new file mode 100644 index 000000000..24b966ffa --- /dev/null +++ b/plugins/aidd-pm/skills/04-spec/references/backlog-link.md @@ -0,0 +1,12 @@ +# Backlog link + +A backlog item is a ticket already resolved, or a path to its Markdown artefact. Never raw text. + +Declare it in the feature folder as `backlog-link.json`, from +[backlog-link-template.json](../assets/backlog-link-template.json). + +`backlog` is the one field: a forge reference or a project-relative Markdown path, never both. +`written_at` is now, ISO 8601 UTC. `written_by` is the skill that wrote it. + +No item named, no file: an undeclared folder is normal. +Never overwrite one, so a correction by hand survives. diff --git a/plugins/aidd-pm/skills/04-spec/references/tbd-marker.md b/plugins/aidd-pm/skills/04-spec/references/tbd-marker.md index 9cbcbda63..2104a7eda 100644 --- a/plugins/aidd-pm/skills/04-spec/references/tbd-marker.md +++ b/plugins/aidd-pm/skills/04-spec/references/tbd-marker.md @@ -1,3 +1,7 @@ # TBD marker `TBD: ` + +A gap is an unresolved decision a required section depends on. Nothing else is one. + +Every gap goes to the template's open questions section, once, and to no other section. diff --git a/scripts/__tests__/a-backlog-link-the-reader-can-read.test.js b/scripts/__tests__/a-backlog-link-the-reader-can-read.test.js index a883c622b..5e388cbed 100644 --- a/scripts/__tests__/a-backlog-link-the-reader-can-read.test.js +++ b/scripts/__tests__/a-backlog-link-the-reader-can-read.test.js @@ -57,20 +57,20 @@ describe("every backlog declaration in this repository is one the report can rea }); /** The two skills that teach the file are the only things that decide what gets written, - * so they have to teach the same three names — and the same three this test asks for. + * so they have to teach the same three names, wherever inside each skill the lesson lives — and the same three this test asks for. * Read from the skills rather than trusted: a taught shape drifting away from the reader * is exactly what produced the two unreadable files, and a guard restating the fields * without checking the lesson would have stayed green through it. */ it("is the shape both skills that write it actually teach", () => { - const TEACHING_ACTIONS = [ - "plugins/aidd-pm/skills/04-spec/actions/01-build.md", + const TEACHING_FILES = [ + "plugins/aidd-pm/skills/04-spec/assets/backlog-link-template.json", "plugins/aidd-dev/skills/01-plan/actions/04-plan.md", ]; - for (const action of TEACHING_ACTIONS) { - const text = fs.readFileSync(path.join(ROOT, action), "utf8"); + for (const file of TEACHING_FILES) { + const text = fs.readFileSync(path.join(ROOT, file), "utf8"); for (const field of REQUIRED_FIELDS) { - assert.ok(text.includes(`"${field}"`), `${action} must teach the field "${field}"`); + assert.ok(text.includes(`"${field}"`), `${file} must teach the field "${field}"`); } } });