docs: add sample quality and model E2E gates

.github/skills/add-feature/SKILL.md

@@ -39,6 +39,18 @@ bug fixes.
    (discover → translate → register → execute) and name the modules touched.
 4. Seed the **Decisions log** with the initial choices and who made them
    (Human vs Agent).
+5. If the feature adds or modifies a runnable sample, complete the
+   **Sample / user journey** section before architecture review:
+   - identify the target learner and the one primary capability the sample teaches;
+   - write a user-centered story, including the natural-language prompt when the
+     sample starts from one, and the observable terminal result;
+   - record an explicit FRD decision to create a dedicated sample or extend an
+     existing one, with the rationale and migration/testing impact;
+   - define the exact setup, run, test, and cleanup commands, the concise evidence
+     the PR will include, and any existing samples affected when code is extracted.
+6. Treat the completed sample design as a gate. Do not advance a sample-bearing
+   feature to architecture review while the learner, capability, journey,
+   dedicated-vs-existing decision, or validation boundary remains ambiguous.
 
 ## Phase 2 — Architecture review (planning mode)  *(gate: human sign-off → `status: Finalized`)*
 
@@ -49,7 +61,7 @@ bug fixes.
      (discovery read-only; registration is the only Azure-aware stage; lazy runner);
    - whether the public surface stays consistent with `docs/front-matter-spec.md`.
 2. Iterate on the FRD until findings are resolved. Append decisions to the log.
-3. Get explicit human sign-off, record it in §8, and set `status: Finalized`.
+3. Get explicit human sign-off, record it in §9, and set `status: Finalized`.
    **Do not implement before the FRD is Finalized.**
 
 ## Phase 3 — Implementation  *(gate: `ruff` + `mypy` clean)*
@@ -71,7 +83,39 @@ bug fixes.
 2. Add tests under `tests/`, mirroring source modules. For config/authoring
    changes, add a scenario folder under `tests/fixtures/config_scenarios/`.
 3. For bug-adjacent work, add a failing regression test first.
-4. Run the full CI-equivalent gate:
+4. If the feature adds or modifies a runnable sample, run an independent sample
+   review (a reviewer other than the sample author, preferably a review sub-agent)
+   and resolve findings about:
+   - whether the sample tells the FRD's user-centered story and keeps one primary
+     capability;
+   - whether its documented commands produce the expected observable terminal
+     result from a clean setup;
+   - whether shared-code extraction or other changes preserve coverage for every
+     affected existing sample.
+5. For behavior where an LLM generates structured execution inputs such as tool
+   arguments, dependency graphs, or configuration, include an opt-in real-model
+   E2E that crosses the complete boundary:
+   **natural-language prompt → model → tool call → parser/runtime → observable
+   terminal result**. Unit or integration tests that inject model output or call
+   the parser/runtime directly cannot substitute for this E2E.
+6. Give each live E2E dependency a uniquely named Foundry environment variable
+   documented in the sample and FRD. When the opt-in variables are absent, the
+   live sample E2E must skip with a clear reason rather than fail; partial
+   configuration should fail with an actionable message. When they are present,
+   a sample-owned setup/test script may:
+   - generate an ignored `local.settings.json`, but must refuse to overwrite an
+     existing or concurrently created user file;
+   - run the real-model E2E and capture exact evidence of the authored tool
+     arguments, dependency graph, or configuration plus the observable result;
+   - track file ownership and clean up only the files that script created.
+   Record the exact commands, prompt, relevant model-authored structured input,
+   and resulting terminal evidence in both the FRD and PR without exposing secrets.
+   The `samples/workflow-retry-policy/scripts/run-e2e.py` proposed in PR #170 is
+   a concrete example of this pattern when available, but these gates are generic
+   and do not depend on that unmerged implementation.
+7. Treat independent sample review and the required live boundary as a sample
+   validation gate. Do not accept bypass-model tests as evidence that it passed.
+8. Run the full CI-equivalent gate:
    ```bash
    python -m pytest --cache-clear --cov=./src/azure_functions_agents --cov-report=xml --cov-branch tests
    ```

docs/frds/_template.md

@@ -56,27 +56,74 @@ branch: <user>/<slug>
 
 > Backward-compatibility notes, deprecations, and migration guidance.
 
-## 5. Decisions log
+## 5. Sample / user journey
+
+> Complete this section when the feature adds or modifies a runnable sample;
+> otherwise state "Not applicable" and why. Keep each sample focused on one
+> primary capability so a learner can understand what its result demonstrates.
+
+- **Target learner:** …
+- **Primary capability:** …
+- **User-centered story / natural-language prompt:** …
+- **Expected observable terminal result:** …
+- **Dedicated sample vs existing sample:** …
+- **Rationale and impact on existing samples:** …
+
+### Model-backed E2E boundary
+
+> If an LLM generates structured execution inputs such as tool arguments,
+> dependency graphs, or configuration, require an opt-in real-model E2E that
+> covers **natural-language prompt → model → tool call → parser/runtime →
+> observable terminal result**. Unit or integration tests that bypass the model
+> cannot substitute for this boundary.
+
+- **Real-model E2E required:** Yes / No — …
+- **Uniquely named Foundry environment variables:** …
+- **Sample-owned setup/test script:** …
+- **Exact setup, run, and test commands:** …
+- **Exact evidence to capture in the PR:** … (prompt, relevant model-authored
+  tool arguments / dependency graph / configuration, and observable result)
+- **Generated ignored files (for example, `local.settings.json`):** … (refuse
+  to overwrite an existing or concurrently created user file)
+- **Cleanup:** … (track ownership and remove only files created by the
+  sample-owned script)
+- **Missing-environment behavior:** Skip clearly when not opted in; fail with an
+  actionable message when only part of the required configuration is present.
+
+## 6. Decisions log
 
 > Append-only. Record every non-trivial choice and **who** made it. This is the
-> durable record that makes the FRD worth committing.
+> durable record that makes the FRD worth committing. When a runnable sample is
+> in scope, include the decision to create a dedicated sample or extend an
+> existing one.
 
 | # | Decision | Options considered | Choice | Decided by | Date |
 | - | -------- | ------------------ | ------ | ---------- | ---- |
 | 1 | …        | A / B / C          | B      | Human / Agent | YYYY-MM-DD |
 
-## 6. Test plan
+## 7. Test plan
 
 > New/changed tests, mirroring source modules under `tests/`. When config or
 > authoring behavior changes, add a scenario folder under
 > `tests/fixtures/config_scenarios/`. List the key cases (happy path, errors,
-> edge cases) and any new fixtures.
+> edge cases) and any new fixtures. For runnable samples, include an independent
+> sample review, exact validation commands and evidence, and tests for existing
+> samples affected by extraction or shared-code changes.
 
 - [ ] Unit: …
 - [ ] Fixture scenario: `tests/fixtures/config_scenarios/<nn_name>/`
 - [ ] Regression (if fixing a bug): …
-
-## 7. Docs impact
+- [ ] Independent sample review (if applicable): …
+- [ ] Existing affected samples: …
+- [ ] Opt-in real-model E2E (if required): prompt → model → tool call →
+      parser/runtime → observable terminal result
+- [ ] Missing E2E environment variables skip clearly rather than fail.
+- [ ] Exact commands, prompt, structured model output, and terminal evidence
+      recorded in the FRD and PR without exposing secrets.
+- [ ] Sample-owned script refuses to overwrite user files and cleans up only
+      the files it created.
+
+## 8. Docs impact
 
 > Which docs change and how.
 
@@ -85,7 +132,7 @@ branch: <user>/<slug>
 - [ ] `docs/triggers.md` — trigger types
 - [ ] `README.md` — user-facing quickstart / examples
 
-## 8. Status & sign-off
+## 9. Status & sign-off
 
 - **Architecture review (phase 2):** <summary of reviewer findings / link>
 - **Human sign-off:** <name, date> → set `status: Finalized` before implementing.