diff --git a/AGENTS.md b/AGENTS.md index d24b40c81..06c0adcd3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,7 +14,7 @@ instead of copying its content here. | add or change localized content | [`docs/translation.md`](docs/translation.md) — plus the matching `docs/references/terminology-.md` when one exists | | add, edit, reorganize, or review any tracked contributor Markdown (this file, `docs/*`, `.github/*.md`, package- and source-local READMEs) | [`docs/DOC-MAINTENANCE.md`](docs/DOC-MAINTENANCE.md) — *if you can't grep it on this branch, don't claim it* | | open or update a pull request | [`docs/pull-request.md`](docs/pull-request.md) | -| manually confirm a feature works | [`docs/verification.md`](docs/verification.md) — a throwaway scratch script against the built extension, not the committed suite | +| manually confirm a feature works | [`docs/verification.md`](docs/verification.md) — drive a throwaway session against the built extension, not the committed suite | ## DeepWiki Context diff --git a/docs/README.md b/docs/README.md index f34c011f5..a4aaa7b7c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,7 +11,7 @@ | [`pull-request.md`](./pull-request.md) | PR 描述指南:代理与贡献者使用的详细章节、按变更类型取舍规则、验证与审查信息要求。 | | [`design.md`](./design.md) | 设计系统参考:主题机制、shadcn 组件选型、新建页面配方总览;令牌完整值拆到 [`references/design-tokens.md`](./references/design-tokens.md),组件清单拆到 [`references/design-components.md`](./references/design-components.md),布局/响应式/动效/状态/无障碍范式拆到 [`references/design-patterns.md`](./references/design-patterns.md)。**做页面/对话框/区块前先读。** | | [`../e2e/README.md`](../e2e/README.md) | E2E 测试台手册:两条赛道(committed smoke / gitignored scratch)、浏览器与 profile 隔离、fixtures 与 helper 清单、协议 mock、`E2E_*` 环境变量、产物与失败排查路径。**跑 / 写 E2E 或一次性验证脚本前先读。** | -| [`verification.md`](./verification.md) | 功能验证指南:用一次性 scratch 脚本驱动真实扩展做端到端验证(不跑全量 E2E、不加永久用例);报告模板拆到 [`references/verification-report-template.md`](./references/verification-report-template.md),调试 FAQ 拆到 [`references/verification-debugging.md`](./references/verification-debugging.md)。**验证改动是否真正跑通时读。** | +| [`verification.md`](./verification.md) | 功能验证指南:启一个常驻会话(默认无头、可多 worktree 并发)逐条命令驱动真实扩展,只在需要复现顺序/时序时才写 spec,不跑全量 E2E、不加永久用例;驱动方法(GM API in-page self-test、SW 消息、主题)拆到 [`references/verification-methods.md`](./references/verification-methods.md),报告模板拆到 [`references/verification-report-template.md`](./references/verification-report-template.md),调试 FAQ 拆到 [`references/verification-debugging.md`](./references/verification-debugging.md)。**验证改动是否真正跑通时读。** | | [`architecture.md`](./architecture.md) | 内部原理总览:多进程模型、消息传递;各子系统深入拆到 [`references/architecture-services.md`](./references/architecture-services.md)(服务层)、[`references/architecture-data.md`](./references/architecture-data.md)(数据层)、[`references/architecture-gm-api.md`](./references/architecture-gm-api.md)(GM API)、[`references/architecture-execution.md`](./references/architecture-execution.md)(脚本执行)、[`references/architecture-build.md`](./references/architecture-build.md)(构建管线)、[`references/architecture-agent.md`](./references/architecture-agent.md)(Agent 子系统)。 | | [`cloud-sync.md`](./cloud-sync.md) | 云同步实现说明:同步文件语义、主流程、状态合并、provider 差异、错误分类、retry 策略和维护注意事项。 | | [`DOC-MAINTENANCE.md`](./DOC-MAINTENANCE.md) | 文档维护与事实核对指南:组织规则、逐条核对清单、跨文档政策一致性核对、隐私清理、以及在 resolved final tree 上的复核方法,覆盖全部 tracked 的 agent/contributor Markdown(不止 `AGENTS.md` + `docs/*`,还包括 `.github/*.md`、package-local README)。**改/审文档前先读。** | diff --git a/docs/references/verification-methods.md b/docs/references/verification-methods.md new file mode 100644 index 000000000..7656cfe86 --- /dev/null +++ b/docs/references/verification-methods.md @@ -0,0 +1,96 @@ +# Verification methods + +[`../verification.md`](../verification.md) chooses the form; this file holds the patterns that reach behaviour the UI does not expose directly. Each is written twice where the two forms differ: driving a session ([`../../e2e/README.md`](../../e2e/README.md#8-verification-sessions)) and authoring a spec. Failures and gotchas are [`verification-debugging.md`](verification-debugging.md)'s. + +## Script execution: GM APIs and injection + +Making a userscript actually inject and run needs two things: the `userScripts` permission granted, and the permission prompt answered. + +A session grants `userScripts` at `start`, so injection works out of the box. It does **not** auto-approve prompts — a GM API that needs a grant opens `confirm.html`, which you answer like any other page: + +```bash +node e2e/drive.mjs pages # 找到 confirm.html +node e2e/drive.mjs use +node e2e/drive.mjs click "[data-testid=confirm-duration-permanent]" +node e2e/drive.mjs click "[data-testid=confirm-allow]" +``` + +In a spec, `testWithUserScripts` and `autoApprovePermissions` solve both ([`../../e2e/README.md`](../../e2e/README.md#3-harness-chain)) — import them rather than re-deriving the launch dance. + +### The in-page self-test pattern + +A userscript runs assertions in the page and prints a summary line the harness parses from the console. The bundled scripts in [`../../example/tests/`](../../example/tests/) do this; the line varies by script, and each emits a `通过`/`Passed` and a `失败`/`Failed` count: + +``` +总计: 12 | 通过: 12 | 失败: 0 # inject_content_test.js / sandbox_test.js (combined line) +总测试数: 12 / 通过: 12 / 失败: 0 # gm_api_sync_test.js / gm_api_async_test.js (counts on separate lines) +Total: 12 | Passed: 12 | Failed: 0 # window_message_test.js (English) +``` + +In a session there is nothing to wire up — the collector already recorded the line, whichever context printed it (a `@background` script prints from `src/sandbox.html`, not from a page): + +```bash +node e2e/drive.mjs console 200 | grep -E "(通过|Passed)[::] *[0-9]+" +``` + +In a spec, collect and assert on it — this regex matches all three layouts: + +```ts +const logs: string[] = []; +let passed = -1; +let failed = -1; +page.on("console", (msg) => { + const text = msg.text(); + logs.push(text); + const pass = text.match(/(通过|Passed)[::]\s*(\d+)/); + const fail = text.match(/(失败|Failed)[::]\s*(\d+)/); + if (pass) passed = parseInt(pass[2], 10); + if (fail) failed = parseInt(fail[2], 10); +}); +// ...navigate to the target page, then: +expect(failed, logs.join("\n")).toBe(0); +expect(passed).toBeGreaterThan(0); +``` + +For a new GM API, write a small self-test userscript in the same style. In a session, `node e2e/drive.mjs install ` installs it through the Service Worker and `node e2e/drive.mjs console` shows the summary line the script printed; in a spec, use `installScriptByCode`. Keep the script inside the scenario directory — it is verification scaffolding, not a committed example. + +## Behaviour fired from extension UI + +The self-test pattern covers only what a userscript observes in the page. Some behaviour is fired from extension UI — a `GM_registerMenuCommand` menu is triggered from the popup. Clicking that button is not drivable ([`verification-debugging.md`](verification-debugging.md#common-gotchas)); sending the message it sends is. + +Clients talk to the Service Worker via `chrome.runtime.sendMessage({ action, data })`, where `action` is `/` and the reply is wrapped as `{ code, data }` — payload is `res.data`, a truthy `code` means error ([`../../packages/message/client.ts`](../../packages/message/client.ts)). Read the tab coordinates you need (`tabId`/`frameId`/`documentId`) from a prior `getPopupData` call. + +```ts +// from a chrome-extension:// page (e.g. options.html); poll until the async registration shows up +const res = await chrome.runtime.sendMessage({ + action: "serviceWorker/popup/getPopupData", + data: { tabId, url }, +}); +const script = res.data.scriptList.find((s) => s.menus.some((m) => m.name === "your-menu")); +await chrome.runtime.sendMessage({ + action: "serviceWorker/popup/menuClick", + data: { uuid: script.uuid, menus: script.menus }, // menus carry the target tabId/frameId/documentId +}); +``` + +From a session the same call is one command, since `eval` already runs on an extension page: + +```bash +node e2e/drive.mjs open options +node e2e/drive.mjs eval "const [tab] = await chrome.tabs.query({active:true,lastFocusedWindow:true}); if (!tab?.id || !tab.url) throw new Error('no active tab'); const r = await chrome.runtime.sendMessage({action:'serviceWorker/popup/getPopupData', data:{tabId:tab.id, url:tab.url}}); return r.data.scriptList" +``` + +This drives the real SW → content → sandbox → callback path, behaviourally identical to the popup button, which discards the DOM event and calls the same message. It is a substitution: the verdict row names it and says the popup's own click path was not covered. + +## A UI change across light and dark theme + +The theme is stored in `localStorage` under `lightMode` with value `"light"` / `"dark"` / `"auto"` ([`../../src/pages/components/theme-provider.tsx`](../../src/pages/components/theme-provider.tsx), and [`../../src/pages/common.ts`](../../src/pages/common.ts), which reads the same key during pre-render to avoid a theme flash). Setting it before the page's own scripts run — `context.addInitScript` — is what applies the theme on first paint instead of flashing the default. + +Confirm that timing for a `chrome-extension://` page in your own setup before relying on it: `addInitScript` timing relative to an extension page's bootstrap can differ from a normal web page. Capture one screenshot per theme as separate evidence; one theme's screenshot does not show the other renders correctly. + +A session has no `addInitScript` hook of its own, so set the key and reload — the pre-render read in `common.ts` then picks it up before first paint: + +```bash +node e2e/drive.mjs eval "localStorage.setItem('lightMode','dark'); return location.reload()" +node e2e/drive.mjs shot settings-dark +``` diff --git a/docs/references/verification-report-template.md b/docs/references/verification-report-template.md index e4de2904a..c83ed4ffe 100644 --- a/docs/references/verification-report-template.md +++ b/docs/references/verification-report-template.md @@ -1,218 +1,97 @@ -# Verification record template + -Before running the browser, create a short verification record in the scenario directory, for example -`e2e/scratch//report.md`. Keep the reusable template headings in English, but write the actual -record content in the user's language. Update it as the run proceeds instead of filling it in only at the end. +# Local verification: -**The snippet below is a filled *example* of the `## Acceptance Evidence` shape** — it shows what a completed -one looks like, not a second section to add. The full template further down has its own heading; use that one -and fill it following this example. - -Evidence is organized **one `###` section per `Verdict` row**, not by artifact type. A reader arrives from a -`V2` row and finds every screenshot, log line, and fixture that decides `V2` in one place, in the order they -were observed. Verdict labels stay in the `Verdict` table and are not repeated here. - -This record exists so a reader can judge whether the implementation is correct, so **evidence is embedded, not -linked**: scrolling `report.md` top to bottom should show the pixels and the deciding log lines without opening -a single side file. A bare link is the fallback for artifacts that genuinely cannot render inline (archives, -binaries, multi-megabyte logs), and it carries a note saying what it holds. - -~~~md -## Acceptance Evidence - -### V1 · The `/` route mounts and lists installed scripts - -![Options root](screenshots/v1-options-root.png) -The script list rendered with the view toggle visible — the route mounted, rather than falling through to a -blank shell. - -```text -[verify] options url = chrome-extension:///src/options.html#/ -``` - -### V2 · `/settings` renders correctly in light and dark - -| Light | Dark | -| --- | --- | -| ![Settings light](screenshots/v2-settings-light.png) | ![Settings dark](screenshots/v2-settings-dark.png) | - -Readable contrast in both themes — the shell picked up the theme tokens instead of falling back to one palette. -One theme's screenshot alone would not show this. - - - -The full navigation from the script list to the settings page. The decisive frames, because a video is neither -skimmable nor playable in every viewer: - -![Before navigation](screenshots/v2-nav-01-list.png) -The settings entry, enabled, before the click. - -![After navigation](screenshots/v2-nav-02-settings.png) -The route changed and the content painted, after the click. - -### V3 · Importing a backup restores every script in it - -`resources/import.yaml` — the input this run consumed: - -```yaml -scripts: - - name: demo-script - source: https://example.com/demo.user.js -``` - -```text -[verify] script count after import = 3 -``` - -Three scripts in the file, three in the list. Full capture: [console.log](console.log) — no unexpected errors -during the run. -~~~ +## Mode -Use this shape: +`verifying a change` | `reproducing a bug` -```md -# Local E2E Verification Record: +## Goal / problem -## Mode +. -`verify-change` | `reproduce-bug` +## Verdict -## Goals / Problem + -- (verify) What behavior should hold, and why it might not -- (reproduce) **Expected:** … **Actual:** … +| # | Requirement / bug claim | Verdict | Real / substituted | How observed | Check it yourself | +|---|---|---|---|---|---| +| V1 | `` | holds / does not hold / not observed | real, or `substituted: ` | `` | `` | -## Reproduction Steps +Summary: . -1. … -2. … +| Label | Use it when | Requires | +|---|---|---| +| `holds` | you observed the behaviour at runtime | the deciding observation, and how a reader reaches it | +| `does not hold` | you observed it failing, or the bug reproducing | the failing output, assertion diff or error screenshot | +| `not observed` | you never reached the check | what stopped it | -## Minimal Reproduction +An unreached check is never `holds`; a run that verified two of three claims is reported as two of three. -- Smallest script/page/steps that trigger it (link `resources/…`) +## Authorization -## Task List + -- [ ] Prerequisite checks passed -- [ ] Built and loaded the real extension -- [ ] Opened target page and confirmed stable anchor -- [ ] Saved screenshots, videos, and logs -- [ ] Every Verdict row filled +| # | Substitute or effect | The user's authorization, verbatim | +|---|---|---| +| V1 | `` | `` | -## Execution Log +## Reproduction steps -| Step | Status | Evidence | Notes | -| --- | --- | --- | --- | -| Open options page | Pending | - | - | + -## Verdict +1. `` -| # | Claim under verification | Verdict | How observed | Check it yourself | -| --- | --- | --- | --- | --- | -| V1 | | holds / does not hold / not observed | | `` | +## Acceptance evidence -Summary: + -- (reproduce) Scratch asserts the **desired** behavior (stays red) or the **current buggy** contract - (passes green; the fix must flip it) — say which +### V1 · `` -## Blockers +```text +[verify] +``` -- None +. Full capture: ``. -## Acceptance Evidence +| Light | Dark | +|---|---| +| `![Settings light](screenshots/v1-light.png)` | `![Settings dark](screenshots/v1-dark.png)` | -### V1 · + - +## Evidence index -## Persistent Data Changes +- Screenshots/video: `` +- Logs: `` +- Resources: `` -| Change | Forward | Backward / backup | Before-after check | -| --- | --- | --- | --- | -| | | | | +## Persistent data changes -## Integrity & Cleanup + -- HEAD at start / end: `` / `` -- `git status --porcelain` at end: `` -- Artifacts, processes, and external data created — and how each was cleaned up: `` -- Redaction performed before saving: `` +| Change | Forward | Backward/backup | Before/after query | +|---|---|---|---| +| `` | `` | `` | `` | -## Evidence Index +## Execution record -- Screenshots / video: -- Logs: -- Resources / data snapshots: -``` +| Step | Status | Evidence/blocker | +|---|---|---| +| `` | pending / passed / failed / blocked | `` | -Fill `Verdict` last — the honest result, per *Step 4 — Report honestly* in -[`verification.md`](../verification.md). Execution Log `Status` moves `Pending` → `Pass` / `Fail` / `Blocked`. +## Integrity and cleanup -### Verdicts are per claim, and there are three of them +- Initial/final HEAD: `` / `` +- Final `git status --porcelain=v1`: `` +- Created artifacts/processes/external data and cleanup: `` +- Redaction performed: `` -One row per claim you set out to verify — split a compound claim rather than averaging it into one row. The -three labels are not interchangeable: +## Evidence rules -| Label | Use it when | Requires | -| --- | --- | --- | -| `holds` | you observed the behavior at runtime | the deciding observation *and* a command a reader can re-run | -| `does not hold` | you observed it failing, or observed the bug reproducing | the failing output, assertion diff, or error screenshot | -| `not observed` | you never reached the check — blocked, out of scope, environment missing | a `Blockers` entry saying what stopped it | - -`not observed` is the one that keeps a report honest: an unreached check is **never** `holds`. A run where two -claims held and one was never exercised is reported as exactly that, not as a pass. When the cause was an -unconfigured environment, name the service and the *variable names* that were missing — never their values. - -The `Check it yourself` column exists so a reviewer can reproduce a row without reconstructing the run; if a row -has no such command, say why in `How observed` rather than leaving it blank. - -### Sections to drop when they don't apply - -- `verify-change` mode: drop `Reproduction Steps` and `Minimal Reproduction`. In `reproduce-bug` mode fill - `Expected`/`Actual` and keep them, so a later reader or AI can re-trigger the bug from `report.md` alone - without reading the code. -- `Persistent Data Changes`: keep only when the run wrote data that outlives it — a real cloud-sync provider, an - imported backup, an OPFS/IndexedDB migration. An ephemeral browser profile that the harness deletes is not a - persistent change. Note the blast radius honestly: "only this test profile" is a valid, useful entry. -- `Integrity & Cleanup`: keep whenever the run touched a real external target or left anything behind. It is - what lets a reviewer confirm the verification didn't quietly modify the working tree or leave a live process - or real remote data around. - -Keep the checklist factual: - -- Start with unchecked tasks that describe what you intend to verify. -- Check items only after the corresponding command/assertion has actually passed. -- If a step is blocked, leave its checkbox unchecked and add a concrete entry under `Blockers`: what failed, - where it failed, and what evidence was captured. - -### Inside an Acceptance Evidence section - -One `###` per `Verdict` row, headed `V · `, holding everything that decides that row — commands, -output, screenshots, fixtures — in the order you observed them. Rules that follow from that: - -- A claim with no evidence section is `not observed`, not `holds`. If a row genuinely needs no artifact beyond - its `Check it yourself` command, say so in one line rather than omitting the section. -- One artifact can back two rows; put it under the row it decides and reference it from the other rather than - pasting it twice. -- Don't restate the verdict word here — the `Verdict` table owns it, and two copies drift apart. -- `Evidence Index` at the end is a **pointer list**, not a second copy: paths, and which row each backs. The - pixels and the deciding lines stay inline in the V sections. - -Keep the evidence embedded: - -- **Screenshots** — `![alt](screenshots/….png)` plus a caption line stating what it proves. Put paired shots - (before/after, light/dark) in a two-column table so the comparison is one glance, not two scrolls. -- **Videos** — ``. This renders as a player only in - viewers that allow inline HTML, and a recording is slow to review either way, so capture the deciding moments - as `page.screenshot()` calls *during* the run and embed those stills next to the video. The stills, not the - recording, are what carries the verdict. -- **Logs** — paste the lines the verdict rests on into a fenced block, then link the full capture for the rest. - A link alone forces the reader to reconstruct which line mattered. -- **Resources** — paste short text fixtures (YAML/JSON/userscript) inline in a fenced block. Link only what is - large or binary, and say what it contains. -- Sanitize tokens, cookies, and real credentials *before* pasting log or resource content inline — embedding - puts it in front of every reader. -- Keep every path relative to `report.md`. The scenario directory, not `report.md` alone, is the unit you hand - to a reviewer; moving the file out of it breaks every embed. +- Every `holds` names how the target was driven — command, or launch plus steps — and the deciding observation. A session already wrote that record to `actions.log`; quote it rather than reconstructing it from memory. +- Where a claim changes state beyond the driven surface, that observation is an independent read: extension storage from an extension page, or the page console. +- Embed decisive text and images inline; scrolling this file should reach a verdict without opening a side file. Link only archives, binaries and full captures, each with a note on what it holds. +- One artifact can back two rows: put it under the row it decides and reference it from the other. +- Keep failed and unchecked steps visible. Redact tokens, cookies and real credentials before saving, and again before embedding. +- Keep every path relative to this file; the scenario directory, not `report.md` alone, is what you hand to a reviewer. diff --git a/docs/verification.md b/docs/verification.md index 1002fb866..acb945204 100644 --- a/docs/verification.md +++ b/docs/verification.md @@ -1,311 +1,121 @@ -# Functional Verification Guide - -> **What this owns.** How to *confirm a change actually works* — or *reproduce a reported bug* — by driving the -> **real built extension** end-to-end, written so an AI coding tool (Claude, Codex, …) or a human can do it -> without inventing a workflow. Both modes use the same harness; the per-scenario `report.md` (below) is the -> consultable record of what was verified or reproduced. This is deliberately **lightweight**: one-shot scratch -> scripts and local-only evidence — kept out of Git and never deleted as part of a run; cleanup is the user's call. -> -> **What this is NOT.** It is *not* the test-suite reference and *not* the harness manual. Vitest mechanics live -> in [develop.md § Testing](./develop.md#testing); the E2E harness itself — fixtures, isolation, protocol mocks, -> `E2E_*` environment variables, artifact locations — is owned by [`e2e/README.md`](../e2e/README.md); the -> TDD-first principle and engineering rules live in -> [AGENTS.md § Engineering Principles](../AGENTS.md#engineering-principles). +# Feature verification ## When to skip this guide -This guide is **workflow routing**, not a TDD blanket exception — it applies when a change needs the built -extension, real Chrome APIs, or cross-context behavior to observe. Skip it (and rely on typecheck + the -relevant committed test instead) for: +Use targeted committed tests alone when they fully observe the changed logic — pure logic, parsers, utilities, +docs, comments, types, and anything the committed suite already proves. Use this guide when the behaviour +depends on cross-context wiring (Service Worker ↔ Content ↔ Inject ↔ Offscreen ↔ Sandbox) or a real Chrome API a +unit test cannot exercise, or when reproducing a runtime-only bug. It does not replace TDD. -- Doc-only, comment-only, or type-only changes with no runtime behavior to observe. -- Pure logic that a targeted unit test already exercises completely (a parser, a utility function, a reducer) — - write/run that test instead of driving a browser for it. -- Any change fully reproducible and provable by a targeted committed test without the built extension. +Verification is not how the committed suite grows: do not run `pnpm run test:e2e` to check one thing, and do not +add an `e2e/*.spec.ts` as part of it. Promotion is a separate decision +([`references/develop-testing.md`](references/develop-testing.md#choosing-a-test-boundary)). -If you're unsure whether a change needs the built extension, the deciding question is: *does this depend on -cross-context wiring or a real browser API that a unit test can't exercise?* If no, a targeted unit test is -the whole verification; don't reach for this guide's scratch-script workflow just to "be thorough." +## Workflow -## The one rule: verification ≠ growing the E2E suite +1. Run `pnpm run typecheck` and `pnpm test -- --run `; run `pnpm test` only when the blast radius is not + confirmed local or a gate requires it. +2. Build the extension with `pnpm run dev` (or `pnpm run build`). The session loads `dist/ext`, so a stale build + silently verifies old code — `session.mjs` refuses to start when `dist/ext/manifest.json` is missing, but it + cannot tell you the build is *old*. +3. Start a session and drive it. Everything it produces lands in `e2e/scratch//`, which is gitignored. -The full E2E suite is **heavy** (two-phase browser launch, real network fetches, multi-minute timeouts). When -you only want to *check that a feature works*, do not pay that cost and do not leave anything behind: + ```bash + node e2e/session.mjs start # 常驻浏览器,默认无头 + node e2e/drive.mjs open options # 一条命令一次操作 + node e2e/drive.mjs snapshot # 要点什么,先看有哪些可交互元素 + node e2e/drive.mjs click '[data-testid="theme-toggle"]' + node e2e/drive.mjs shot after-toggle + node e2e/session.mjs stop + ``` -- ❌ **Never** run the whole suite (`pnpm run test:e2e`) just to verify one thing *during casual verification*. - This rule scopes this guide's workflow — it is not a release/CI policy; CI and pre-release gates run the full - suite as their own separate, deliberate check. -- ❌ **Never** add a permanent `e2e/*.spec.ts` as part of casual verification. -- ✅ Write a **throwaway scratch script** under `e2e/scratch/` (git-ignored), run it, and keep any evidence local. +4. Before running, create `report.md` in that directory from + [`references/verification-report-template.md`](references/verification-report-template.md); update it as + evidence arrives. +5. Record how the target was driven, deciding runtime observations, gaps and shortest user reproduction steps. + `actions.log` already holds the driving record verbatim — quote it rather than reconstructing it. -Promoting a scenario into the permanent suite is a *separate, deliberate* decision — only when it deserves -permanent regression coverage. The criteria live in -[develop-testing.md § Choosing a test boundary](./references/develop-testing.md#choosing-a-test-boundary). +## Choosing the form -**Reproducing a bug you intend to fix is *not* "casual verification."** A scratch reproduction is the *确定 bug -存在* step in [`../AGENTS.md`](../AGENTS.md)'s TDD / Confirm-before-you-fix policy. In the general case it -confirms the bug is real but is not itself the required test — promote it into a committed failing test before -fixing. Only under that policy's infeasible-automated-coverage exception (criteria in -[develop-testing.md § When TDD doesn't apply](./references/develop-testing.md#when-tdd-doesnt-apply)) does the -scratch reproduction — its `report.md`, screenshots, and observations — stand as the required evidence on its -own. +Drive the live session by default. Author a spec only when the extra cost buys something. -Choose the reproduction method by what the bug depends on: a failing unit test for pure logic/parser/utility -bugs; this guide's scratch-script workflow (above) when it depends on the built extension, browser APIs, or -cross-context behavior. +| To observe the target | You author | +|---|---| +| a one-off state, visual or console check, however many steps it takes | nothing — drive the session | +| a sequence that must be replayed identically, or where timing/concurrency *is* the contract | a scratch spec | +| a flow worth protecting from regression forever | a committed `e2e/*.spec.ts`, as a separate decision | -## Prerequisite gate (cheap signals first, proportional to risk) +A session survives between commands, so exploring costs one command per question instead of one edit-and-rerun +cycle per question. A spec earns its cost when the *ordering* is the thing under test — `drive.mjs` gives no +guarantee about the gap between two invocations. -Driving a browser is the *last* check, not the first. Confirm the cheap signals are green before you build — -but scale which signals proportionally, not mechanically: +Scratch specs still run through the scratch config, and still belong to a scenario directory: ```bash -pnpm run typecheck # tsc --noEmit — always -pnpm test -- --run path/to/file.test.ts # targeted unit test(s) for the change — the default -pnpm test # full Vitest suite — only when the blast radius isn't confirmed - # local (shared utils, config, public interfaces), or a gate requires it +pnpm exec playwright test --config playwright.scratch.config.ts -g "" ``` -Green unit tests do **not** mean the feature works — they mean the units you tested behave. Cross-context wiring -(Service Worker ↔ Content ↔ Inject ↔ Offscreen ↔ Sandbox) and real Chrome APIs only exercise in a loaded -extension. That gap is exactly what this guide closes. - -## Step 1 — Build a loadable extension - -```bash -pnpm run dev # development build with source maps → writes dist/ext -# or: pnpm run build # production build, also → dist/ext -``` - -Every fixture loads `dist/ext`, so a stale build silently verifies old code — rebuild first. Setup details and -what a rebuild does or doesn't require live in [`e2e/README.md § Setup`](../e2e/README.md#2-setup). - -## Step 2 — Write a scratch verification script - -Each verification gets its own scenario directory **`e2e/scratch//`** holding the script *and* every -artifact it produces. The scripts reuse the committed harness, so you write almost no boilerplate — the -fixtures, the page openers, the script installer, and the environment variables are catalogued in -[`e2e/README.md`](../e2e/README.md). The short version: - -```ts -import { test, expect } from "../../fixtures"; // context + extensionId, onboarding dismissed -import { openOptionsPage } from "../../utils"; // page openers, installScriptByCode, … -``` - -### Evidence location - -Keep the script and all of its throwaway evidence together under **`e2e/scratch//`**: - -- the script itself: `e2e/scratch//*.spec.ts` -- screenshots: `e2e/scratch//screenshots/*.png` -- videos: `e2e/scratch//videos/*.webm` -- logs / notes / short verification reports: `e2e/scratch//*.md` or `*.log` -- additional test resources: `e2e/scratch//resources/` +## Driving the session -`e2e/scratch/` is git-ignored, so these files are local evidence only and must not be committed. Keep them out -of `test-results/` as well: Playwright wipes that directory at the start of every run, so evidence parked there -disappears the next time anyone runs the suite. Do not put verification screenshots, videos, or notes under -`docs/` or committed source directories unless you are deliberately adding permanent documentation assets. +[`../e2e/README.md`](../e2e/README.md#8-verification-sessions) owns the command reference. What matters for a +verdict: -Use `resources/` for any extra local inputs or outputs needed to understand or reproduce the run, for example: +- **Observe from a path the driven surface does not share.** `drive.mjs storage` reads `chrome.storage.local` + from an extension page, and `drive.mjs sw` evaluates inside the Service Worker — neither goes through the UI + you just clicked. +- **The session records continuously, from every context.** `console`, uncaught exceptions and log entries from + the Service Worker, the Offscreen document, the Sandbox (where `@background` / `@crontab` scripts run) and + every page all land in `console.log`, tagged with their origin — including output produced before you thought + to look. That is what makes a console-asserting userscript self-test observable without authoring a spec. Every + `drive.mjs` command appends to `actions.log`. +- **Screenshots are captured while the run is alive**, into `/shots/`, numbered in capture order. +- **`sw` runs *inside* the Service Worker**, so `chrome.runtime.sendMessage` there does not reach the extension + — send those from an extension page with `drive.mjs eval`. -- inline userscripts copied out of a scratch file for readability -- mock API responses, fixture JSON/YAML, import/export files, generated ZIPs, or downloaded artifacts -- temporary HTML pages, saved network payloads, or before/after data snapshots +Sessions are headless: verification must not steal desktop focus, and several worktrees verify at once. Add +`--headed` only to watch by eye. -Surface these resources in `report.md` — short text fixtures pasted into a fenced block, anything large or -binary as a relative link such as `[Exported backup](resources/backup.zip)`. Keep secrets and real credentials -out of the resource directory; sanitize them before saving evidence, and again before pasting any of it inline. +## Running more than one at a time -`report.md` is read to decide whether the implementation is correct, so embed the evidence instead of linking -it: screenshots as `![alt](screenshots/….png)`, videos as `