Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

- **feat(e2e): 浏览器工具角色路由 + 会话状态语义 + doctor storageState 诊断(`e2e-mcp-tool-routing`)**. 起因:会话同时挂着 `playwright-test`(test-runner MCP)与用户自装的 `chrome-devtools-mcp` 两套浏览器工具时,e2e 模板原句 "Browser exploration is tool-agnostic" 对选择只字未提——模型按表面相似度随机选;且两套工具会话语义相反(chrome-devtools attach 真实 Chrome persistent profile、登录态跨会话记住;test-runner 的 browser_* 中继进 paused test 的 fixture context、每次 launch 全新),用户把后者的"失忆"误判为配置缺陷。模板侧:路由句改为角色路由——test_*/planner_*/generator_* 只在 playwright-test(生成/执行/修复不换轨)、交互探索优先可回退(编辑器未装或需 CDP 独有能力如 Performance trace 时用 chrome-devtools)、真并行走 `openspec-pw explore --parallel N`,就近声明「persistent profile 登录态不可当 E2E 前提」;Step 4 增补 Session state 小节:fresh context = E2E 隔离设计而非缺陷、登录态唯一恢复路径 = auth.setup → playwright.config storageState 依赖链、明确警示 `browser_storage_state` 类工具在工具清单里但 paused-test 会话不收(storage capability 未启用)防按清单误选。doctor 侧:新增可选检查 `storage-state`——config 声明 storageState 而 auth state 文件缺失 → 黄 ⚠ + `npx playwright test --project=setup` 修复命令(不阻断、不影响 ok 与退出码);路径提取判则防误报:字面量(`.auth`/`user.json` 特征,含 `join(...)` 拼接片段)优先,未命中时仅当 auth.setup.ts 存在(模板布局)才按默认路径判断,config 未声明或判则都不达 → 不输出。工具事实经实测落地(playwright 1.63.0):89 工具、storage 类 capability 为 "storage" 被内层 `capabilities: ["testing"]` 硬编码过滤器排除、双进程跨 launch localStorage 不留存——断言只写实测证实的行为。
- `templates/e2e-command.md`、`src/commands/doctor.ts`、`tests/commands/doctor.test.ts`(+4 用例)、`README.md`、`README.zh-CN.md`、`CHANGELOG.md`

## [0.3.99] - 2026-10-10

- **feat(gitignore): 受管块新增探索截图目录(`managed-screenshots-local`)**. `tests/playwright/screenshots/` 入受管 .gitignore 清单(21→22 项)。依据:探索截图本性时效(随开发过时,与 §6「截图证明像素,不证明行为」同判),且其同层证据——消费文档 `app-exploration.md` 与 `openspec/reports/`——早已是本地文件(受管忽略),截图是探索证据层里唯一还在入 git 的错位项(活路由截图靠同 slug 覆盖自愈,死路由截图永久沉积)。转本地后三层一致:本地证据本地存、耐久资产(specs 等)才进库。存量已提交截图走既有「已追踪检测」advisory(`git rm --cached`)。双 README 段落同步,测试 +1 用例。
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,7 @@ Run through these steps in order when using the E2E workflow for the first time:
| **Playwright MCP** | test-runner server configured for each **authorized** editor (command artifacts exist; unauthorized editors get an info line pointing at `init --tools <id>` — never a warning) | `playwright-cli` (@playwright/cli on PATH) — optional ⚠, never block |
| **Vendored Agents** | — | official agent snapshots' presence + ownership (owned/modified); non-blocking ⚠ when their `playwright-test` MCP dependency is missing |
| **Sync** | standards in sync when initialized (drift → `openspec-pw update`; AGENTS.md without markers counts as not-initialized, always ok) | not initialized (gated, non-blocking) |
| **Tests** | `tests/playwright/` directory exists | `auth.setup.ts` presence |
| **Tests** | `tests/playwright/` directory exists | `auth.setup.ts` presence · `storage-state` integrity — config declares `storageState` but the state file is missing → ⚠ + `npx playwright test --project=setup` (never blocks) |
| **Seed Test** | — | `seed.spec.ts` presence |
| **App Server** | — | dev script, base URL, reachability |
| **CodeGraph** | — | CLI availability, index presence, MCP installation (warnings, non-blocking) |
Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ audit **只报告、绝不删除**——测试退役永远是人工决策。存
| **Playwright MCP** | 每个已授权(有命令工件)编辑器的 test-runner server 配置;未授权编辑器降为信息行(提示 `init --tools` 添加,不阻断) | `playwright-cli`(PATH 上的 @playwright/cli)——可选 ⚠,不阻断 |
| **Vendored Agents** | — | 官方 agents 快照的存在与归属(owned/modified);依赖的 `playwright-test` MCP 缺失时 ⚠(不阻断) |
| **Sync** | 已初始化时标准同步(漂移 → `openspec-pw update`;AGENTS.md 无标记视为未初始化,恒 ok:true) | 未初始化(闸门,不阻断) |
| **测试目录** | `tests/playwright/` 目录存在 | `auth.setup.ts` 是否存在 |
| **测试目录** | `tests/playwright/` 目录存在 | `auth.setup.ts` 是否存在 · `storage-state` 完整性——config 声明 `storageState` 而 state 文件缺失 → ⚠ + `npx playwright test --project=setup`(不阻断) |
| **种子测试** | — | `seed.spec.ts` 是否存在 |
| **应用服务器** | — | 开发脚本、基础 URL、可达性 |
| **CodeGraph** | — | CLI 可用性、索引存在、MCP 安装(警告,不阻断) |
Expand Down
35 changes: 35 additions & 0 deletions dist/commands/doctor.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/commands/doctor.js.map

Large diffs are not rendered by default.

36 changes: 36 additions & 0 deletions src/commands/doctor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ const OPTIONAL_NAMES = new Set([
"engines",
"specs",
"auth-setup",
"storage-state",
"seed",
"dev-script",
"base-url",
Expand Down Expand Up @@ -622,6 +623,41 @@ export async function doctor(options: DoctorOptions = {}) {
message: hasAuthSetup ? "found" : "not found (optional)",
});

// storageState declared but state file missing → every non-setup context
// launches anonymous; the login flows under test then 401 silently.
if (configPath) {
let text = "";
try {
text = readFileSync(join(projectRoot, configPath), "utf-8");
} catch {
// config exists but unreadable — the config check above already reports
// its presence; skip instead of crashing doctor
}
if (/\bstorageState\b/.test(text)) {
const literals = [...text.matchAll(/['"]([^'"]+)['"]/g)].map((m) => m[1]);
const explicit =
literals.find((s) => /(^|\/)user\.json$/.test(s) && s.includes("/")) ??
literals.find((s) => s.includes(".auth") && s.includes("/")) ??
(/\bjoin\([^)]*['"]\.auth['"]/.test(text) ? "playwright/.auth/user.json" : undefined);
// No literal and no template layout (auth.setup.ts) → the path is
// user-owned; guessing the default would report their custom layout
// as broken, so leave it out entirely.
const stateFile = explicit ?? (hasAuthSetup ? "playwright/.auth/user.json" : undefined);
if (stateFile) {
const file = stateFile.startsWith("/") ? stateFile : join(projectRoot, stateFile);
const hasState = existsSync(file);
checks.push({
category: "Tests",
name: "storage-state",
ok: hasState,
message: hasState
? `found ${stateFile}`
: `storage state file missing (${stateFile}) — test contexts launch anonymous (run: npx playwright test --project=setup)`,
});
}
}
}

// Seed test
const hasSeed = existsSync(
join(testsDir, "seed.spec.ts"),
Expand Down
15 changes: 13 additions & 2 deletions templates/e2e-command.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,16 @@
| 9 execute | generated spec | seed smoke only (`npx playwright test tests/playwright/seed.spec.ts`) |
| 10 report | full report | routes + Page Objects + seed result; omit Healer sections |

Browser exploration is tool-agnostic: the `playwright-test` MCP server's browser tools or `openspec-pw explore --parallel N`.
**Browser tool routing**:

| Situation | Action |
| --------- | ------ |
| Generation / execution / repair | `playwright-test` MCP — sole home of its `test_*` / `planner_*` / `generator_*` tools |
| Interactive exploration (Step 4) | prefer the `playwright-test` browser tools |
| Exploration fallback | chrome-devtools or similar — only when the `playwright-test` browser tools are absent, failing or hanging (switch rather than retrying into the same failure), or for abilities `playwright-test` lacks (Performance trace, full network panel) |
| ≥5 routes | `openspec-pw explore --parallel N` |

Never switch tools to gain or reuse state — a persistent-profile login is not a valid E2E precondition. State semantics: Step 4 → Session state.

## Testing Principles

Expand Down Expand Up @@ -172,6 +181,8 @@ Record findings in `app-exploration.md`. Output path:

**Idempotency**: If `app-exploration.md` exists → read, verify routes, update only changed/new routes. **Route Snapshot Hash**: Navigate to sitemap.xml → hash content → if unchanged since last exploration, skip re-exploration entirely. Store hash in `app-knowledge.md` → `Exploration State`.

**Session state**: the `playwright-test` browser tools drive the **paused test's fixture context** — every `planner_setup_page` / test run launches a fresh context with empty storage. That is correct E2E isolation, not a configuration problem. To explore logged-in flows, wire the login once via the storageState chain: `auth.setup.ts` (Step 7) writes `playwright/.auth/user.json`, and `playwright.config.ts` feeds it into every test context (`use.storageState` + `dependencies: ['setup']`). Do not try to save/restore state through storage tools mid-session — `browser_storage_state`-class tools are listed by the test-runner MCP but rejected by its paused-test session (the storage capability is not enabled there) — and never treat another tool's persistent-profile login as a test precondition (see Browser tool routing).

**4.3. Update shared knowledge**: Extract project-level findings to `tests/playwright/app-knowledge.md`. Auto-de-duplicate by key.

### 5. Generate test plan (change mode only)
Expand Down Expand Up @@ -288,7 +299,7 @@ When tests fail → **Healer** (3 phases):

2-5. **Selector repair**: Pick per the selector priority (Testing Principles — single source of truth): verified selector from app-exploration.md > `[data-testid]` (getByTestId) > getByRole > getByLabel > getByText > CSS id/class > positional. If the `playwright-test` MCP exposes `browser_generate_locator`, use it to generate and validate the candidate against the live page before rewriting.

2-6. **Fix + verify**: Apply fix → verify via the test-runner MCP tools when available (`test_list` → `test_debug` (pass the test id from `test_list`) → repair → `test_run` with `locations` scoped to the single test file for structured results; fall back to `npx playwright test --grep "<test-name>"` if the playwright-test MCP server is not configured) → if passes, log to app-knowledge.md (Selector Fixes / Assertion Fixes). Max 3 heal attempts per test. **One error per round**: fix a single error, re-run, then address the next.
2-6. **Fix + verify**: Apply fix → verify via the test-runner MCP tools when available (`test_list` → `test_debug` (pass the test id from `test_list`) → repair → `test_run` with `locations` scoped to the single test file for structured results; fall back to `npx playwright test --grep "<test-name>"` if the playwright-test MCP server is not configured or its test tools fail) → if passes, log to app-knowledge.md (Selector Fixes / Assertion Fixes). Max 3 heal attempts per test. **One error per round**: fix a single error, re-run, then address the next.

2-7. **Log**: Append healed selector/assertion to `app-knowledge.md`. Auto-de-duplicate by Route + Old Selector (selectors) or Test + Old Assertion (assertions).

Expand Down
Loading
Loading