diff --git a/README.md b/README.md index 0a7566c3..6aaf7bb4 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@

-

Describe what to test. Let AI help to test the user flow.

+

Describe a goal. Test the real app.

-

Describe the goal and let AI operate the real app. FSQ records evidence and turns the exploration into a test you can review and replay.

+

Describe what to do and what should happen. Let AI operate the real app and verify the outcome. Inspect screenshots, actions, and results to see what happened.

An open-source project from FSQ Lab.

@@ -27,64 +27,100 @@ Contribute

-## See FSQ in action +## See a complete test -https://github.com/user-attachments/assets/aa9d0a12-2f93-4894-8349-52a013424939 +https://github.com/user-attachments/assets/3acfecf4-bf48-4c76-8245-2eeef0dedb1f -[Watch the full demo on YouTube](https://youtu.be/QqCahxGDdS0). +[![Android Quick Test cover showing a real device browser and FSQ execution evidence](docs/media/quick-test-poster.png)](https://youtu.be/f1sobvVHNg0) -## How it works +[Watch on YouTube](https://youtu.be/f1sobvVHNg0) · [English captions](docs/media/quick-test.en.srt) · [Media sources and review status](docs/media/quick-test-demo-provenance.md) -1. **Describe:** You define the platform and target, goal, success criteria, and allowed and forbidden actions. -2. **Observe** as AI tests the real UI flow and FSQ captures screenshots, interface state, events, and may produce a reviewable test. -3. **Control** what happens next: inspect the evidence and, if FSQ returns a test, inspect it unchanged before separately requesting any deterministic replay. +The video shows one real Android Quick Test: find the FSQ repository, follow its About website link, and verify both homepage headlines. You can start with Web; platform setup and execution differ, and search results and outcomes can vary. -You get an evidence-backed result and, when FSQ returns one, a portable, reusable test you can review, keep, and replay without surrendering control. +## From goal to evidence + +**Describe the journey and the checks.** Find `FSQ-lab/FSQ`, follow its About website link, and verify `Describe a goal.` and `Test the real app.`. The card adds editorial explanation around a recorded goal; the surrounding text is separate from the product UI. + +![Editorial task card with a real Quick Test goal: search for the repository, follow About to its website, and check both headlines](docs/media/quick-test-task.png) + +**Watch AI operate the real app.** The Android browser screen sits beside the FSQ action timeline, connecting execution with captured screenshots and interface state. + +![Real FSQ execution capture showing the action timeline beside the Android browser device view](docs/media/quick-test-execution.png) + +**Check the result against the evidence.** The captured page shows both required headlines. The same Run reports execution success, verification success, and complete evidence separately. The card pairs real page and report captures with editorial explanation. + +![Editorial result card pairing the captured homepage headlines with the same Run report showing execution and verification success and complete evidence](docs/media/quick-test-result.png) ## Quick start -Start with the real outcome you want to test. You need Python 3.11+ and the [prerequisites for your platform and target](docs/platform-prerequisites.md). Choose one of the two paths below. Enter Provider credentials only in the FSQ UI—not in Coding Agent chat. Review the result first and, if FSQ returns a Case, review it unchanged before separately requesting any replay. +Choose **Coding Agent** or **FSQ UI**. Both lead to one Quick Test and evidence review. You need Python 3.11+, a supported configured Provider and explicitly selected model, and [platform prerequisites](docs/platform-prerequisites.md). Enter Provider credentials privately in local FSQ Settings; never paste them into Coding Agent chat. + +Quick Test prepares `~/.fsq/quick-test/` automatically. It requires no manual project Workspace setup, does not save a formal Case, and does not perform Strict Replay. + +### Coding Agent + +Add the FSQ Skill, reload your Coding Agent, and ask it to run the complete goal below with FSQ Quick Test. + +
+Expand: one-time setup and first run -### 1. Coding Agent +Open your project in the Coding Agent and paste: -**Initial setup.** Open your project in a Coding Agent and paste: -> The official FSQ project is https://github.com/FSQ-lab/FSQ. Establish the exact opened project root. Batch the read-only Python, `fsq`, and Skill-presence checks with their exact working directories, commands, purpose, and effects. Then request one approval for the listed changes, covering only installation or upgrade of `fsq-agent` and one `fsq init --agent skill` in this project; disclose the exact commands and Python-environment and project-file effects before asking. Perform only the approved operations, verify `fsq`, and report the exact installed Skill path. This one-time setup does not initialize a Workspace, configure a Provider, or run a test. Tell me to reload or start a new Coding Agent session, then stop. +> The official FSQ project is https://github.com/FSQ-lab/FSQ. Establish the exact opened project root. Batch and disclose read-only Python, fsq, and Skill-presence checks with their working directories, commands, purpose, and effects. Request one approval covering only installation or upgrade of fsq-agent and one fsq init --agent skill in this project; show exact commands and Python-environment and project-file effects first. Perform only approved operations, verify fsq, and report the installed Skill path. This setup does not initialize a Workspace, configure a Provider, or run a test. Ask me to reload or start a new Coding Agent session, then stop. -Complete that reload before continuing. +After reloading, select Web and an available browser, then send “Use FSQ Quick Test for the following goal” with the complete goal. Follow the installed Skill: it discovers readiness, asks for unresolved decisions, and reports actual progress and evidence. After a package upgrade, rerun `fsq init --agent skill` and reload to update the copied Skill. -**First test.** Copy this prompt into your Coding Agent: -> Use FSQ to test this user goal: launch Microsoft Edge Stable, use Bing to search for `FSQ GitHub`, then inspect the organic search results and report the current position/ranking of the FSQ GitHub repository. Select the Web platform and Microsoft Edge Stable. Follow the installed FSQ Skill and ask only for decisions it cannot establish. +
-Treat the platform and exact target as facts or user-owned decisions: provide them when asked rather than letting the agent guess a target ID or path, and do not substitute a different Edge channel. +[Open Coding Agent setup guide](docs/getting-started.md#coding-agent) -Review the result first. If FSQ returns a Case, review it unchanged before deciding whether to request replay separately. +### FSQ UI -### 2. FSQ UI +Install FSQ, open the local UI, choose **Quick Test**, and paste the same goal. -**Initial setup.** +
+Expand: install and start Quick Test ```bash python -m pip install --upgrade fsq-agent fsq ui ``` -Choose **Quick Test** for exploration using the automatic `~/.fsq/quick-test/` Workspace, or **FSQ Case Manager** for Workspace/Case management, history, and Strict Replay. Both share **Settings**, where you configure the FSQ LLM Provider and explicitly select a model. +1. Configure a supported Provider and explicitly select a model in **Settings**. +2. Return to **Quick Test**, select Web and an available browser, and paste the goal below. +3. Select **Start Quick Test**. FSQ interprets the app and validates the target and environment before testing. Unresolved intent, zero or multiple target matches, or preparation failure stop the request without follow-up dialogue. +4. Use **Open report** to inspect the outcome and evidence. Target availability does not prove the model understood the intended app correctly. -1. Select the platform and, where applicable, the device or browser. -2. Describe what to do and verify. For Web, provide a URL or a clear search goal, for example: `Search FSQ on GitHub and find its official website.` You do not need to know the result URL beforehand. -3. Select **Start Quick Test**. The model identifies the application; FSQ validates the resolved target and environment before exploration. Unresolved intent, zero/multiple lookup results, and preparation failures stop the request without a follow-up dialogue. Target availability does not prove the model understood the application correctly. -4. Review the result and evidence using **Open report**, or choose **New test**. Quick Test does not save Cases; in Case Manager, review any generated Case unchanged before separately requesting replay. +
-![FSQ Test Runner for describing a test goal](docs/media/01-describe-goal.png) +[Open FSQ UI setup guide](docs/getting-started.md#fsq-ui) -[Detailed first-test guide](docs/getting-started.md) +### Try a complete user journey -## Platforms +Web is an accessible first-use option; the recording shows Android. After selecting a browser, copy this complete goal into either entry path: -FSQ supports Web, Android, Windows, and macOS targets; see [platform prerequisites](docs/platform-prerequisites.md), and help broaden support through [contributions](CONTRIBUTING.md). +```text +Use the selected browser to search for the FSQ-lab/FSQ GitHub repository. Open that repository, follow the website link in its About section, and verify that the destination page displays both “Describe a goal.” and “Test the real app.” Report the outcome with the captured page evidence. Only browse public pages. Do not sign in, change repository content, or submit forms. Stop and report a blocker if the repository or About website link cannot be identified. +``` + +After the run, inspect the report and captured page evidence. Treat failed, inconclusive, or preparation-error outcomes as observed; they do not mean the checks passed. + +## Go further when you need to + +- **Inspect the run:** review screenshots, interface state, events, and reports to compare actual behavior with your expectations. +- **Manage reusable Cases:** use **FSQ Case Manager** for registered Workspaces, explicit Case creation or saving an available candidate, and run history. A run may produce a Case; generation is conditional. +- **Choose deterministic replay:** review any returned Case unchanged, then separately request Strict Replay. Replay operates the target again and requires prerequisite and side-effect checks; Quick Test never runs it automatically. + +FSQ supports Web, Android, Windows, and macOS targets, subject to [platform prerequisites](docs/platform-prerequisites.md). + +## Help shape FSQ + +If FSQ is useful to you, [visit the repository and Star FSQ](https://github.com/FSQ-lab/FSQ) to help others discover it. Tried it? [Share your experience](https://github.com/FSQ-lab/FSQ/issues/new/choose): what worked, where you got stuck, and what you want to test next. + +Include the FSQ version, platform, sanitized goal, expected and actual outcome, and only reviewed diagnostic excerpts. Remove credentials, personal data, private URLs, paths, and device identifiers before sharing screenshots or logs. Report security issues through [Security](SECURITY.md). > [!IMPORTANT] -> FSQ is alpha software for evaluation and contribution. Evidence is stored locally and can contain visible application data or other private target data. Review it before sharing, and do not commit evidence or credentials. Public interfaces may evolve before 1.0; see [support and stability](docs/support-and-stability.md). +> FSQ is alpha software for evaluation and contribution. Evidence is stored locally and may contain private app data; review it before sharing, and do not commit evidence or credentials. AI operations send necessary provider-bound inputs to your configured Provider. Public interfaces may evolve before 1.0; see [support and stability](docs/support-and-stability.md). ## Documentation @@ -94,11 +130,9 @@ FSQ supports Web, Android, Windows, and macOS targets; see [platform prerequisit - [FSQ Case format](docs/case-format.md) - [Advanced Codex compatibility](docs/codex-integration.md) - [Platform prerequisites](docs/platform-prerequisites.md) -- [Support and stability](docs/support-and-stability.md) - -## Built by FSQ Lab and the community +- [Contribute](CONTRIBUTING.md) -Built by [FSQ Lab](https://github.com/FSQ-lab) with thanks to [all contributors](https://github.com/FSQ-lab/FSQ/graphs/contributors). Want to help? Read [CONTRIBUTING.md](CONTRIBUTING.md) and join us. +Built by [FSQ Lab](https://github.com/FSQ-lab) and the community, with thanks to [all contributors](https://github.com/FSQ-lab/FSQ/graphs/contributors). ## License diff --git a/README.zh-CN.md b/README.zh-CN.md index d0ea93f6..f91bd3a7 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -6,9 +6,9 @@

-

描述你要测试什么,让 AI 帮你测试用户流程。

+

描述测试目标,让 AI 测试真实应用。

-

描述测试目标,AI 替你操作真实 App。FSQ 记录执行证据,并把探索过程转化为可审查、可重放的测试。

+

用自然语言描述要做什么、应该发生什么。AI 操作真实应用并验证结果,FSQ 保留截图、操作和结果,供你检查。

来自 FSQ Lab 的开源项目。

@@ -20,70 +20,107 @@

+ 官网 · 快速开始 · English · 指南 · 参与贡献

-## 查看 FSQ 演示 +## 看一次完整测试 -https://github.com/user-attachments/assets/aa9d0a12-2f93-4894-8349-52a013424939 +https://github.com/user-attachments/assets/3acfecf4-bf48-4c76-8245-2eeef0dedb1f -[在 YouTube 观看完整演示](https://youtu.be/QqCahxGDdS0)。 +[![Android Quick Test 演示封面:真实设备浏览器和 FSQ 执行证据](docs/media/quick-test-poster.png)](https://youtu.be/f1sobvVHNg0) -## 工作原理 +[在 YouTube 观看](https://youtu.be/f1sobvVHNg0) · [英文字幕](docs/media/quick-test.en.srt) · [媒体来源与审查状态](docs/media/quick-test-demo-provenance.md) -1. **描述:**你定义平台和目标、测试目的、成功标准,以及允许和禁止的操作。 -2. **观察** AI 测试真实 UI 流程,FSQ 记录截图、界面状态和事件,并可能生成可审查的测试。 -3. **掌控**下一步:检查证据;如果 FSQ 返回了测试,再原样检查它,然后单独请求任何确定性重放。 +视频展示 Android 上的一次真实 Quick Test:搜索 FSQ 仓库,沿 About 官网链接访问主页,验证两句主标题。你可以先使用 Web 试用;不同平台的环境和执行过程不同,搜索结果和测试结果也可能变化。 -最终会得到带证据的结果;如果 FSQ 返回了测试,你还会得到可移植、可复用的测试,可供审查、保留和重放,控制权始终在你手中。 +## 从目标到证据 + +**描述完整旅程和检查点。** 找到 `FSQ-lab/FSQ`,进入 About 链接的官网,验证 `Describe a goal.` 和 `Test the real app.`。下图是在真实目标输入截图周围添加的说明,不是产品界面的一部分。 + +![任务说明卡:真实 Quick Test 输入包含搜索仓库、进入 About 官网链接、验证两句标题的完整目标](docs/media/quick-test-task.png) + +**观察 AI 操作真实应用。** Android 浏览器中的页面与 FSQ 操作时间线对应,截图和界面状态保留执行过程。 + +![真实执行截图:FSQ 操作时间线旁显示 Android 浏览器的设备画面](docs/media/quick-test-execution.png) + +**对照证据检查结果。** 页面证据展示两句标题;同次 Run 报告分别显示执行成功、验证成功、证据完整。说明卡将真实页面与报告截图并列,便于独立阅读。 + +![结果说明卡:捕获的官网两句标题,与同次 Run 中执行成功、验证成功和证据完整的报告对照](docs/media/quick-test-result.png) ## 快速开始 -从你真正想测试的用户结果开始。你需要 Python 3.11 或更高版本,并满足[平台和目标的前置条件](docs/platform-prerequisites.md)。请从以下两条路径中选择一条。Provider 凭据只能由你在 FSQ UI 中输入,不要发送到 Coding Agent 聊天中。先审查结果;如果 FSQ 返回了 Case,再原样审查它,然后另行决定是否请求重放。 +选择 **Coding Agent** 或 **FSQ UI**,两条路径都从一次 Quick Test 和结果审查开始。需要 Python 3.11+、已配置的受支持 Provider 与明确选择的模型,以及[平台前置条件](docs/platform-prerequisites.md)。Provider 凭据请在本地 FSQ 设置中输入,不要发送到 Coding Agent 聊天。 + +Quick Test 自动准备 `~/.fsq/quick-test/`,无需手动初始化项目 Workspace,不保存正式 Case,也不执行 Strict Replay。 + +### Coding Agent + +安装 FSQ Skill、重新加载后,请 Coding Agent 使用 FSQ Quick Test 执行下方完整目标。 + +
+展开:一次性安装与首次运行 -### 1. Coding Agent +在 Coding Agent 中打开你的项目,粘贴: -**初始设置。** 在 Coding Agent 中打开你的项目,然后粘贴: -> FSQ 的官方项目地址是 https://github.com/FSQ-lab/FSQ。先确定准确打开的项目根目录。把 Python、`fsq` 和 Skill 是否存在的只读检查批量披露,列出准确工作目录、命令、目的和影响。然后提出一次修改批准请求,且只涵盖安装或升级 `fsq-agent`,以及在本项目运行一次 `fsq init --agent skill`;请求前列出准确命令及其对 Python 环境和项目文件的影响。只执行获准操作,验证 `fsq`,并报告准确的 Skill 安装路径。此次一次性设置不会初始化 Workspace、配置 Provider 或运行测试。最后让我重新加载 Coding Agent 或启动新 session,然后停止。 +> FSQ 的官方项目地址是 https://github.com/FSQ-lab/FSQ。先确定准确打开的项目根目录。批量披露 Python、fsq 和 Skill 是否存在的只读检查,列出工作目录、命令、目的和影响。提出一次批准请求,只涵盖安装或升级 fsq-agent,以及在本项目运行一次 fsq init --agent skill;请求前列出准确命令及其对 Python 环境和项目文件的影响。只执行获准操作,验证 fsq,并报告 Skill 安装路径。此次设置不初始化 Workspace、不配置 Provider、不运行测试。最后让我重新加载 Coding Agent 或启动新 session,然后停止。 -完成重新加载后再继续。 +重新加载后,指定 Web 和可用的浏览器,发送“使用 FSQ Quick Test 测试下方目标”,附上完整目标。遵循已安装的 Skill;它会发现前置条件、询问未解决的决定,并报告实际进度和证据。包升级后需再次运行 `fsq init --agent skill` 并重新加载,才能更新复制到项目的 Skill。 -**首次测试。** 将这段提示词复制给你的 Coding Agent: -> 使用 FSQ 测试这个用户目标:启动 Microsoft Edge Stable,使用 Bing 搜索 `FSQ GitHub`,然后检查自然搜索结果,并报告 FSQ GitHub 仓库当前所在的位置/排名。选择 Web 平台和 Microsoft Edge Stable。遵循已安装的 FSQ Skill,只询问它无法确定的决定。 +
-平台和准确目标是事实或由用户决定的事项:当 Skill 询问时请明确提供,不要让 Agent 猜测目标 ID 或路径,也不要替换为其他 Edge 渠道。 +[打开 Coding Agent 详细指南](docs/getting-started.zh-CN.md#coding-agent) -先审查结果。如果 FSQ 返回了 Case,再原样审查它,然后另行决定是否请求重放。 +### FSQ UI -### 2. FSQ UI +安装并打开本地 UI,选择 **Quick Test**,粘贴相同目标。 -**初始设置。** +
+展开:安装与开始 Quick Test ```bash python -m pip install --upgrade fsq-agent fsq ui ``` -选择 **Quick Test**,自动使用 `~/.fsq/quick-test/` Workspace 进行探索;或选择 **FSQ Case Manager** 管理 Workspace、Case、历史运行及 Strict Replay。两个入口共用 **Settings**,在其中配置 FSQ LLM Provider 并显式选择模型。 +1. 在 **Settings** 配置受支持的 Provider,并明确选择模型。 +2. 返回 **Quick Test**,选择 Web 和可用浏览器,粘贴下方目标。 +3. 点击 **Start Quick Test**。FSQ 解析应用并验证目标和环境后开始测试;意图不明确、目标零匹配或多匹配、准备失败会终止请求,不补问。 +4. 通过 **Open report** 检查结果与证据。目标可运行不等于模型一定理解正确。 -1. 选择平台,以及适用的设备或浏览器。 -2. 描述要执行的操作和预期结果。Web 可以提供网址,也可以描述明确的搜索目标,例如:`搜索 FSQ github,并且帮我找到 FSQ 的官网地址。` 无需预先知道要找到的网址。 -3. 点击 **Start Quick Test**。模型识别应用,FSQ 验证解析出的目标和环境后才开始探索。意图不明确、目标查询零匹配或多匹配、准备失败时直接终止,不补问;目标可运行不等于已证明模型理解正确。 -4. 通过 **Open report** 审查结果和证据,或选择 **New test**。Quick Test 不保存 Case;在 Case Manager 中应先原样审查生成的 Case,再另行决定是否重放。 +
-![用于描述测试目标的 FSQ Test Runner](docs/media/01-describe-goal.png) +[打开 FSQ UI 详细指南](docs/getting-started.zh-CN.md#fsq-ui) -[详细首次测试指南](docs/getting-started.zh-CN.md) +### 试一次完整的用户旅程 -## 平台 +Web 是便于开始的试用选项;视频录制的是 Android。选择浏览器后,将以下完整目标复制到任一入口: -FSQ 支持 Web、Android、Windows 和 macOS 目标;请查看[平台前置条件](docs/platform-prerequisites.md),也欢迎通过[贡献](CONTRIBUTING.md)帮助扩展支持。 +```text +使用已选择的浏览器搜索 FSQ-lab/FSQ GitHub 仓库。打开该仓库,访问 About 区域中的官网链接,验证目标页面同时显示 “Describe a goal.” 和 “Test the real app.”,并结合捕获的页面证据报告结果。只浏览公开页面,不登录、不修改仓库内容、不提交表单。如果无法识别仓库或 About 官网链接,停止并报告阻碍。 +``` + +运行后,检查报告中的结果和页面证据。失败、无法确定或准备错误应按实际结果处理,不能当作检查已通过。 + +## 需要时再进阶 + +- **审查执行证据:**检查截图、界面状态、事件和报告,定位实际行为与预期的差异。 +- **管理可复用 Case:**在 **FSQ Case Manager** 中管理注册的 Workspace、显式创建或保存可用的候选 Case,并查看运行历史。Run 可能产生 Case,生成并不保证成功。 +- **另行选择确定性重放:**原样审查返回的 Case 后,再单独请求 Strict Replay。重放会再次操作目标,需检查前置条件与副作用;Quick Test 不会自动重放。 + +FSQ 支持 Web、Android、Windows 和 macOS 目标,请查看[平台前置条件](docs/platform-prerequisites.md)。 + +## 一起完善 FSQ + +如果 FSQ 对你有帮助,欢迎[前往 GitHub 仓库点击 Star](https://github.com/FSQ-lab/FSQ),让更多人发现它。试用后,欢迎[反馈体验](https://github.com/FSQ-lab/FSQ/issues/new/choose):什么有效、哪里卡住、下一步想测试什么。 + +反馈请提供 FSQ 版本、平台、脱敏目标、预期与实际结果,以及审查后的少量诊断信息。移除凭据、个人数据、私有网址、路径和设备标识,再分享截图或日志。安全问题请使用 [Security](SECURITY.md) 中的渠道。 > [!IMPORTANT] -> FSQ 是用于评估和贡献的 Alpha 软件。证据保存在本地,可能包含界面中可见的数据或其他私有目标数据;分享前请审查,不要提交证据或凭据。公开接口在 1.0 前仍可能调整,请阅读[支持与稳定性说明](docs/support-and-stability.md)。 +> FSQ 是用于评估和贡献的 Alpha 软件。证据保存在本地,可能包含私有应用数据,分享前请审查,不要提交证据或凭据。AI 操作会将必要的 Provider 输入发送到你配置的 Provider。公开接口在 1.0 前可能变化,请阅读[支持与稳定性](docs/support-and-stability.md)。 ## 文档 @@ -93,11 +130,9 @@ FSQ 支持 Web、Android、Windows 和 macOS 目标;请查看[平台前置条 - [FSQ Case format](docs/case-format.md) - [Codex 高级兼容性](docs/codex-integration.md) - [平台前置条件](docs/platform-prerequisites.md) -- [支持与稳定性](docs/support-and-stability.md) - -## 由 FSQ Lab 与社区共同打造 +- [参与贡献](CONTRIBUTING.md) -由 [FSQ Lab](https://github.com/FSQ-lab) 与社区共同打造,感谢[所有贡献者](https://github.com/FSQ-lab/FSQ/graphs/contributors)。想加入我们?请阅读 [CONTRIBUTING.md](CONTRIBUTING.md) 并参与贡献。 +由 [FSQ Lab](https://github.com/FSQ-lab) 与社区共同打造,感谢[所有贡献者](https://github.com/FSQ-lab/FSQ/graphs/contributors)。 ## 许可证 diff --git a/SPEC.md b/SPEC.md index ab199fa5..4aa32b58 100644 --- a/SPEC.md +++ b/SPEC.md @@ -146,7 +146,7 @@ Run provenance contains sanitized source snapshots, producer versions, content d `fsq runs export` produces public JSON, JUnit, portable HTML, or an evidence bundle without browser interaction. Report owns `fsq.report/v1`, deterministic comparisons, and exporters; Application authorizes Workspace/Run/relationship/artifact/destination scope; adapters present results. Control Plane Runs exposes the same persisted reports at stable Workspace/platform/Run/step addresses independently of live request state. Execution, verification, evidence, and processing statuses remain distinct. JUnit uses one root Case/Goal invocation per testcase, not one tool call. Export or upload failure never replaces the original test exit status. -Public demonstration content is an inspectable static export of real public-target Explore, reviewed Case, Strict baseline, and changed-target failure evidence. Captures and replay outcomes are not fabricated; candidate, reviewed Case, screenshot playback, and deterministic execution remain distinct. Portable reports embed their displayed evidence. Explicit sharing creates separate selected/redacted/masked copies with integrity and transformation inventories, never changes local evidence, and does not claim automatic anonymity. Public hosting is separate from local generation and never exposes the Control Plane server. +Public demonstration content uses real public-target execution evidence. Evidence exports can cover Explore, reviewed Case, Strict baseline, and changed-target failure; an Android Quick Test demonstration shows goal-based exploration, verification, and report inspection without implying Case publication or Strict Replay. Captures and replay outcomes are not fabricated; candidate, reviewed Case, screenshot playback, and deterministic execution remain distinct. Portable reports embed their displayed evidence. Explicit sharing creates separate selected/redacted/masked copies with integrity and transformation inventories, never changes local evidence, and does not claim automatic anonymity. Public hosting is separate from local generation and never exposes the Control Plane server. Local data produced by v0.1.0 and subsequent releases remains usable through existing built-in CLI and Control Plane workflows. Historical reports, events, source markers, errors, evidence, and artifact paths retain their original contents; inspection does not rewrite them or re-evaluate the Run. Current dynamic provenance uses `agent_runtime.runner` and `agent_runtime.verifier`, and generic runtime failures use `agent_runtime_error`. Historical values coexist with current values as persisted data; saved artifact paths are resolved directly, not reconstructed from current labels. Python compatibility aliases and external scripts matching old output strings are not part of this local-data contract. @@ -234,6 +234,20 @@ Optional queries use only an exact persistent Run ID, platform, and registered r The bundled `assets/getting-started.web.fsq.yaml` is a canonical provider-free troubleshooting Case, not a prerequisite for the requested goal. It opens package-owned inline HTML, checks the `FSQ Web Strict Replay` heading, and closes the configured browser with no AI assertion, runtime-secret reference, nested Case, or shell hook. Explicit use retains static/Doctor checks, one execution, exact Run lookup, unchanged YAML, and no automatic retry. +## Public Product Presentation + +The English and Chinese READMEs and the public Website introduce the same goal-driven alpha workflow: describe a test goal, let AI operate the real application and verify the outcome, then inspect the recorded evidence. The Website retains the headline `Describe a goal. Test the real app.` and its light, white-and-green visual system. Broad AI-era automation positioning is secondary to the concrete user outcome. + +The first-use journey is Quick Test through either Coding Agent or FSQ UI as equal alternatives. Concise entry instructions precede expandable setup details and authoritative guides. Python 3.11+, a supported configured Provider and selected model, and platform-specific prerequisites are visible before execution. Credentials remain outside Coding Agent chat. Default Quick Test requires no manually initialized project Workspace and does not save a formal Case or perform Strict Replay. + +The shared first-test example uses a selected browser to search for the `FSQ-lab/FSQ` GitHub repository, follow its About website link, and verify both `Describe a goal.` and `Test the real app.` on the destination page. Web is an accessible first-use option; the recorded demonstration shows Android. Search ranking, speed, universal success, and equivalence across platforms are not promised. Supporting screenshots explain the task, real execution, and verification evidence without relying on video narration. Editorial explanations remain distinguishable from captured product UI. + +Current promotional surfaces use the approved Android Quick Test demonstration and matching screenshots, cover, captions, and source metadata. Active standalone demo pages and media documentation share those references. Historical release records retain their version-specific evidence and are not presented as the current demonstration. Large videos remain outside Git history. Source versions, capture dates, runtime conditions, and privacy-review results are recorded only when established; an edit date or current repository HEAD does not establish recording provenance. + +Progressive capability descriptions distinguish Quick Test evidence review from Case Manager's explicit Case management and separate Strict Replay. Star and experience-feedback links follow the trial instructions and lead to the canonical GitHub repository; they neither gate testing nor perform a Star or submit feedback for the visitor. Feedback guidance requests only sanitized diagnostic information. The Website remains English-first with a visible Chinese getting-started destination, and both READMEs provide equivalent journeys. + +Publication readiness distinguishes a prepared source change from an obtainable package, successful clean-environment first-use validation, deployed Website, and verified canonical public page. Missing external publication or live first-use evidence remains explicit rather than being inferred from a build or unit test. + ## Module Table | Module | SPEC | Purpose | diff --git a/docs/assets/social-preview.png b/docs/assets/social-preview.png index d2622e14..9d0c224b 100644 Binary files a/docs/assets/social-preview.png and b/docs/assets/social-preview.png differ diff --git a/docs/demo.html b/docs/demo.html index 0d17659c..866ebf19 100644 --- a/docs/demo.html +++ b/docs/demo.html @@ -1,119 +1,75 @@ - - - - FSQ v0.1.0 Demo - - - - -
-
-

FSQ v0.1.0 Demo

-

Evidence-first AI UI automation: describe a goal, inspect evidence, review a candidate Case, replay deterministically, and open the static report.

-
- -
- -
- - -
- + + + + FSQ — Android Quick Test demonstration + + + + +
+

Describe a goal. Test the real app.

+

One Android Quick Test: find FSQ-lab/FSQ, follow its About website link, and verify both homepage headlines. Review the recorded outcome and page evidence.

+
+ +
+

+ +

Alpha software. Evidence is local and may contain private app data; review it before sharing. AI operations send necessary provider-bound inputs to your configured Provider. This page does not run tests or collect credentials.

+
+ + diff --git a/docs/getting-started.md b/docs/getting-started.md index 3798fc7e..50e04d2a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,132 +1,58 @@ # Getting Started -This guide uses the package-owned Skill for a guided, platform-neutral AI -Explore, with optional review and user-initiated deterministic replay. FSQ is alpha -software; review [support and stability](support-and-stability.md) before -production adoption. +Describe a goal, let FSQ Quick Test operate the real app and verify the outcome, then inspect the evidence. **Coding Agent** and **FSQ UI** are equal alternatives. FSQ is alpha software; read [support and stability](support-and-stability.md). ## Prerequisites -- Python 3.11 or newer. -- The prerequisites for the platform and target you want to test. -- A directory you can open as the exact local Coding Agent project root. - -AI exploration and suggestion analysis also require OpenAI, DeepSeek, Google Gemini, Kimi, GitHub Copilot, or Azure OpenAI. Deterministic Case replay does not require a planning LLM unless the authored Case contains an AI assertion. - -## Install the CLI and project-local Skill - -Open the project in your Coding Agent and use this one-time setup: - -> The official FSQ project is https://github.com/FSQ-lab/FSQ. Establish the -> exact opened project root. Batch and disclose read-only Python, `fsq`, and -> Skill-presence checks. Then request one approval covering only -> `python -m pip install --upgrade fsq-agent` and one -> `fsq init --agent skill` from that project root, with exact commands and -> effects. Perform only approved operations, verify `fsq`, and report the exact -> installed Skill path. This does not initialize a Workspace, configure a -> Provider, or run a test. Tell me to reload or start a new Coding Agent -> session, then stop. - -Reload or start a new Coding Agent session so the client discovers -`.agents/skills/fsq/`. After a package upgrade, rerun -`fsq init --agent skill` and reload again to update the copied Skill. - -Then state your actual goal concisely: - -> Test this user goal with FSQ: **[describe the outcome]**. Follow the installed -> FSQ Skill and ask only for decisions it cannot establish. - -Examples—not defaults—include: Android, “a signed-out user sees a clear error -for invalid sign-in”; Web, “a shopper can filter results and open one product”; -Windows, “a user can create and save a draft”; macOS, “a user can change a -preference and see it persist.” Do not guess a target ID or path. - -## Guided AI Explore - -The installed Skill leads the dialogue and uses the public FSQ CLI. It will: - -1. Prefer the exact opened project root established by the host; if ambiguous, - treat the current directory only as a candidate and ask you to resolve it. - From that root it runs Doctor before asking for facts Doctor can provide, - including the platform and exact current target. -2. Reuse a valid registered Workspace at the project root. Otherwise inspect - only the exact `/fsq_workspace`: reuse it if it is a valid - registered Workspace; initialize an existing empty ordinary directory; or, - after approval, create one empty absent directory with the host's native - filesystem capability and initialize it. -3. Block on a symlink, non-directory, unreadable/indeterminate path, or non-empty - invalid directory. It never searches broadly, creates a nested Workspace, - follows or replaces a symlink, or overwrites, deletes, moves, or migrates - content. It does not prescribe `mkdir` or assume a shell. -4. Before creation or initialization, show one cohesive recommendation with - exact project and Workspace roots, platform, exact target, fact sources, - candidates, exact existing `fsq init` argv, effects, and unauthorized - actions. One approval covers the exact directory creation, when needed, and - one initialization. It then verifies the actual returned root with Doctor. -5. Keep FSQ facts, host facts, user decisions, and unverified model candidates - distinct. Candidate app IDs, browser targets, executable paths, bundle IDs, - devices, or accounts never enter argv until their exact value is confirmed. -6. Ask for the smallest cohesive group of unresolved user-owned decisions. - It may propose Goal criteria, actions, and restrictions, but automatic - defaults only reduce authority or stop on ambiguity. It never silently - selects a target, account, recipient, exact content, amount, device, or - irreversible action, and shows exact literals and side effects in one - complete Goal before a separate create confirmation. Initialization, - creation, and replay permissions remain separate. - -Related read-only checks use one compact disclosure of their exact working -directories, commands, purpose, and read-only effect; ordinary read-only -commands do not each require another approval. Ambiguous scope or an unmet -prerequisite blocks rather than being silently defaulted. - -### Provider handoff - -If AI Explore needs Provider setup, this is a user-controlled handoff. API -keys, tokens, and authentication secrets must not enter chat. The Coding Agent -asks you to open `fsq ui` yourself and enter credentials in **Settings**; the -private interactive `fsq providers configure ""` command is the -fallback. After you report completion, it checks Provider status and Doctor readiness -again, then resumes the same Workspace, target, and Goal draft without -repeating settled questions unless facts changed. Readiness is not a connection -test or a successful execution. - -### Create, review, and optionally replay - -After showing the complete confirmed Goal, the Skill may submit one approved -AI Explore with `fsq --output json --non-interactive case create ...`. It starts -once, retains the same host process/session handle, waits for exactly one -terminal result/error envelope, and does not retry an unknown, interrupted, -failed, malformed, multiple, or contradictory result. Submission consumes that -confirmation and is reported once. At recurring host opportunities, the Skill -targets a low-noise heartbeat about every 20 seconds, never more frequently, -stating only that the host process remains active and no terminal FSQ result -exists. This is best-effort, not an exact timer; hosts without recurring -model-visible opportunities cannot provide it. The Skill does not expose raw -machine output, infer FSQ stages or percentages, or claim real-time progress. - -The Skill summarizes the authoritative execution result first and maps your -observable acceptance criteria to supported, unmet, or unknown evidence before -separately reporting recording, publication, and warnings. It selects only the -exact returned Case path: `published_case_path`, or otherwise -`candidate_case_path` marked as unpublished. - -It reviews that exact returned Case, unchanged and read-only, only if you ask. -Replay is also initiated only by your request. After that request, it runs -read-only Doctor and static prechecks, including: +- Python 3.11+ and [prerequisites for the chosen platform and target](platform-prerequisites.md). +- A supported configured Provider (OpenAI, DeepSeek, Google Gemini, Kimi, GitHub Copilot, or Azure OpenAI) and explicitly selected model. +- For Coding Agent setup, an exact project root in which to install the Skill. + +Quick Test prepares the user-level `~/.fsq/quick-test/` Workspace automatically. No registered project Workspace, manual project initialization, or project Doctor is required. It does not inherit the opened project's knowledge, private environment, or credentials; put necessary non-secret project facts in the goal. It does not save a formal Case or automatically replay one. + +## Coding Agent + +Open your project in the Coding Agent and use this one-time setup: + +> The official FSQ project is https://github.com/FSQ-lab/FSQ. Establish the exact opened project root. Batch and disclose read-only Python, fsq, and Skill-presence checks with their working directories, commands, purpose, and effects. Request one approval covering only installation or upgrade of fsq-agent and one fsq init --agent skill in this project; show exact commands and Python-environment and project-file effects first. Perform only approved operations, verify fsq, and report the installed Skill path. This setup does not initialize a Workspace, configure a Provider, or run a test. Ask me to reload or start a new Coding Agent session, then stop. + +Reload or start a new session so the client discovers `.agents/skills/fsq/`. After upgrading the package, rerun `fsq init --agent skill` and reload to update the copied Skill. + +Then ask: “Use FSQ Quick Test for the goal below, with Web and my confirmed available browser. Follow the installed FSQ Skill and ask only for decisions it cannot establish.” Include the complete goal below. + +The Skill discloses one batch of read-only discovery and uses `quick-test context` to inspect the platform, device or browser, and prerequisites. It preserves confirmed exact targets without guessing or substituting browser channels, device IDs, or paths. It asks only for unresolved decisions that affect execution. Your test request authorizes one normal Quick Test without another FSQ-layer run confirmation; host and operating-system approvals remain independent. + +If Provider setup is needed, open `fsq ui` yourself and enter credentials privately in **Settings**, selecting a model, or use private interactive Provider configuration. Never put keys, tokens, or secrets in chat, visible command arguments, goals, or Cases. After you report completion, the Skill checks safe status and refreshes context for the same target. Readiness is not a connection test or successful execution. + +The Skill submits one `quick-test run`, consumes actual JSONL progress and a terminal result, and retains the same process. It never automatically resubmits after failure, timeout, interruption, lost output, or malformed output. It reports authoritative execution status, whole-goal verification, actual target, and available evidence. Mapping individual criteria to supported, unmet, or unknown is the Coding Agent's interpretation, not multiple independent FSQ verdicts. + +## FSQ UI ```bash -fsq case format "" --check --json +python -m pip install --upgrade fsq-agent +fsq ui ``` -It then shows the exact path and target and strongly warns that replay is a new -target operation that may repeat side effects. Known failure/recovery or -repeated corrective mutation is included as risk, but does not automatically -rewrite or refuse replay. If you still want replay after that warning, the -Skill restates the residual risk and requires a second explicit confirmation. -Only after that confirmation may it submit the unchanged Case once with -`fsq --output json --non-interactive case test --platform "" -""`, without suggestions. Creating a Case never pre-approves -replay, and Explore success does not prove deterministic replay success. +1. Choose **Quick Test**. +2. Configure a Provider and explicitly select a model in shared **Settings**. +3. Select Web and an available browser, paste the goal below, and choose **Start Quick Test**. You do not need to know the website URL the search will discover. +4. Use **Open report** to inspect the result and evidence, or choose **New test**. Unresolved intent, zero or multiple target matches, and preparation failures stop without follow-up dialogue. Target availability does not prove the model understood the intended app. + +## Complete first test + +Web is an accessible starting option; the [current video](https://youtu.be/f1sobvVHNg0) shows Android. Setup and execution differ, and search results and outcomes can vary. + +```text +Use the selected browser to search for the FSQ-lab/FSQ GitHub repository. Open that repository, follow the website link in its About section, and verify that the destination page displays both “Describe a goal.” and “Test the real app.” Report the outcome with the captured page evidence. Only browse public pages. Do not sign in, change repository content, or submit forms. Stop and report a blocker if the repository or About website link cannot be identified. +``` + +## Review results before choosing advanced workflows + +Inspect actual status, captured page evidence, Run ID, Workspace, and available report paths. Preparation failure may have no Run; missing evidence remains unknown. Any Quick Test recording remains Run-local and unpublished; Quick Test does not save a formal Case. + +For reusable Cases, separately choose **FSQ Case Manager** or explicitly request Case creation. This is another UI execution with registered Workspace and Doctor checks, not saving the completed Quick Test. Reviewable content exists only when a candidate is actually generated. + +For replay, first inspect the exact returned Case unchanged, then request replay separately. The Skill performs static and Doctor checks and discloses the exact Case, target, and repeated side effects. Only your explicit confirmation after that disclosure authorizes one Strict Replay. Quick Test success does not prove replay success; the Skill does not automatically rewrite Cases or retry. ## Provider-free Web troubleshooting fallback diff --git a/docs/getting-started.zh-CN.md b/docs/getting-started.zh-CN.md index e8ccadf2..9d19665b 100644 --- a/docs/getting-started.zh-CN.md +++ b/docs/getting-started.zh-CN.md @@ -1,112 +1,58 @@ # 快速开始 -这份指南使用包内置 Skill,引导你完成平台中立的 AI Explore,并可按需审查 -或由用户发起确定性重放。FSQ 是 Alpha 软件;生产采用前请先阅读 -[支持与稳定性说明](support-and-stability.md)。 +用自然语言描述目标,让 FSQ Quick Test 操作真实应用并验证结果,然后检查执行证据。**Coding Agent** 与 **FSQ UI** 是平等的替代入口。FSQ 是 Alpha 软件,请先阅读[支持与稳定性](support-and-stability.md)。 ## 前置条件 -- Python 3.11 或更新版本。 -- 满足要测试的平台和目标的前置条件。 -- 一个可作为准确本地 Coding Agent 项目根目录打开的目录。 - -AI 探索和 suggestion 分析还需要 OpenAI、DeepSeek、Google Gemini、Kimi、GitHub Copilot 或 Azure OpenAI。确定性 Case 重放不需要规划 LLM,除非已编写的 Case 包含 AI assertion。 - -## 安装 CLI 和项目本地 Skill - -在 Coding Agent 中打开项目,并使用这段一次性设置: - -> FSQ 的官方项目地址是 https://github.com/FSQ-lab/FSQ。先确定准确打开的项目 -> 根目录。批量披露 Python、`fsq` 和 Skill 是否存在的只读检查。然后请求一次 -> 批准,且只涵盖 `python -m pip install --upgrade fsq-agent`,以及从该项目 -> 根目录运行一次 `fsq init --agent skill`;列出准确命令和影响。只执行获准 -> 操作,验证 `fsq`,并报告准确的 Skill 安装路径。这不会初始化 Workspace、 -> 配置 Provider 或运行测试。最后让我重新加载或启动新的 Coding Agent -> session,然后停止。 - -重新加载或启动新的 Coding Agent session,让客户端发现 -`.agents/skills/fsq/`。升级包后,应再次运行 `fsq init --agent skill` 并再次 -重载,以更新已复制的 Skill。 - -然后简洁描述你的真实目标: - -> 测试这个用户目标:**[描述期望结果]**。遵循已安装的 FSQ Skill,只询问它 -> 无法确定的决定。 - -以下只是示例而非默认值:Android,“未登录用户输入无效信息时会看到清晰 -错误”;Web,“购物者可以筛选结果并打开一个商品”;Windows,“用户可以创建 -并保存草稿”;macOS,“用户可以修改偏好设置并看到它保持生效”。不要猜测目标 -ID 或路径。 - -## 引导式 AI Explore - -安装后的 Skill 负责对话引导,并使用公开 FSQ CLI。它会: - -1. 优先使用 host 确定的准确打开的项目根目录;若有歧义,只把当前目录当作 - 候选并请你解决。它从该 root 先运行 Doctor,再询问 Doctor 无法提供的 - 信息,包括平台和准确的当前目标。 -2. 复用项目 root 上有效注册的 Workspace;否则只检查准确的 - `/fsq_workspace`:若为有效注册 Workspace 则复用;若为现有 - 空普通目录则初始化;若路径不存在,则在批准后用 host 原生文件系统能力 - 创建一个空目录并初始化。 -3. 若该路径是 symlink、非目录、不可读/状态不确定,或非空且无效,则阻止 - 初始化。它不会广泛搜索、创建嵌套 Workspace、跟随或替换 symlink,也不会 - 覆盖、删除、移动或迁移内容;不会规定 `mkdir` 或假设存在 shell。 -4. 创建目录或初始化前,它会提出一次内聚建议,列出准确的项目和 Workspace - root、平台、准确目标、事实来源、候选值、现有 `fsq init` 的准确 argv、 - 影响及仍未授权的操作。一次批准覆盖所需的准确目录创建和一次初始化;之后 - 用 Doctor 验证实际返回的 root。 -5. 明确区分 FSQ 事实、host 事实、用户决定和未经验证的模型候选。候选 app - ID、浏览器目标、可执行文件路径、bundle ID、设备或账号,必须先确认准确值 - 才能进入 argv。 -6. 只询问最小且内聚的一组尚未解决的用户决定。Skill 可以建议 Goal 标准、 - 操作和限制,但自动默认值只能缩小权限或在歧义时停止;不会静默选择目标、 - 账号、接收者、准确内容、金额、设备或不可逆操作。它会在一次完整 Goal 中 - 展示准确字面量和副作用,再单独请求 create 确认。 - -相关只读检查会一次紧凑披露准确工作目录、命令、目的和只读影响;普通只读 -命令不需要逐条再次批准。范围不明确或前置条件未满足时会阻止流程,而不是 -静默使用默认值。 - -### Provider 交接 - -如果 AI Explore 需要设置 Provider,这一步由用户控制。API Key、token 和认证 -secret 不得进入聊天。Coding Agent 会请你自行打开 `fsq ui`,在 **Settings** -中输入凭据;私下交互式运行 `fsq providers configure ""` 是后备方式。 -你报告完成后,它会再次检查 Provider status 和 Doctor readiness,然后恢复 -同一个 Workspace、目标和 Goal 草稿;除非事实变化,否则不会重复已确定的 -问题。readiness 不等于连接测试或成功执行。 - -### 创建、审查并选择是否重放 - -展示完整且已确认的 Goal 后,Skill 可以用 -`fsq --output json --non-interactive case create ...` 提交一次获准的 AI -Explore。它只启动一次,保留同一个 host 进程/session handle,等待且只接受 -一个终态 result/error envelope;对于结果未知、中断、失败、格式错误、多份 -终态或相互矛盾的执行都不会重试。提交开始即消耗该确认,且只报告一次提交。 -在 host 提供重复的模型可见机会时,Skill 以约每 20 秒一次且绝不更频繁为目标, -只低噪声说明 host 进程仍在运行且尚无 FSQ 终态结果。这是尽力而为而非精确 -计时器;没有重复模型可见机会的 host 无法提供该心跳。Skill 不会暴露原始机器 -输出、推断 FSQ 阶段或百分比,也不会声称提供实时进度。 - -Skill 会先总结结果,把每项可观察验收标准映射为有支持、未满足或未知,然后 -分别说明记录、发布和 warning。随后只选择准确返回的 Case 路径:优先 -`published_case_path`,否则使用 `candidate_case_path` 并标明尚未发布。 - -它仅在你提出请求时以只读方式原样审查该 Case。重放也只能由你主动发起。 -收到请求后,它会执行只读 Doctor 和静态预检查,包括: +- Python 3.11+ 和[对应平台与目标的前置条件](platform-prerequisites.md)。 +- 已配置受支持的 Provider(OpenAI、DeepSeek、Google Gemini、Kimi、GitHub Copilot 或 Azure OpenAI),并明确选择模型。 +- 使用 Coding Agent 时,打开一个准确的项目根目录以安装 Skill。 + +Quick Test 自动准备用户级 `~/.fsq/quick-test/` Workspace,无需注册或初始化项目 Workspace,也无需先运行项目 Doctor。它不继承项目的知识、私有环境或凭据;必要的非敏感项目事实应写入目标。它不保存正式 Case,也不自动重放。 + +## Coding Agent + +在 Coding Agent 中打开项目,用以下提示完成一次性安装: + +> FSQ 的官方项目地址是 https://github.com/FSQ-lab/FSQ。先确定准确打开的项目根目录。批量披露 Python、fsq 和 Skill 是否存在的只读检查,列出工作目录、命令、目的和影响。提出一次批准请求,只涵盖安装或升级 fsq-agent,以及在本项目运行一次 fsq init --agent skill;请求前列出准确命令及其对 Python 环境和项目文件的影响。只执行获准操作,验证 fsq,并报告 Skill 安装路径。此次设置不初始化 Workspace、不配置 Provider、不运行测试。最后让我重新加载 Coding Agent 或启动新 session,然后停止。 + +重新加载或开启新 session,让客户端发现 `.agents/skills/fsq/`。升级包后需重新运行 `fsq init --agent skill` 并重载,以更新复制的 Skill。 + +然后说:“使用 FSQ Quick Test 测试下方目标,选择 Web 和我确认的可用浏览器。遵循已安装的 FSQ Skill,只询问无法确定的决定。”附上下方完整目标。 + +Skill 批量披露只读发现,使用 `quick-test context` 检查平台、设备或浏览器及前置条件;它保留已确认的准确目标,不猜测或替换浏览器渠道、设备 ID 或路径。只询问影响执行的未解决决定。你的测试请求授权一次正常 Quick Test,无需另一层 FSQ 执行确认;host 与操作系统批准仍独立生效。 + +若 Provider 尚未配置,请自行打开 `fsq ui` 在 **Settings** 私下输入凭据并选模型,或使用私有交互式 Provider 配置。不要将 Key、token 或 secret 放进聊天、命令参数、目标或 Case。报告配置完成后,Skill 检查安全状态并刷新同一目标的 Quick Test context;ready 不等于连接测试或执行成功。 + +Skill 只提交一次 `quick-test run`,消费实际 JSONL 进度和终态结果,保留同一进程。失败、超时、中断、输出丢失或格式错误时不会自动重提。报告权威执行状态、整体验证结论、实际目标和已有证据;逐项标准的支持/未满足/未知是 Agent 的证据解读,不是多个独立 FSQ verdict。 + +## FSQ UI ```bash -fsq case format "" --check --json +python -m pip install --upgrade fsq-agent +fsq ui ``` -随后它会显示准确路径和目标,并强烈警告重放是新的目标操作,可能重复副作用。 -已知的失败/恢复或重复纠正 mutation 会作为风险说明,但不会自动改写或拒绝 -重放。如果你在警告后仍想重放,Skill 会重申剩余风险并要求第二次明确确认。 -只有在该确认后,Skill 才能用 -`fsq --output json --non-interactive case test --platform "" -""` 原样提交一次,且不启用 suggestion。创建 Case 不会预先 -授权重放,Explore 成功也不代表确定性重放会成功。 +1. 选择 **Quick Test**。 +2. 在共享的 **Settings** 配置 Provider 并明确选模型。 +3. 选择 Web 与可用浏览器,粘贴下方目标,点击 **Start Quick Test**。无需预先知道搜索发现的官网网址。 +4. 使用 **Open report** 检查结果和证据,或选择 **New test**。意图不明、目标零匹配或多匹配、准备失败会终止请求,不补问。目标可运行不等于模型一定理解正确。 + +## 完整首次测试 + +Web 便于开始;[当前视频](https://youtu.be/f1sobvVHNg0) 展示 Android。两者设置和执行不同,搜索结果和测试结果可能变化。 + +```text +使用已选择的浏览器搜索 FSQ-lab/FSQ GitHub 仓库。打开该仓库,访问 About 区域中的官网链接,验证目标页面同时显示 “Describe a goal.” 和 “Test the real app.”,并结合捕获的页面证据报告结果。只浏览公开页面,不登录、不修改仓库内容、不提交表单。如果无法识别仓库或 About 官网链接,停止并报告阻碍。 +``` + +## 检查结果,再选择进阶操作 + +先检查真实状态、页面证据、Run ID、Workspace 和报告路径。准备失败可能没有 Run;缺失证据保持未知。Quick Test 的候选记录只留在 Run 内,不发布或保存正式 Case。 + +需要可复用 Case 时,另行使用 **FSQ Case Manager** 或明确请求 Case 创建。这是单独的 UI 执行,使用注册 Workspace 与相应 Doctor 检查,并非保存已经完成的 Quick Test。仅当实际生成候选时才有可审查内容。 + +需要重放时,先原样审查准确返回的 Case,然后另行请求。Skill 执行静态检查与 Doctor,披露准确 Case、目标及重复副作用;你明确确认后才执行一次 Strict Replay。Quick Test 成功不证明重放成功,不自动修改 Case 或重试。 ## 无需 Provider 的 Web 故障排查后备流程 diff --git a/docs/media/README.md b/docs/media/README.md index dab0cfa0..b48fe50e 100644 --- a/docs/media/README.md +++ b/docs/media/README.md @@ -1,22 +1,23 @@ -# Release Media +# Public demonstration media -This directory owns the public demo media, product screenshots, subtitles, naming rules, privacy review, and new-user acceptance. +The current promotional demonstration is **Android Quick Test**: find the FSQ-lab/FSQ repository, follow its About website link, and verify both homepage headlines. It shows goal entry, real-app execution, verification, and evidence review. It does not demonstrate a guaranteed Case publication or Strict Replay. -The README first screen uses `fsq-control-plane-demo.gif`, a 20-second silent tour captured from the current Control Plane UI. It shows the real `Goal → Evidence → Candidate YAML → Strict Replay → Runs report → Portable HTML` workflow. The existing v0.1.0 full demo remains available through GitHub media attachment and YouTube. +- [GitHub-hosted video](https://github.com/user-attachments/assets/3acfecf4-bf48-4c76-8245-2eeef0dedb1f) +- [YouTube video](https://youtu.be/f1sobvVHNg0) +- `quick-test-poster.png`: full-frame recorded execution cover, 1920×1080. +- `quick-test-task.png`: editorial task explanation containing a real goal-input crop, 1800×1040. +- `quick-test-execution.png`: real execution crop, 1593×705. +- `quick-test-result.png`: editorial explanation pairing real captured page and Run-report crops, 1800×1040. +- `quick-test.en.srt`: English captions supplied with the new edit. +- `quick-test.en.vtt`: the same cues converted to WebVTT for the native browser player; old subtitle files are not used for this edit. +- [Source identities and known/unknown provenance](quick-test-demo-provenance.md) +- [Observed acceptance and pending verification](media-acceptance.md) +- [Current sequence](demo-storyboard.md) -The current-UI tour is delivered as a lightweight GIF. A separate live-demo video can be recorded and hosted later without committing large video files to Git history. Original local captures are intentionally not committed. +The READMEs link the GitHub attachment and matching cover; the Website and [standalone demo](../demo.html) activate native playback only on a click. Both retain explicit GitHub and YouTube viewing links. The Website bundles captions and images on the Pages origin. No large video is committed. -## Naming +The share image `../assets/social-preview.png` uses the same white/green identity and the headline “Describe a goal. Test the real app.”. Its byte-identical Website publication copy is `frontend/website/public/social-preview.png`; the stable social metadata URL is `https://fsq-lab.github.io/FSQ/social-preview.png`. -- `fsq-v0.1.0-demo-thumbnail.png` — linked video thumbnail; -- `fsq-v0.1.0-android-demo-preview.gif` — lightweight README animation built from approved product screenshots; -- `fsq-control-plane-demo.gif` — current Control Plane README first-screen tour; -- `https://github.com/user-attachments/assets/aa9d0a12-2f93-4894-8349-52a013424939` — GitHub README media preview; -- `01-describe-goal.png` — goal entry and setup; -- `02-execute-workflow.png` — execution timeline; -- `03-capture-evidence.png` — captured evidence; -- `04-generate-candidate.png` — Run-local candidate Case; -- `05-inspect-run.png` — static report inspection; -- `demo.en.srt` and `demo.zh-CN.srt` — subtitles. +## Historical assets -Do not commit empty placeholder media files. README references are added only when the corresponding approved artifact exists. Large video files stay outside Git history; the GIF is a short preview, not the canonical full demo. +The versioned thumbnail and Android GIF, `fsq-control-plane-demo.gif`, its [provenance](fsq-control-plane-demo-provenance.md), numbered screenshots `01-describe-goal.png` through `05-inspect-run.png`, and `demo.en.srt` / `demo.zh-CN.srt` remain historical evidence for existing release references. They are not the current promotional demonstration. Retaining them does not validate them against current runtime behavior. diff --git a/docs/media/demo-storyboard.md b/docs/media/demo-storyboard.md index 23eecc67..c033b0bb 100644 --- a/docs/media/demo-storyboard.md +++ b/docs/media/demo-storyboard.md @@ -1,47 +1,18 @@ -# FSQ README Demo Storyboard +# Android Quick Test demonstration sequence -## Deliverable +The current approved edit is 130.708333 seconds, 1920×1080. The new edit uses English Edge TTS narration, new English captions, and opening/closing covers. The source video remains outside Git; see [provenance](quick-test-demo-provenance.md). -- Duration: 20 seconds. -- Canvas: 1280×720 source captures; export a 960×540 optimized README GIF. -- Language: silent; concise English overlays make every scene understandable without audio. -- Source: current Control Plane build, a public TodoMVC target, dedicated demo Workspace, and a Provider configuration with no visible secrets. +| Edited time | What appears | Meaning | +|---|---|---| +| 00:00–00:06.750 | Opening cover | Describe a goal, test the real app, inspect evidence | +| 00:06.750–00:47.958 | FSQ UI and Android goal entry | Search for the FSQ repository, follow About to the website, define the two checks | +| 00:47.958–01:16.958 | Real Android browser and timeline | Actual navigation and captured action evidence | +| 01:16.958–01:36.958 | Destination content and verification | Check the two required homepage headlines | +| 01:36.958–02:03.208 | Run report and evidence | Inspect the actual execution, verification, and evidence statuses | +| 02:03.208–02:10.708 | Closing cover | Invitation to try a goal locally | -## Shot list +The edit includes explanatory pauses, original-speed/2× segments, and shortened waits. Watching the recording is not FSQ Strict Replay. These edited chapter boundaries describe the delivered video, not real-time execution speed. No speed claim or cross-platform success guarantee follows from them. -| Time | Picture | Caption | Proof shown | -|---|---|---|---| -| 00:00–00:04 | Current Test Runner Explore composer | “1 · Goal” | Real natural-language goal and ready Web target | -| 00:04–00:08 | Current Activity and Live evidence split view | “2 · Evidence” | Real persisted TodoMVC screenshot evidence | -| 00:08–00:12 | Current completed Explore run and Save YAML dialog | “3 · Candidate YAML” | Explicit publication boundary for the Run-local recording | -| 00:12–00:16 | Current Strict Replay setup | “4 · Strict Replay” | Validated provider-free deterministic path | -| 00:16–00:20 | Current Runs durable report | “5 · Runs report” | Passed gate, complete evidence, action timeline, and portable HTML export path | +The three supporting images tell the same story without narration: an editorial task card around a real goal input; a real execution crop; an editorial result card pairing actual page/report captures. Consumers provide equivalent text and descriptive alt text. Website images open at full size for details. -## Recording setup - -1. Install the final wheel into a clean environment; do not record from an editable checkout. -2. Set the OS account name, host name, browser profile, bookmarks, notifications, clock, and terminal prompt to neutral demo values. -3. Use a new Workspace named `fsq-demo` under a neutral path. Clear unrelated Workspaces and Runs from the UI. -4. Use a public no-account demo target only. Do not open email, chat, internal portals, private repositories, or account pages. -5. Set browser zoom to 100%, Control Plane zoom to 100%, and terminal font large enough for 1280px export. -6. Disable notifications, password managers, autofill, and clipboard-history overlays. -7. Keep every scene tied to the same public goal, generated Case, evidence chain, and strict replay result. - -## Capture checkpoints - -- `01-describe-goal.png`: Control Plane goal entry with neutral, real state. -- `02-execute-workflow.png`: active execution timeline without codes, tokens, paths, or device identifiers. -- `03-capture-evidence.png`: real Before/After or UI Tree evidence from the public demo target. -- `04-generate-candidate.png`: Run-local candidate Case generated from persisted execution facts. -- `05-inspect-run.png`: current Runs report with redacted neutral metadata and export entry. - -The final repository must not contain placeholder or synthetic product screenshots. If any checkpoint cannot be captured safely, omit it rather than staging false evidence. - -## Export - -- README GIF: 960×540, 10 fps, optimized palette, target under 8 MB. -- Thumbnail: 1280×720 PNG, derived from a real approved frame. - -## Acceptance - -Watch the export with audio muted and verify that a first-time user can identify the Goal, evidence, candidate Case, strict replay, durable Runs report, and portable HTML export. Then complete every item in [media acceptance](media-acceptance.md). +The older 20-second Control Plane GIF and numbered screenshots are retained only as historical release evidence; they are not the current sequence. See [media acceptance](media-acceptance.md) before publication. diff --git a/docs/media/media-acceptance.md b/docs/media/media-acceptance.md index 7ee0990d..e075b429 100644 --- a/docs/media/media-acceptance.md +++ b/docs/media/media-acceptance.md @@ -1,47 +1,26 @@ -# Public Media Acceptance +# Current public media acceptance -## Provenance +This record concerns the Android Quick Test edit linked from the current README and Website source. It replaces the older blanket checked assertions. Approval of a proposed asset is distinct from a complete privacy review or publication readiness. -- [x] The current Control Plane tour is captured from the recorded source commit; the v0.1.0 full demo is captured from its release-candidate build. -- [x] Synthetic or generated brand art is not presented as product evidence. -- [x] Commit SHA, OS, browser, FSQ version, capture date, and exported media hashes are recorded outside the media frame for the current Control Plane demo. -- [x] Every command, label, and result shown matches the current v0.1.0 release materials. +| Area | Status | Evidence or remaining work | +|---|---|---| +| Approved source selection | Established | User supplied the new YouTube destination and accepted the three-image direction; the assigned asset set and destinations are recorded in provenance. | +| Asset identity | Established | SHA-256, dimensions, transforms, and caption identity are recorded in [provenance](quick-test-demo-provenance.md). Large videos remain outside Git. | +| Recorded Run identity | Established for the matching local record | A visible Run suffix and exact report summary match a local Run with FSQ 0.2.1; source revision and recording/export time remain unknown. | +| Explanatory imagery | Established in source | Task/result cards identify their editorial text; real UI captures remain distinct. README and Website supply equivalent task/result prose and alt text. | +| Captions | Prepared; playback verification pending | New-edit English SRT has 25 cues; VTT preserves text and timing. Browser track loading, synchronization, readability, and muted comprehension require final visual verification. No new Chinese subtitle file is available. | +| No automatic external player | Component-tested; real browser pending | Website creates native video only after activation; explicit viewing links survive failure. Standalone demo follows the same pattern without an iframe. | +| Full video privacy review | Unverified | No claim of frame-by-frame inspection, metadata scan, or complete absence of credentials, account/avatar data, paths, device identifiers, private URLs, notifications, or unrelated content is established by this change. Review the entire exported video and linked destinations before broad promotion. | +| Screenshot privacy | Partially inspected, not certified | Selected images have been visually inspected for layout/content; a complete independent privacy/security checklist remains pending. Do not infer clearance from an asset hash or editorial crop. | +| Source version and runtime | Partial | Producer version/backend and Run timestamps are known; source commit, app/browser/OS versions, capture/export time, and complete capture conditions remain unknown. | +| Package availability | Unverified for the promoted workflow | Source metadata, an installed local version, a package-index response, and a GitHub Release are different facts. No clean public installation of the required workflow has been demonstrated by this change. | +| Real first use | Unverified | No clean-environment end-to-end run or three-person new-user comprehension test has been performed for these public materials. | +| Deployment | Pending | A local production build does not deploy the Website. The intended canonical HTTPS page, assets, link destinations, and current package must be verified after authorized publication. | -## Privacy and security +## Publication checks still required -- [x] No token, API key, password, cookie, authorization header, device code, QR code, or Provider response is visible. -- [x] No personal name, account avatar, email, tenant, organization, repository, hostname, device serial, IP address other than loopback, or private URL is visible. -- [x] No absolute personal path, shell history, notification, browser bookmark, autofill suggestion, clipboard content, or unrelated Workspace is visible. -- [x] Run metadata, logs, screenshots, UI trees, and reports have been inspected frame by frame. -- [x] The current Control Plane GIF contains no private capture metadata and has been scanned after export. +The release owner must review privacy and accessibility for the entire export, confirm the obtainable package supports the instructions, and exercise the public goal in a clean environment. Observe failures as failures; do not substitute a local source checkout or preconfigured machine for first-use evidence. -## Accuracy +Ask at least three people who did not build FSQ to use only the materials. Record whether they can explain FSQ, choose either entry path, satisfy prerequisites, run a Quick Test, inspect evidence, and find feedback/privacy guidance. Record actual times and blockers instead of promising installation or testing within a fixed duration. A repeated comprehension or setup failure needs a documented repair before promotion. -- [x] The demo says alpha and does not imply a 1.0 stability guarantee. -- [x] FSQ is described as complementing Playwright, uiautomator2, pywinauto, and Appium. -- [x] Claims are observable in the captured run or supported by linked public documentation. -- [x] No “zero flakiness,” “always,” “perfect,” or unsupported competitor comparison appears. -- [x] AI exploration, evidence, review, and deterministic replay are visually distinguishable. - -## Accessibility and delivery - -- [x] The story is understandable with audio muted. -- [x] The current tour is silent with embedded English overlays; the English and Chinese subtitle files match the v0.1.0 full edit. -- [x] Current-tour overlays remain readable at 960×540 and do not cover the primary evidence. -- [x] Screenshots have descriptive alt text in consuming documents. -- [x] The README first screen uses a lightweight current-UI animated GIF and retains the GitHub-hosted v0.1.0 full video plus one YouTube link. -- [x] The README does not depend on a separate autoplay demo page because GitHub sanitizes embedded video players. -- [x] Large video files are hosted outside Git history; committed media is limited to screenshots, thumbnail, subtitles, and documentation. - -## New-user test - -Ask at least three people who did not build FSQ to use only the public materials. Record whether they can: - -1. explain FSQ in one sentence after 30 seconds; -2. distinguish FSQ from its platform backends; -3. find and complete installation in five minutes; -4. identify where evidence and Runs are stored; -5. locate prerequisites, privacy guidance, and troubleshooting; and -6. complete the public deterministic Web example within 15 minutes after prerequisites are available. - -Do not publish while a repeated comprehension or setup failure has no documented repair. +Tech Lead and the user own publication decisions. Outstanding media privacy, installation, first-use, deployment, or canonical-page evidence remains explicit; unit tests and a build do not satisfy those checks. diff --git a/docs/media/quick-test-demo-provenance.md b/docs/media/quick-test-demo-provenance.md new file mode 100644 index 00000000..4665f25e --- /dev/null +++ b/docs/media/quick-test-demo-provenance.md @@ -0,0 +1,58 @@ +# Android Quick Test media provenance + +## Destinations and source identity + +- [GitHub video](https://github.com/user-attachments/assets/3acfecf4-bf48-4c76-8245-2eeef0dedb1f) +- [YouTube](https://youtu.be/f1sobvVHNg0) +- The selected files came from the supplied local `FSQ_video/edit_v2/selected_stills/` asset set. No original video was modified in this repository change. +- The externally stored edited master and upload copy are both H.264/AAC, 1920×1080, 130.708333 seconds as observed with ffprobe. “2:11” is a rounded duration label, not a performance claim. + +| Local edit basename (outside Git) | Bytes | SHA-256 | +|---|---:|---| +| FSQ_Android_Quick_Test_EdgeTTS_v2.mp4 | 11250266 | 0584d48fdd24c47ad46f8d99bd32d925b7a8afc280cb083762f515837dd1943b | +| FSQ_Android_Quick_Test_EdgeTTS_v2_GitHub.mp4 | 8018382 | 114f7774c330ce0b0069818ec50e0eb8bea969a00d067b955c8cadb4d88b2792 | + +The hashes identify local files; they do not independently prove byte identity of a remote attachment or YouTube transcode. Recording time, export time, source commit, source application revision, browser version, Android version, and capture-host OS are not established. File modification times and the current repository HEAD are not recording provenance. + +## Matching recorded Run + +The selected report capture's visible Run suffix `7f837a` and exact summary match a local Android Quick Test record. Only the following safe fields are recorded here; full Run files and private paths are not copied into Git: + +- Recorded Run start: `2026-10-20T09:24:13.077261Z` (UTC); completion: `2026-10-20T09:26:00.078623Z` (UTC). These are execution timestamps, not capture/export dates. +- Producer: FSQ `0.2.1`, Python `3.14.7`, backend `uiautomator2` `3.5.2`. +- Run outcome: `success`; whole-goal verification: `success`. +- Report summary: “The supplied repository and landing-page UI records support that the About-linked landing page contains both required phrases.” +- Source application and repository revisions: `null`, with reason `not_reported`. +- Local `run.json` SHA-256: `f0b543c07f60bd7b760efc5a1ea2880918d1d696e471976d9efcdee27a187242`. +- Local `execution-result.json` SHA-256: `0e4a61ab7c110bfd297761592b40c6fb41bbec6047618a5081089b0f4d92bebf`. + +This matching record supports the recorded run, not current-package availability, cross-platform equivalence, or a successful new run. + +## Committed derivatives + +| File | Dimensions or cues | Bytes | SHA-256 | +|---|---|---:|---| +| quick-test-task.png | 1800×1040 | 210400 | 0f0921bc31529034ae3cc8b9bd6f408283576328cd9f973e7b8a0cbabc594d12 | +| quick-test-execution.png | 1593×705 | 337216 | fea487cb2eff842c2dc0bd63a2fff9d7c05f8b2e21d60483f0614297f77cab88 | +| quick-test-result.png | 1800×1040 | 391821 | e0efa713da99b63d8d1ec616ee40c182e5d55587bec39534a44b1221d3af0a72 | +| quick-test-poster.png | 1920×1080 | 445322 | 25328dcea127bf2475ad06f02f8404c135220487725555077eacef4cbe968fad | +| quick-test.en.srt | 25 cues | 2197 | f6696ac2a1859de703139a17e5de8654e6408c3edc314512d048115496635a82 | +| quick-test.en.vtt | 25 cues | 2205 | e31616ad9e341420f250bdd8f2258f4e86a3f87ba19a967c7d9842300b8de005 | + +- `quick-test-task.png` is a byte-for-byte copy of `revised-01-task.png`: a real goal-entry crop within editorial task framing. +- `quick-test-execution.png` is a byte-for-byte copy of `selected-02-execution.png`: a crop of recorded execution. The exact extraction timestamp and source-crop coordinates are not established. +- `quick-test-result.png` is a byte-for-byte copy of `revised-03-result.png`: actual destination-page and report crops within editorial framing. +- `quick-test-poster.png` is a byte-for-byte copy of `selected-video-poster.png`, a full-frame execution view. The exact frame timestamp is not established. +- The supplied context-card script uses crops `(86,246,1836,914)` from `01-describe-goal.png`, `(133,235,756,907)` from `07-verify-content.png`, and `(91,314,1838,557)` from `03-inspect-report.png`, then resizes within the cards. The editorial text is not product UI or independent test evidence. +- The English SRT is copied unchanged from the supplied v2 edit. WebVTT changes only the header and timestamp decimal separators; all 25 cue texts/times are preserved. Old English/Chinese SRT files remain historical and are not assigned to the new player. +- Existing local edit records identify Edge TTS `7.3.1`, voice `en-US-AndrewMultilingualNeural`, and opening/closing covers with 0.75-second fades. These are edit-record observations, not a newly performed narration or source-recording audit. + +## Share image + +`docs/assets/social-preview.png` is a 1200×630 editorial brand graphic with white/green styling and the retained headline. It is not product evidence. `frontend/website/public/social-preview.png` is byte-identical and publishes at `https://fsq-lab.github.io/FSQ/social-preview.png` without a runtime network dependency. + +SHA-256 for both copies: `46d980ce499f168d14f6cffb5faa2456c8f24d5322254fbb85883d1954c7872b`. + +## Review limits + +The selected images have been visually inspected for content/layout and are approved as the promotional direction. A complete frame-by-frame video privacy/security review, independent muted-comprehension/caption review, clean public installation, first-use run, and deployed-page verification are not established by these facts. See [acceptance status](media-acceptance.md). No checkbox implies those checks passed. diff --git a/docs/media/quick-test-execution.png b/docs/media/quick-test-execution.png new file mode 100644 index 00000000..c93ef753 Binary files /dev/null and b/docs/media/quick-test-execution.png differ diff --git a/docs/media/quick-test-poster.png b/docs/media/quick-test-poster.png new file mode 100644 index 00000000..1b204a58 Binary files /dev/null and b/docs/media/quick-test-poster.png differ diff --git a/docs/media/quick-test-result.png b/docs/media/quick-test-result.png new file mode 100644 index 00000000..1d247988 Binary files /dev/null and b/docs/media/quick-test-result.png differ diff --git a/docs/media/quick-test-task.png b/docs/media/quick-test-task.png new file mode 100644 index 00000000..3501cbc1 Binary files /dev/null and b/docs/media/quick-test-task.png differ diff --git a/docs/media/quick-test.en.srt b/docs/media/quick-test.en.srt new file mode 100644 index 00000000..277742ab --- /dev/null +++ b/docs/media/quick-test.en.srt @@ -0,0 +1,100 @@ +1 +00:00:00,650 --> 00:00:06,650 +Meet FSQ. Describe a goal, test the real app, and inspect the evidence. + +2 +00:00:07,100 --> 00:00:13,300 +This is the FSQ web interface, launched with the fsq ui command. + +3 +00:00:14,100 --> 00:00:20,400 +On the left, a real Android phone is mirrored so you can follow the test. + +4 +00:00:21,100 --> 00:00:23,200 +Open Quick Test. + +5 +00:00:23,600 --> 00:00:25,200 +Select Android. + +6 +00:00:25,800 --> 00:00:28,400 +Choose the connected Pixel 6. + +7 +00:00:29,300 --> 00:00:34,750 +Describe the task in natural language: open Edge and find the FSQ GitHub repository. + +8 +00:00:35,300 --> 00:00:39,150 +Then follow the link in its About section to the website. + +9 +00:00:39,800 --> 00:00:45,150 +Specify what to verify: Describe a goal, and Test the real app. + +10 +00:00:45,800 --> 00:00:47,650 +Then, start the test. + +11 +00:00:48,300 --> 00:00:53,200 +FSQ begins executing the task on the connected Android phone. + +12 +00:00:53,800 --> 00:00:55,950 +It opens Edge. + +13 +00:00:56,800 --> 00:01:01,650 +It searches for the repository, just as a user would. + +14 +00:01:02,300 --> 00:01:05,600 +Then it opens the FSQ repository. + +15 +00:01:06,300 --> 00:01:12,750 +The activity timeline shows the actions. Captured screenshots provide a record of what happened. + +16 +00:01:13,300 --> 00:01:16,850 +The website opens. Next, check the expected content. + +17 +00:01:17,300 --> 00:01:26,350 +FSQ checks for the two phrases in the test goal: Describe a goal, and Test the real app. + +18 +00:01:27,300 --> 00:01:30,600 +The recorded assertion passes. + +19 +00:01:31,300 --> 00:01:36,450 +The run reports success, with a summary of the verification result. + +20 +00:01:37,300 --> 00:01:43,650 +Review the captured screens to see how the task progressed. + +21 +00:01:44,300 --> 00:01:45,900 +Open the report. + +22 +00:01:46,300 --> 00:01:51,650 +Inspect the execution status, verification result, and evidence availability. + +23 +00:01:52,300 --> 00:01:57,650 +Explore the recorded actions and structured logs for more detail. + +24 +00:01:58,300 --> 00:02:03,000 +From a natural-language goal to a test result you can inspect. + +25 +00:02:04,008 --> 00:02:09,658 +Ready to try your own test? Visit the FSQ website to get started. + diff --git a/docs/media/quick-test.en.vtt b/docs/media/quick-test.en.vtt new file mode 100644 index 00000000..345ea82f --- /dev/null +++ b/docs/media/quick-test.en.vtt @@ -0,0 +1,102 @@ +WEBVTT + +1 +00:00:00.650 --> 00:00:06.650 +Meet FSQ. Describe a goal, test the real app, and inspect the evidence. + +2 +00:00:07.100 --> 00:00:13.300 +This is the FSQ web interface, launched with the fsq ui command. + +3 +00:00:14.100 --> 00:00:20.400 +On the left, a real Android phone is mirrored so you can follow the test. + +4 +00:00:21.100 --> 00:00:23.200 +Open Quick Test. + +5 +00:00:23.600 --> 00:00:25.200 +Select Android. + +6 +00:00:25.800 --> 00:00:28.400 +Choose the connected Pixel 6. + +7 +00:00:29.300 --> 00:00:34.750 +Describe the task in natural language: open Edge and find the FSQ GitHub repository. + +8 +00:00:35.300 --> 00:00:39.150 +Then follow the link in its About section to the website. + +9 +00:00:39.800 --> 00:00:45.150 +Specify what to verify: Describe a goal, and Test the real app. + +10 +00:00:45.800 --> 00:00:47.650 +Then, start the test. + +11 +00:00:48.300 --> 00:00:53.200 +FSQ begins executing the task on the connected Android phone. + +12 +00:00:53.800 --> 00:00:55.950 +It opens Edge. + +13 +00:00:56.800 --> 00:01:01.650 +It searches for the repository, just as a user would. + +14 +00:01:02.300 --> 00:01:05.600 +Then it opens the FSQ repository. + +15 +00:01:06.300 --> 00:01:12.750 +The activity timeline shows the actions. Captured screenshots provide a record of what happened. + +16 +00:01:13.300 --> 00:01:16.850 +The website opens. Next, check the expected content. + +17 +00:01:17.300 --> 00:01:26.350 +FSQ checks for the two phrases in the test goal: Describe a goal, and Test the real app. + +18 +00:01:27.300 --> 00:01:30.600 +The recorded assertion passes. + +19 +00:01:31.300 --> 00:01:36.450 +The run reports success, with a summary of the verification result. + +20 +00:01:37.300 --> 00:01:43.650 +Review the captured screens to see how the task progressed. + +21 +00:01:44.300 --> 00:01:45.900 +Open the report. + +22 +00:01:46.300 --> 00:01:51.650 +Inspect the execution status, verification result, and evidence availability. + +23 +00:01:52.300 --> 00:01:57.650 +Explore the recorded actions and structured logs for more detail. + +24 +00:01:58.300 --> 00:02:03.000 +From a natural-language goal to a test result you can inspect. + +25 +00:02:04.008 --> 00:02:09.658 +Ready to try your own test? Visit the FSQ website to get started. + diff --git a/frontend/website/SPEC.md b/frontend/website/SPEC.md index 8bd9408d..e5392647 100644 --- a/frontend/website/SPEC.md +++ b/frontend/website/SPEC.md @@ -11,7 +11,7 @@ The Website does not execute FSQ, collect credentials or visitor input, replace - The parent frontend workspace owns React, React DOM, TypeScript, Vite, Vitest, Testing Library, the root npm manifest, and the lock file. - Repository-owned FSQ logo, approved public media, media provenance, and social-preview assets provide local presentation assets. - Canonical public product statements and setup destinations come from root `SPEC.md`, `README.md`, `docs/getting-started.md`, `docs/platform-prerequisites.md`, `docs/support-and-stability.md`, `SECURITY.md`, and `CONTRIBUTING.md`. -- Browser built-ins provide semantic navigation and the optional Clipboard API interaction. +- Browser built-ins provide semantic navigation, native disclosure elements, the optional Clipboard API interaction, and user-initiated HTML video playback with native controls and local caption tracks. - The Website has no Control Plane API, Python runtime, remote font, analytics, third-party widget, or automatic video-player dependency. - The Website must not import Control Plane source or another frontend entry's private implementation. @@ -20,21 +20,26 @@ The Website does not execute FSQ, collect credentials or visitor input, replace - `index.html` is the English GitHub Pages entry below the `/FSQ/` project-site base path. - The canonical URL is `https://fsq-lab.github.io/FSQ/`. - The page presents FSQ as open-source alpha software for goal-driven real-UI testing that records inspectable evidence. It states that a run may produce a Case, that users review results and any returned Case, and that deterministic replay is a separate user choice. -- The page contains a concise hero, an approved replaceable demonstration-media area, a three-part workflow explanation, supported-platform context, two equal setup paths, alpha/privacy guidance, and project/documentation links. +- The page retains its light white-and-green visual system and the hero headline `Describe a goal. Test the real app.`. Its concise description leads with natural-language goals, real-UI execution and verification, and inspectable screenshots, actions, and results. +- The page contains an approved replaceable demonstration-media area, a three-part workflow explanation with contextualized real screenshots, supported-platform context, two equal setup paths, a shared complete first-test example, progressive capability descriptions, alpha/privacy guidance, and project/documentation links. - The two setup paths are titled `Coding Agent` and `FSQ UI`. They use the same semantic heading level, typography, visual weight, container treatment, and action prominence. They are alternatives, carry no sequential numbering or progression language, and remain visibly equal when stacked at narrow widths. - The Coding Agent path links to the authoritative approval-aware Coding Agent setup and states that Provider credentials must not be pasted into Coding Agent chat. - The FSQ UI path links to the authoritative UI setup, may offer the documented installation and launch commands for copying, and states that Provider and model configuration occurs in the local UI. Copy failure leaves selectable command text and exposes an explicit nearby failure message. +- Both setup paths lead to one Quick Test and result/evidence review. Short entry summaries precede expandable setup details; Python 3.11+, a supported configured Provider and selected model, and platform prerequisites remain visible. Quick Test needs no manual project Workspace setup and does not save a formal Case or perform Strict Replay. +- A shared copyable first-test example searches for `FSQ-lab/FSQ`, follows the repository About website link, and checks both homepage headlines. It distinguishes the Web first-use option from the recorded Android run. Copy feedback is local to the owning control and preserves selectable text on failure. +- Case management and Strict Replay are described as separately chosen advanced workflows. Star and experience-feedback links appear after the trial guidance and navigate to the canonical GitHub repository without authentication widgets, submission, tracking, or automatic starring. A visible link leads to the Chinese getting-started guide. - Web, Android, Windows, and macOS are presented as supported target platforms subject to platform-specific prerequisites. -- Media playback or an external video destination begins only after an explicit user action. The page does not preload a third-party player. +- The demonstration initially shows a repository-owned 16:9 real-device/execution poster, an accessible play button, and a duration label. Activating the button replaces the poster in place with native HTML video controls for playback, pause, seeking, and fullscreen where supported. The player uses the approved external GitHub video attachment and same-origin captions; no external video URL is assigned to a loading element before activation, and there is no page-load autoplay or third-party iframe/player library. +- Explicit GitHub-video and YouTube links remain available independently of playback. Loading and playback errors expose nearby status or fallback guidance without hiding those links or the rest of the page. Playback start is claimed only after the browser confirms playback; rejected playback leaves native/manual controls usable. - Primary project links use `FSQ-lab/FSQ`; `microsoft/FSQ` is not presented as the canonical project. ## Data And State Flow - Product copy, navigation destinations, media references, captions, alternative text, and provenance labels are static, typed Website content. - Media references remain centralized so an approved replacement can change the source and metadata without redesigning the media container or surrounding copy. -- The only transient browser state is command-copy success or failure feedback owned by the FSQ UI path. +- Transient browser state is local to command/example copying and demonstration activation, loading, playback, or failure. Disclosure expansion is native browser state. No state persists between visits. - The Website stores no visitor state, sets no project-owned cookie, uses no local or session storage, submits no visitor data, and performs no behavioral analytics or tracking. -- Essential fonts, styles, scripts, logos, and rendered media load from the Pages origin. External destinations are contacted only after explicit navigation. +- Essential fonts, styles, scripts, logos, posters, screenshots, and caption tracks load from the Pages origin. External media is requested only after explicit playback activation; other external destinations are contacted only after explicit navigation. No preconnect, preload, or automatically embedded external player is used. ## Internal Structure @@ -49,8 +54,8 @@ The Website does not execute FSQ, collect credentials or visitor input, replace - Architecture level: Level 1 Static Or Minimal Interactive Page. - Runtime boundary: static browser assets hosted by GitHub Pages; no backend calls or service worker. -- State boundary: copy feedback remains local to the control that owns it; all other content is static. -- Integration boundary: ordinary links lead to canonical GitHub documentation and user-selected media destinations. +- State boundary: copy feedback and video lifecycle state remain local to their owning controls; page content and destinations are static. +- Integration boundary: ordinary links lead to canonical GitHub documentation and user-selected media destinations; the browser video element requests the approved external attachment only after activation. - Dependency direction: Website source may consume parent-workspace dependencies and repository-owned public assets; it does not import sibling application source or backend implementation. - Current framework and language: React with TypeScript/TSX and authored CSS, compiled by Vite. @@ -68,9 +73,9 @@ The Website does not execute FSQ, collect credentials or visitor input, replace - The production artifact resolves HTML, scripts, styles, images, metadata, and canonical links below `/FSQ/`. - Browser verification covers 320, 768, 1024, and 1440 CSS-pixel widths, 200% zoom/reflow, no page-level horizontal overflow, media containment, and equal presentation of both setup paths. - Accessibility verification covers English page language, one page-level heading, logical heading order, semantic landmarks, keyboard navigation, visible focus, 44-by-44 CSS-pixel touch targets where applicable, text and control contrast, alternative text, and reduced-motion behavior. -- JavaScript-disabled and media-unavailable checks preserve the value proposition, alpha/privacy warning, both setup paths, and setup links. -- Browser network and storage inspection confirms no project-added tracking, remote font, automatic video-player load, cookie, visitor storage, or visitor-data submission. -- Link verification covers the canonical repository, both setup paths, getting started, prerequisites, support/stability, security, contribution, media destination, and license. +- JavaScript-disabled and media-unavailable checks preserve the value proposition, alpha/privacy warning, both setup paths, setup links, a Chinese guide destination, and an explicit current demonstration link. +- Browser network and storage inspection confirms no project-added tracking, remote font, external video request before activation, third-party player, project-owned cookie, visitor storage, or visitor-data submission. Player checks cover activation, native controls, loading, captions, playback rejection/error fallback, and external viewing links. +- Link verification covers the canonical repository, both setup paths, English and Chinese getting started, prerequisites, support/stability, security, contribution, Star/feedback destinations, both media destinations, and license. - Published media follows `docs/media/media-acceptance.md`, has reviewed provenance and privacy, and remains understandable without audio. ## Current Invariants diff --git a/frontend/website/index.html b/frontend/website/index.html index 5780cc6f..5a3fca07 100644 --- a/frontend/website/index.html +++ b/frontend/website/index.html @@ -6,19 +6,19 @@ - + - + FSQ — Describe a goal. Test the real app. @@ -35,17 +35,22 @@

Describe a goal. Test the real app.

records screenshots, interface state, and events so you can inspect the evidence. A run may return a Case; review it before making the separate choice to replay it.

-

Choose either setup path

+

Choose either Quick Test path

+

Python 3.11+, a supported configured Provider and selected model, and platform prerequisites are required. Quick Test needs no manual project Workspace and does not save a formal Case or perform Strict Replay.

Coding Agent

-

Use the approval-aware local workflow. Never paste Provider credentials into Coding Agent chat.

-

Open Coding Agent setup

+

Add the FSQ Skill, reload the Coding Agent, and ask it to run your Quick Test goal. Never paste Provider credentials into Coding Agent chat.

+

Open Coding Agent setup

FSQ UI

-

Configure your Provider and model privately in the local FSQ UI.

-

Open FSQ UI setup

+

Install fsq-agent, run fsq ui, and choose Quick Test. Configure your Provider and model privately in Settings.

+

Open FSQ UI setup

+

Try a complete user journey

+

In a selected Web browser, search for the FSQ-lab/FSQ GitHub repository, follow the website link in About, and verify both “Describe a goal.” and “Test the real app.” on the destination page. Only browse public pages; do not sign in or submit forms. Inspect the result and captured page evidence.

+

The recorded demonstration uses Android: Open GitHub video · Watch on YouTube.

+

中文快速开始

Alpha and privacy: Evidence is stored locally and may contain private app data. Review it before sharing. Web, Android, Windows, and macOS require platform-specific diff --git a/frontend/website/public/social-preview.png b/frontend/website/public/social-preview.png new file mode 100644 index 00000000..9d0c224b Binary files /dev/null and b/frontend/website/public/social-preview.png differ diff --git a/frontend/website/src/WebsiteApp.test.tsx b/frontend/website/src/WebsiteApp.test.tsx index 03e13678..e5906fbe 100644 --- a/frontend/website/src/WebsiteApp.test.tsx +++ b/frontend/website/src/WebsiteApp.test.tsx @@ -1,132 +1,110 @@ -import { fireEvent, render, screen, within } from '@testing-library/react'; +import { fireEvent, render, screen, waitFor, within } from '@testing-library/react'; import userEvent from '@testing-library/user-event'; - import { WebsiteApp } from './WebsiteApp'; import { websiteContent } from './content'; import indexHtml from '../index.html?raw'; +const videoSource = 'https://github.com/user-attachments/assets/3acfecf4-bf48-4c76-8245-2eeef0dedb1f'; +const youtube = 'https://youtu.be/f1sobvVHNg0'; +function clipboard(writeText = vi.fn().mockResolvedValue(undefined)) { + Object.defineProperty(navigator, 'clipboard', { configurable: true, value: { writeText } }); + return writeText; +} + describe('WebsiteApp', () => { - it('presents the page truth with a logical heading hierarchy', () => { + afterEach(() => vi.restoreAllMocks()); + it('keeps the headline and gives equal unnumbered Quick Test entries', () => { render(); - - expect( - screen.getByRole('heading', { level: 1, name: 'Describe a goal. Test the real app.' }), - ).toBeInTheDocument(); expect(screen.getAllByRole('heading', { level: 1 })).toHaveLength(1); - expect(screen.getByText(/open-source alpha software/i)).toBeInTheDocument(); - expect(screen.getByText(/screenshots, interface state, and events/i)).toBeInTheDocument(); - expect(screen.getByText(/may produce a Case/i)).toBeInTheDocument(); - expect(screen.getByText(/deterministic replay is a separate user choice/i)).toBeInTheDocument(); - - const main = screen.getByRole('main'); - expect(within(main).getByRole('heading', { level: 2, name: 'From goal to evidence.' })).toBeInTheDocument(); - expect(within(main).getByRole('heading', { level: 2, name: 'Choose how to begin.' })).toBeInTheDocument(); - }); - - it('renders Coding Agent and FSQ UI as exactly two unnumbered equal alternatives', () => { - render(); - - const choices = screen.getByTestId('setup-choices'); - const cards = within(choices).getAllByRole('article'); + expect(screen.getByRole('heading', { level: 1 })).toHaveTextContent('Describe a goal. Test the real app.'); + const cards = within(screen.getByTestId('setup-choices')).getAllByRole('article'); expect(cards).toHaveLength(2); - expect(within(cards[0]).getByRole('heading', { level: 3, name: 'Coding Agent' })).toBeInTheDocument(); - expect(within(cards[1]).getByRole('heading', { level: 3, name: 'FSQ UI' })).toBeInTheDocument(); - expect(cards[0]).toHaveClass('setup-card'); - expect(cards[1]).toHaveClass('setup-card'); - expect(choices).not.toHaveTextContent(/\b0?[12][.)/]\s*(Coding Agent|FSQ UI)/i); - expect(within(cards[0]).getByRole('link', { name: /Coding Agent setup/i })).toHaveAttribute( - 'href', - websiteContent.links.codingAgent, - ); - expect(within(cards[1]).getByRole('link', { name: /FSQ UI setup/i })).toHaveAttribute( - 'href', - websiteContent.links.fsqUi, - ); + for (const [index, title] of ['Coding Agent', 'FSQ UI'].entries()) { + expect(within(cards[index]).getByRole('heading', { level: 3, name: title })).toBeInTheDocument(); + expect(cards[index]).toHaveClass('setup-card'); + expect(cards[index].querySelector('details')).not.toHaveAttribute('open'); + expect(within(cards[index]).getByRole('link', { name: 'Open ' + title + ' setup' })).toHaveClass('button--secondary'); + } + expect(screen.getByTestId('setup-choices')).not.toHaveTextContent(/\b[12][.)]\s*(Coding Agent|FSQ UI)/); + expect(screen.getByText(/Python 3.11\+/)).toHaveTextContent(/configured Provider and selected model/); + expect(screen.getByText(/no manual project Workspace/i)).toBeInTheDocument(); + expect(screen.getByText(/Never paste Provider credentials into Coding Agent chat/)).toBeInTheDocument(); }); - - it('uses canonical links and centralized media metadata', () => { - render(); - - expect(screen.getByRole('link', { name: /GitHub repository/i })).toHaveAttribute( - 'href', - websiteContent.links.repository, - ); - expect(screen.getByRole('link', { name: /platform.*prerequisites/i })).toHaveAttribute( - 'href', - websiteContent.links.prerequisites, - ); - expect(screen.getByRole('link', { name: /Watch the full demonstration/i })).toHaveAttribute( - 'href', - websiteContent.media.destination, - ); - - const media = screen.getByRole('img', { name: websiteContent.media.alt }); - expect(media).toHaveAttribute('src', expect.stringContaining('fsq-v0.1.0-demo-thumbnail')); - expect(media).toHaveAttribute('width', String(websiteContent.media.width)); - expect(media).toHaveAttribute('height', String(websiteContent.media.height)); - expect(screen.getByRole('link', { name: /Play the recorded demonstration/i })).toHaveAttribute( - 'href', - expect.stringContaining('fsq-control-plane-demo'), - ); - expect(screen.getByText(websiteContent.media.provenanceLabel)).toBeInTheDocument(); - expect(screen.getByText(/recorded from FSQ 0.1.0/i)).toBeInTheDocument(); - expect(screen.queryByTitle(/youtube/i)).not.toBeInTheDocument(); + it('does not load external media before activation and exposes both viewing links', () => { + const { container } = render(); + expect(container.querySelector('video, iframe, source, track')).toBeNull(); + expect(screen.getByRole('button', { name: /Play Android Quick Test demonstration/ })).toHaveTextContent('2:11'); + expect(screen.getByRole('link', { name: /Open GitHub video/ })).toHaveAttribute('href', videoSource); + expect(screen.getByRole('link', { name: /YouTube/ })).toHaveAttribute('href', youtube); + expect(screen.getByRole('img', { name: websiteContent.media.alt })).toHaveAttribute('src', expect.stringContaining('quick-test-poster')); }); - - it('states provider, privacy, alpha, and platform boundaries', () => { - render(); - - expect(screen.getByText(/never paste Provider credentials into Coding Agent chat/i)).toBeInTheDocument(); - expect(screen.getByText(/configure your Provider and model in the local FSQ UI/i)).toBeInTheDocument(); - expect(screen.getAllByText(/configured Provider/i).length).toBeGreaterThan(0); - expect(screen.getByText(/Evidence is stored locally and may contain private app data/i)).toBeInTheDocument(); - expect( - screen.getByText((_, element) => element?.classList.contains('platform-line') ?? false), - ).toHaveTextContent(/Web, Android, Windows, macOS/i); - expect(screen.getAllByText(/platform-specific prerequisites/i).length).toBeGreaterThan(0); + it('activates native inline playback with captions and waits for the playing event', async () => { + vi.spyOn(HTMLMediaElement.prototype, 'play').mockResolvedValue(undefined); + const { container } = render(); + await userEvent.click(screen.getByRole('button', { name: /Play Android Quick Test demonstration/ })); + const video = container.querySelector('video')!; + expect(video).toHaveAttribute('src', videoSource); + expect(video).toHaveAttribute('controls'); + expect(video).toHaveAttribute('tabindex', '0'); + expect(video).toHaveFocus(); + expect(video).toHaveAttribute('playsinline'); + expect(video).not.toHaveAttribute('autoplay'); + expect(video.querySelector('track')).toHaveAttribute('kind', 'captions'); + expect(video.querySelector('track')).toHaveAttribute('srclang', 'en'); + expect(video.querySelector('track')).toHaveAttribute('src', expect.stringContaining('quick-test.en.vtt')); + expect(screen.getByTestId('media-status')).toHaveTextContent(/Loading/); + fireEvent.playing(video); + expect(screen.getByTestId('media-status')).toHaveTextContent('Playing'); + fireEvent.waiting(video); + expect(screen.getByTestId('media-status')).toHaveTextContent(/Loading/); + fireEvent.pause(video); + expect(screen.getByTestId('media-status')).toHaveTextContent('Paused'); }); - - it('reports clipboard success only after commands are written', async () => { - const writeText = vi.fn().mockResolvedValue(undefined); - Object.defineProperty(navigator, 'clipboard', { - configurable: true, - value: { writeText }, - }); + it('retains controls and links after play rejection or media error', async () => { + vi.spyOn(HTMLMediaElement.prototype, 'play').mockRejectedValue(new Error('blocked')); + const { container } = render(); + await userEvent.click(screen.getByRole('button', { name: /Play Android Quick Test demonstration/ })); + await waitFor(() => expect(screen.getByTestId('media-status')).toHaveTextContent(/Use the video controls/)); + const video = container.querySelector('video')!; + expect(video).toHaveAttribute('controls'); + fireEvent.error(video); + expect(screen.getByTestId('media-status')).toHaveTextContent(/Could not load/); + expect(screen.getByRole('link', { name: /Open GitHub video/ })).toHaveAttribute('href', videoSource); + expect(screen.getByRole('link', { name: /YouTube/ })).toHaveAttribute('href', youtube); + expect(screen.getByRole('heading', { name: 'Choose how to begin.' })).toBeInTheDocument(); + }); + it('offers the full scenario with local copy feedback and progressive capabilities', async () => { + const writeText = clipboard(); render(); - - await userEvent.click(screen.getByRole('button', { name: 'Copy commands' })); - - expect(writeText).toHaveBeenCalledWith(websiteContent.setupCommands.join('\n')); - expect(screen.getByRole('status')).toHaveTextContent('Commands copied.'); + const scenario = screen.getByLabelText('First Quick Test goal'); + for (const text of ['FSQ-lab/FSQ', 'About', 'Describe a goal.', 'Test the real app.', 'Do not sign in']) expect(scenario).toHaveTextContent(text); + await userEvent.click(screen.getByRole('button', { name: 'Copy test goal' })); + expect(writeText).toHaveBeenCalledWith(scenario.textContent); + expect(screen.getByText('Test goal copied.')).toBeInTheDocument(); + expect(screen.queryByText('Commands copied.')).toBeNull(); + expect(screen.getByText(/recorded demonstration uses Android/)).toBeInTheDocument(); + expect(screen.getByText(/does not save a formal Case or perform Strict Replay/)).toBeInTheDocument(); + expect(screen.getByRole('link', { name: /Star FSQ/ })).toHaveAttribute('href', 'https://github.com/FSQ-lab/FSQ'); + expect(screen.getByRole('link', { name: /Share your experience/ })).toHaveAttribute('href', 'https://github.com/FSQ-lab/FSQ/issues/new/choose'); + expect(screen.getByRole('link', { name: /中文快速开始/ })).toBeInTheDocument(); }); - - it('leaves selectable commands and explicit guidance when clipboard copying fails', async () => { - const writeText = vi.fn().mockRejectedValue(new Error('denied')); - Object.defineProperty(navigator, 'clipboard', { - configurable: true, - value: { writeText }, - }); + it('preserves selectable text and explicit feedback when copying fails', async () => { + clipboard(); render(); - + Object.defineProperty(navigator, 'clipboard', { configurable: true, value: undefined }); await userEvent.click(screen.getByRole('button', { name: 'Copy commands' })); - - expect(screen.getByRole('status')).toHaveTextContent( - 'Could not copy automatically. Select and copy the commands above.', - ); - const commands = screen.getByLabelText('Commands to install FSQ and open the local UI'); - expect(commands).toHaveTextContent('python -m pip install --upgrade fsq-agent'); - expect(commands).toHaveTextContent('fsq ui'); - fireEvent.select(commands); + expect(screen.getByText('Could not copy automatically. Select and copy the commands above.')).toBeInTheDocument(); + expect(screen.getByLabelText('Commands to install FSQ and open the local UI')).toHaveTextContent('fsq ui'); + clipboard(vi.fn().mockRejectedValue(new Error('denied'))); + await userEvent.click(screen.getByRole('button', { name: 'Copy test goal' })); + expect(screen.getByText('Could not copy automatically. Select and copy the test goal above.')).toBeInTheDocument(); + expect(screen.getByLabelText('First Quick Test goal')).toHaveTextContent('FSQ-lab/FSQ'); }); - - it('keeps essential product and setup guidance in the JavaScript-disabled HTML boundary', () => { - expect(indexHtml).toContain('Describe a goal. Test the real app.'); - expect(indexHtml).toContain('Coding Agent'); - expect(indexHtml).toContain('FSQ UI'); - expect(indexHtml).toContain('#1-coding-agent'); - expect(indexHtml).toContain('#2-fsq-ui'); - expect(indexHtml).toMatch(/alpha/i); - expect(indexHtml).toMatch(/credentials.*Coding Agent chat/i); - expect(indexHtml).toMatch(/Evidence is stored locally/i); + it('keeps essential guidance and current demo without JavaScript or automatic embeds', () => { + for (const text of ['Describe a goal. Test the real app.', 'Coding Agent', 'FSQ UI', 'Python 3.11+', 'Quick Test', 'Alpha and privacy', '中文快速开始', youtube]) expect(indexHtml).toContain(text); + expect(indexHtml).not.toMatch(/); + expect(container.querySelector('iframe, form, input')).toBeNull(); + for (const img of container.querySelectorAll('img')) expect(img).toHaveAttribute('alt'); }); }); diff --git a/frontend/website/src/WebsiteApp.tsx b/frontend/website/src/WebsiteApp.tsx index 3f8c6415..921bfa67 100644 --- a/frontend/website/src/WebsiteApp.tsx +++ b/frontend/website/src/WebsiteApp.tsx @@ -1,211 +1,155 @@ -import { useState } from 'react'; - +import { useRef, useState } from 'react'; import { websiteContent as content } from './content'; -const externalLinkProps = { - target: '_blank', - rel: 'noreferrer', -} as const; - -function Arrow() { - return ; -} +const externalLinkProps = { target: '_blank', rel: 'noreferrer' } as const; +function Arrow() { return ; } -function CopyCommands() { +type CopyTextProps = { + text: string; label: string; accessibleName: string; copyAction: string; + copiedAction: string; success: string; failure: string; wrap?: boolean; +}; +function CopyText(props: CopyTextProps) { const [status, setStatus] = useState<'idle' | 'success' | 'failure'>('idle'); - const commands = content.setupCommands.join('\n'); - - async function copyCommands() { + async function copy() { try { - if (!navigator.clipboard) { - throw new Error('Clipboard API unavailable'); - } - await navigator.clipboard.writeText(commands); + if (!navigator.clipboard) throw new Error('Clipboard unavailable'); + await navigator.clipboard.writeText(props.text); setStatus('success'); - } catch { - setStatus('failure'); - } + } catch { setStatus('failure'); } } - return (

- {content.commandPanel.label} -
-
-        {commands}
+      
+        {props.text}
       
-

- {status === 'success' && content.commandPanel.success} - {status === 'failure' && content.commandPanel.failure} +

+ {status === 'success' && props.success}{status === 'failure' && props.failure}

); } +function Demonstration() { + const [activated, setActivated] = useState(false); + const [status, setStatus] = useState(''); + const requestedPlayback = useRef(false); + const failed = useRef(false); + function attachVideo(video: HTMLVideoElement | null) { + if (!video || requestedPlayback.current) return; + requestedPlayback.current = true; + video.focus(); + void video.play().catch(() => { + if (!failed.current) setStatus('Playback did not start. Use the video controls or open a viewing link below.'); + }); + } + return ( +
+ +
+ {activated ? ( + + ) : ( + + )} +
+
+

{content.media.caption}

+

{status}

+ +
+
+ ); +} + export function WebsiteApp() { return ( <> - - {content.skipLink} - + {content.skipLink}
- - {content.logo.alt} - + {content.logo.alt}
-

{content.hero.eyebrow}

-

- Describe a goal. - Test the real app. -

+

Describe a goal. Test the real app.

{content.hero.description}

- - -
- - {content.media.alt} -
-

{content.media.caption}

- -
-
- -

- {content.platformLine.label} {content.platforms.join(', ')} —{' '} - {content.platformLine.qualifier}{' '} - - {content.platformLine.action} - - . -

+ + +

{content.platformLine.label} {content.platforms.join(', ')} — {content.platformLine.qualifier}{' '}{content.platformLine.action}.

-
-

{content.navigation.workflow}

-

{content.workflow.title}

+

{content.navigation.workflow}

{content.workflow.title}

{content.workflow.introduction}

-
+
{content.workflow.steps.map((step, index) => ( -
- -

{step.title}

-

{step.body}

+ ))}
-
-

{content.setup.label}

-

{content.setup.title}

-

{content.setup.introduction}

+

{content.setup.label}

{content.setup.title}

+

{content.setup.introduction} Check prerequisites

+

{content.setup.quickTest}

- - - + {[content.setup.codingAgent, content.setup.fsqUi].map((path, index) => ( +
+

{path.title}

{path.body}

{path.warning}

+
{path.details}

{path.detail}

+ {index === 1 && } +
+ {path.action} +
+ ))} +
+
+

{content.example.title}

{content.example.introduction}

+ +

After the run, inspect the outcome and page evidence. A failed or inconclusive run is a result to investigate, not a reason to assume the checks passed.

- - +
+

Beyond the first test

{content.capabilities.title}

{content.capabilities.introduction}

+
{content.capabilities.items.map(item =>

{item.title}

{item.body}

)}
+
+
+

{content.community.title}

{content.community.body}

+ +

{content.community.feedback}

+
+
- - + ); } diff --git a/frontend/website/src/content.ts b/frontend/website/src/content.ts index 72543e04..62e96d38 100644 --- a/frontend/website/src/content.ts +++ b/frontend/website/src/content.ts @@ -1,140 +1,147 @@ import logoUrl from '../../../docs/assets/logo-light.svg'; -import mediaPosterUrl from '../../../docs/media/fsq-v0.1.0-demo-thumbnail.png'; -import mediaSourceUrl from '../../../docs/media/fsq-control-plane-demo.gif'; +import mediaPosterUrl from '../../../docs/media/quick-test-poster.png'; +import taskUrl from '../../../docs/media/quick-test-task.png'; +import executionUrl from '../../../docs/media/quick-test-execution.png'; +import resultUrl from '../../../docs/media/quick-test-result.png'; +import captionsUrl from '../../../docs/media/quick-test.en.vtt?url&no-inline'; const repository = 'https://github.com/FSQ-lab/FSQ'; -const repositoryFile = `${repository}/blob/main`; +const repositoryFile = repository + '/blob/main'; export type WebsiteMedia = Readonly<{ poster: string; source: string; destination: string; + captions: string; alt: string; caption: string; provenanceLabel: string; provenanceUrl: string; width: number; height: number; + duration: string; }>; export const websiteContent = { canonicalUrl: 'https://fsq-lab.github.io/FSQ/', - logo: { - source: logoUrl, - alt: 'FSQ — Fully Self Quality', - }, + logo: { source: logoUrl, alt: 'FSQ — Fully Self Quality' }, navigation: { - label: 'Main navigation', - home: 'FSQ website home', - workflow: 'How it works', - setup: 'Get started', - repository: 'GitHub repository', + label: 'Main navigation', home: 'FSQ website home', workflow: 'How it works', + setup: 'Try FSQ', repository: 'GitHub repository', }, skipLink: 'Skip to content', hero: { eyebrow: 'Open source · Alpha · Human in control', title: 'Describe a goal. Test the real app.', - description: - 'FSQ is open-source alpha software for goal-driven testing of real user interfaces. Inspect the evidence, review any returned Case, and decide separately whether to replay it.', - setupAction: 'Choose a setup path', - mediaAction: 'See the demonstration', + description: 'Describe what to do and what should happen. Let AI operate the real app and verify the outcome. Inspect screenshots, actions, and results to see what happened.', + setupAction: 'Try FSQ locally', mediaAction: 'Watch the demo', }, workflow: { title: 'From goal to evidence.', - introduction: - 'FSQ explores the real app with your configured Provider while you keep authority over the target, boundaries, and what happens next.', + introduction: 'One journey: find the FSQ repository, follow its website link, and check the homepage. These recorded Android screens show the task, execution, and result; the surrounding explanations are editorial.', steps: [ { - title: 'Describe the outcome.', - body: 'Choose the platform and exact target. State the goal, observable success criteria, allowed actions, and forbidden actions.', + title: 'Describe the journey and the checks.', + body: 'Search for FSQ-lab/FSQ on GitHub, open the website linked in About, and verify “Describe a goal.” and “Test the real app.” on the destination page.', + image: taskUrl, width: 1800, height: 1040, + alt: 'Editorial task card with a real Quick Test goal: find the FSQ repository, follow About to its website, and verify both homepage headlines.', + caption: 'Task explanation around a recorded Quick Test goal. Define the outcome before execution.', }, { - title: 'Inspect what happened.', - body: 'FSQ operates the real UI and records screenshots, interface state, and events so the result can be examined instead of taken on trust.', + title: 'Watch AI operate the real app.', + body: 'The recorded Android run navigates the browser while FSQ captures actions and screen evidence. The execution view connects the live device screen with the action timeline.', + image: executionUrl, width: 1593, height: 705, + alt: 'Recorded FSQ execution view showing the Android browser beside the test action timeline.', + caption: 'Real execution capture from the Android demonstration.', }, { - title: 'Review before replay.', - body: 'A run may produce a Case. Review the evidence and any returned Case first; deterministic replay is a separate user choice.', + title: 'Check the result against the evidence.', + body: 'The captured destination shows both required headlines. The Run report separately shows execution success, verification success, and complete evidence. Review the recorded page as well as the reported outcome.', + image: resultUrl, width: 1800, height: 1040, + alt: 'Editorial result card pairing the captured FSQ homepage headlines with the same test’s Run report showing execution and verification success.', + caption: 'Recorded page and Run report, with editorial explanation. A result remains inspectable.', }, ], }, platforms: ['Web', 'Android', 'Windows', 'macOS'], - platformLine: { - label: 'Supported targets:', - qualifier: 'subject to', - action: 'platform-specific prerequisites', - }, + platformLine: { label: 'Supported targets:', qualifier: 'subject to', action: 'platform-specific prerequisites' }, setup: { - label: 'Get started locally', - title: 'Choose how to begin.', - introduction: - 'Coding Agent and FSQ UI are equal alternatives. Both run FSQ locally and require Python 3.11 plus platform-specific prerequisites.', + label: 'Get started locally', title: 'Choose how to begin.', + introduction: 'Two equal ways to run one Quick Test. You need Python 3.11+, a supported configured Provider and selected model, and platform-specific prerequisites.', + quickTest: 'Quick Test needs no manual project Workspace setup and does not save a formal Case or perform Strict Replay. Review the result and evidence first.', codingAgent: { title: 'Coding Agent', - body: 'Install the FSQ Skill in your project, then ask your Coding Agent to test a real user goal through the approval-aware workflow.', + body: 'Add the FSQ Skill, then ask your Coding Agent to run the goal below with FSQ.', warning: 'Never paste Provider credentials into Coding Agent chat.', + details: 'Skill setup and first run', + detail: 'Follow the approval-aware setup guide to install or upgrade fsq-agent and run fsq init --agent skill in your project. Reload the Coding Agent, choose Web and an available browser, then ask: “Use FSQ Quick Test for this goal” and paste the goal below. The Skill discovers readiness and reports actual progress and evidence.', action: 'Open Coding Agent setup', }, fsqUi: { title: 'FSQ UI', - body: 'Open the local interface to configure a Workspace, select an exact target, describe a goal, and inspect the resulting evidence.', + body: 'Install FSQ, open the local UI, and choose Quick Test. Paste the same goal below.', warning: 'Configure your Provider and model in the local FSQ UI.', + details: 'Install and start Quick Test', + detail: 'Run the commands below. In Settings, configure a supported Provider and explicitly select a model. Return to Quick Test, select Web and an available browser, paste the goal, and choose Start Quick Test. Open the report to inspect the result and evidence.', action: 'Open FSQ UI setup', }, }, setupCommands: ['python -m pip install --upgrade fsq-agent', 'fsq ui'], commandPanel: { - label: 'Terminal', - copyAction: 'Copy commands', - copiedAction: 'Copied', + label: 'Terminal', copyAction: 'Copy commands', copiedAction: 'Copied commands', accessibleName: 'Commands to install FSQ and open the local UI', - success: 'Commands copied. Paste them into your terminal when ready.', - failure: 'Could not copy automatically. Select and copy the commands above.', + success: 'Commands copied.', failure: 'Could not copy automatically. Select and copy the commands above.', + }, + example: { + title: 'Try a complete user journey.', + introduction: 'Start with Web in your selected browser. The recorded demonstration uses Android; platform setup and execution differ. Search results and outcomes can vary.', + goal: 'Use the selected browser to search for the FSQ-lab/FSQ GitHub repository. Open that repository, follow the website link in its About section, and verify that the destination page displays both “Describe a goal.” and “Test the real app.” Report the outcome with the captured page evidence. Only browse public pages. Do not sign in, change repository content, or submit forms. Stop and report a blocker if the repository or About website link cannot be identified.', + label: 'First Quick Test goal', copyAction: 'Copy test goal', copiedAction: 'Copied test goal', + success: 'Test goal copied.', failure: 'Could not copy automatically. Select and copy the test goal above.', + }, + capabilities: { + title: 'Go further when you need to.', + introduction: 'Quick Test starts with a goal and inspectable evidence. Case management and replay are separate workflows.', + items: [ + { title: 'Inspect the run.', body: 'Review screenshots, interface state, and events in the report. Keep failures and uncertain outcomes visible alongside successes.' }, + { title: 'Manage reusable Cases.', body: 'Use FSQ Case Manager for registered Workspaces, explicit Case creation or saving when a run returns a candidate, and run history. A run may produce a Case; generation is conditional.' }, + { title: 'Choose deterministic replay.', body: 'Review any returned Case unchanged. Deterministic replay is a separate user choice with its own target checks and repeated side effects; Quick Test does not run it for you.' }, + ], + }, + community: { + title: 'Help shape FSQ.', + body: 'If FSQ is useful to you, Star the repository to help others discover it. Tried it? Share what worked, where you got stuck, and what you want to test next.', + feedback: 'Include the FSQ version, platform, sanitized goal, expected and actual outcome, and only diagnostic excerpts you have reviewed. Remove credentials, personal data, private URLs, paths, and identifiers before sharing.', }, boundary: { title: 'Alpha software, local evidence.', - body: - 'Evidence is stored locally and may contain private app data. Review it before sharing. AI operations send necessary provider-bound inputs to your configured Provider; this website does not run tests, collect credentials, or submit visitor data.', + body: 'FSQ is open-source alpha software. Evidence is stored locally and may contain private app data. Review it before sharing. AI operations send necessary provider-bound inputs to your configured Provider; this website does not run tests, collect credentials, or submit visitor data.', action: 'Read support and stability guidance', }, media: { poster: mediaPosterUrl, - source: mediaSourceUrl, - destination: 'https://youtu.be/QqCahxGDdS0', - alt: - 'FSQ Control Plane demonstration preview showing a goal-driven real application test and its inspectable evidence.', - caption: - 'Recorded demonstration of a real public-target FSQ workflow. This is not a live testing interface or a capture of current HEAD.', - provenanceLabel: 'Recorded from FSQ 0.1.0 · September 2026', - provenanceUrl: `${repositoryFile}/docs/media/fsq-control-plane-demo-provenance.md`, - width: 1920, - height: 1080, + source: 'https://github.com/user-attachments/assets/3acfecf4-bf48-4c76-8245-2eeef0dedb1f', + destination: 'https://youtu.be/f1sobvVHNg0', captions: captionsUrl, + alt: 'Android Quick Test demonstration cover showing the real device browser and FSQ execution evidence.', + caption: 'Android Quick Test: search for the FSQ repository, follow its website link, and verify both homepage headlines. English captions are available in the player.', + provenanceLabel: 'Media sources and review status', + provenanceUrl: repositoryFile + '/docs/media/quick-test-demo-provenance.md', + width: 1920, height: 1080, duration: '2:11', } satisfies WebsiteMedia, - mediaActions: { - play: 'Play the recorded demonstration', - youtube: 'Watch the full demonstration on YouTube', - }, - mediaChrome: { - title: 'FSQ / recorded demonstration', - context: 'Evidence-first workflow', - }, + mediaActions: { play: 'Play Android Quick Test demonstration', github: 'Open GitHub video', youtube: 'Watch on YouTube' }, + mediaChrome: { title: 'FSQ / Android Quick Test', context: 'Recorded real-app testing' }, footer: { - credit: 'FSQ · Built by FSQ Lab and the community.', - navigationLabel: 'Project links', - gettingStarted: 'Getting started', - security: 'Security', - contributing: 'Contribute', - license: 'MIT License', + credit: 'FSQ · Built by FSQ Lab and the community.', navigationLabel: 'Project links', + gettingStarted: 'Getting started', security: 'Security', contributing: 'Contribute', license: 'MIT License', }, links: { - repository, - codingAgent: `${repository}#1-coding-agent`, - fsqUi: `${repository}#2-fsq-ui`, - gettingStarted: `${repositoryFile}/docs/getting-started.md`, - prerequisites: `${repositoryFile}/docs/platform-prerequisites.md`, - stability: `${repositoryFile}/docs/support-and-stability.md`, - security: `${repositoryFile}/SECURITY.md`, - contributing: `${repositoryFile}/CONTRIBUTING.md`, - license: `${repositoryFile}/LICENSE`, + repository, codingAgent: repositoryFile + '/docs/getting-started.md#coding-agent', + fsqUi: repositoryFile + '/docs/getting-started.md#fsq-ui', + gettingStarted: repositoryFile + '/docs/getting-started.md', + chineseGuide: repositoryFile + '/docs/getting-started.zh-CN.md', + prerequisites: repositoryFile + '/docs/platform-prerequisites.md', + stability: repositoryFile + '/docs/support-and-stability.md', + security: repositoryFile + '/SECURITY.md', contributing: repositoryFile + '/CONTRIBUTING.md', + license: repositoryFile + '/LICENSE', feedback: repository + '/issues/new/choose', }, } as const; diff --git a/frontend/website/src/styles.css b/frontend/website/src/styles.css index 0c0f6def..d0d66d3a 100644 --- a/frontend/website/src/styles.css +++ b/frontend/website/src/styles.css @@ -47,6 +47,7 @@ a:hover { a:focus-visible, button:focus-visible, +summary:focus-visible, [tabindex]:focus-visible { outline: 3px solid var(--focus); outline-offset: 3px; @@ -307,7 +308,8 @@ h3 { } .workflow-step p, -.setup-card p { +.setup-card > div > p, +.setup-card details > p { margin: 0; color: var(--muted); font-size: 0.9rem; @@ -533,3 +535,29 @@ h3 { box-shadow: none; } } + +.media-player { width: 100%; aspect-ratio: 16 / 9; background: #16251b; border-radius: 0.5rem; overflow: hidden; } +.media-player:focus-within { outline: 3px solid var(--focus); outline-offset: 3px; } +.media-player video { display: block; width: 100%; height: 100%; } +.media-play { position: relative; display: block; width: 100%; height: 100%; padding: 0; border: 0; cursor: pointer; background: transparent; } +.media-play__label { position: absolute; bottom: 1rem; left: 50%; transform: translateX(-50%); display: flex; align-items: center; justify-content: center; gap: 0.8rem; min-height: 48px; padding: 0.6rem 1rem; border-radius: 0.4rem; background: #254e36; color: white; font-weight: 700; white-space: nowrap; box-shadow: 0 0 0 1px white; } +.media-status { min-height: 1.5em; margin-top: 0.4rem !important; color: var(--ink); } +.workflow-stories { margin-top: 3rem; display: grid; gap: 3.5rem; } +.workflow-story { min-width: 0; } +.workflow-story__copy { max-width: 760px; margin-bottom: 1.25rem; } +.workflow-story__copy h3 { margin-top: 0.5rem; font-size: clamp(1.3rem, 3vw, 1.8rem); } +.workflow-story__copy p, .first-test > p { color: var(--muted); } +.workflow-story figure { margin: 0; } +.workflow-story img { display: block; width: 100%; height: auto; border: 1px solid var(--line); border-radius: 0.5rem; } +.workflow-story figcaption { margin-top: 0.75rem; color: var(--muted); font-size: 0.85rem; } +.workflow-story figcaption a { display: inline-flex; min-height: 44px; align-items: center; gap: 0.4rem; } +.setup-card details { color: var(--muted); font-size: 0.9rem; } +.setup-card summary { min-height: 44px; cursor: pointer; color: var(--brand); font-weight: 700; } +.setup-card details > p { margin: 0.6rem 0 1rem; } +.first-test { margin-top: 3rem; max-width: 860px; } +.first-test h3 { font-size: clamp(1.5rem, 3vw, 2rem); } +.commands--wrap { white-space: pre-wrap; overflow-wrap: anywhere; line-height: 1.8; } +.community .hero__actions { justify-content: flex-start; } +.site-footer nav { display: flex; flex-wrap: wrap; gap: 0.25rem 1rem; } +.site-footer nav a { display: inline-flex; align-items: center; min-height: 44px; } +@media (max-width: 420px) { .media-play__label { bottom: 0.6rem; font-size: 0.8rem; } .command-panel__header { gap: 0.4rem; } } diff --git a/tests/test_distribution_contract.py b/tests/test_distribution_contract.py index 116bb051..fc0c1ea0 100644 --- a/tests/test_distribution_contract.py +++ b/tests/test_distribution_contract.py @@ -9,6 +9,7 @@ from pathlib import Path from typing import Any +import pytest import yaml from packaging.requirements import Requirement from packaging.utils import canonicalize_name @@ -290,170 +291,167 @@ def test_canonical_skill_resources_are_checked_out_with_lf() -> None: assert attributes[(resource, "eol")] == "lf" -def test_concise_readmes_cover_the_same_install_and_ui_contract() -> None: - readmes = { - "en": (ROOT / "README.md").read_text(encoding="utf-8"), - "zh": (ROOT / "README.zh-CN.md").read_text(encoding="utf-8"), - } - - for language, readme in readmes.items(): - assert len(readme.splitlines()) <= 105 - for required in ( - "https://github.com/FSQ-lab/FSQ", - "python -m pip install --upgrade fsq-agent", - "fsq ui", - "docs/media/01-describe-goal.png", - "aa9d0a12-2f93-4894-8349-52a013424939", - "QqCahxGDdS0", - ): - assert required in readme +def _markdown_section(document: str, heading: str) -> str: + start = re.search(rf"(?m)^{re.escape(heading)}$", document) + assert start is not None, heading + level = len(heading.split(" ", 1)[0]) + rest = document[start.end() :] + end = re.search(rf"(?m)^#{{1,{level}}} ", rest) + return rest[: end.start()] if end else rest + + +def test_readmes_share_current_media_and_resolvable_local_destinations() -> None: + current_video = "https://github.com/user-attachments/assets/3acfecf4-bf48-4c76-8245-2eeef0dedb1f" + current_youtube = "https://youtu.be/f1sobvVHNg0" + expected_images = {f"docs/media/quick-test-{name}.png" for name in ("poster", "task", "execution", "result")} + for name in ("README.md", "README.zh-CN.md"): + path = ROOT / name + readme = path.read_text(encoding="utf-8") + assert current_video in readme + assert current_youtube in readme + assert "aa9d0a12-2f93-4894-8349-52a013424939" not in readme + assert "QqCahxGDdS0" not in readme + images = re.findall(r"!\[([^\]]+)\]\((docs/media/[^)]+)\)", readme) + assert {target for _, target in images} == expected_images + for alt, target in images: + assert alt.strip() + assert (ROOT / target).read_bytes().startswith(b"\x89PNG\r\n\x1a\n") + assert "docs/media/quick-test.en.srt" in readme + assert "docs/media/quick-test-demo-provenance.md" in readme + assert "https://fsq-lab.github.io/FSQ/" in readme + assert "Web" in readme + assert "Android" in readme assert readme.index("FSQ Lab") < readme.index("##") - documentation_heading = {"en": "## Documentation", "zh": "## 文档"}[language] - assert readme.index("> [!IMPORTANT]") < readme.index(documentation_heading) + assert "> [!IMPORTANT]" in readme + + # Check actual linked files and heading fragments, not just expected strings. + targets = re.findall(r'\]\(([^)\s]+)\)|(?:href|src|srcset)="([^"]+)"', readme) + assert targets + for markdown_target, html_target in targets: + target = markdown_target or html_target + if target.startswith(("https://", "http://")): + continue + local, _, fragment = target.partition("#") + destination = path.parent / local if local else path + assert destination.is_file(), target + if fragment: + headings = re.findall(r"(?m)^#{1,6} (.+)$", destination.read_text(encoding="utf-8")) + anchors = {re.sub(r"[^\w\s-]", "", heading).lower().replace(" ", "-") for heading in headings} + assert fragment in anchors, target + + srt = (ROOT / "docs/media/quick-test.en.srt").read_text(encoding="utf-8") + assert re.search(r"\d{2}:\d{2}:\d{2},\d{3} --> ", srt) + expected_vtt = "WEBVTT\n\n" + re.sub(r"(\d{2}:\d{2}:\d{2}),(\d{3})", r"\1.\2", srt) + assert (ROOT / "docs/media/quick-test.en.vtt").read_text(encoding="utf-8") == expected_vtt def test_bilingual_readmes_present_two_complete_quick_start_paths() -> None: - contracts = { - "en": { - "path": ROOT / "README.md", - "quick_start": "## Quick start", - "agent": "### 1. Coding Agent", - "ui": "### 2. FSQ UI", - "reload": "reload or start a new Coding Agent session", - "prompt": "> Use FSQ to test this user goal", - "setup_approval": "one approval for the listed changes", - "setup_scope": "does not initialize a Workspace, configure a Provider, or run a test", - "first_test": ("Microsoft Edge Stable", "Bing", "`FSQ GitHub`", "organic search results", "current position/ranking"), - "target_guidance": "facts or user-owned decisions", - "optional_case": "if FSQ returns a Case", - "replay": "separately requesting any replay", - }, - "zh": { - "path": ROOT / "README.zh-CN.md", - "quick_start": "## 快速开始", - "agent": "### 1. Coding Agent", - "ui": "### 2. FSQ UI", - "reload": "重新加载 Coding Agent 或启动新 session", - "prompt": "> 使用 FSQ 测试这个用户目标", - "setup_approval": "一次修改批准请求", - "setup_scope": "不会初始化 Workspace、配置 Provider 或运行测试", - "first_test": ("Microsoft Edge Stable", "Bing", "`FSQ GitHub`", "自然搜索结果", "当前所在的位置/排名"), - "target_guidance": "事实或由用户决定的事项", - "optional_case": "如果 FSQ 返回了 Case", - "replay": "另行决定是否请求重放", - }, - } - - for contract in contracts.values(): - readme = contract["path"].read_text(encoding="utf-8") - quick_start = readme[readme.index(contract["quick_start"]) :] - agent_section = quick_start[quick_start.index(contract["agent"]) : quick_start.index(contract["ui"])] - ui_section = quick_start[quick_start.index(contract["ui"]) :] - - assert agent_section.index("https://github.com/FSQ-lab/FSQ") < agent_section.index("fsq init --agent skill") - assert agent_section.index("fsq init --agent skill") < agent_section.index(contract["reload"]) < agent_section.index(contract["prompt"]) - assert contract["setup_approval"] in agent_section - assert contract["setup_scope"] in agent_section - assert all(value in agent_section for value in contract["first_test"]) - assert contract["target_guidance"] in agent_section - assert "[describe the outcome]" not in agent_section - assert "[描述期望结果]" not in agent_section - assert "Skill" in agent_section - assert contract["optional_case"] in quick_start - assert contract["replay"] in quick_start - assert "mutation approval" not in agent_section - assert "mutation 批准" not in agent_section - assert "bootstrap" not in agent_section - assert all(value in ui_section for value in ("python -m pip install --upgrade fsq-agent", "fsq ui", "Settings", "Workspace")) + contracts = ( + ("README.md", "## Quick start", "reload or start a new Coding Agent session", "one approval", "does not initialize a Workspace, configure a Provider, or run a test"), + ("README.zh-CN.md", "## 快速开始", "重新加载 Coding Agent 或启动新 session", "一次批准请求", "不初始化 Workspace、不配置 Provider、不运行测试"), + ) + for name, heading, reload, approval, setup_boundary in contracts: + readme = (ROOT / name).read_text(encoding="utf-8") + quick_start = _markdown_section(readme, heading) + assert not re.search(r"(?m)^### \d+[.)]\s*(?:Coding Agent|FSQ UI)", quick_start) + sections = {title: _markdown_section(quick_start, f"### {title}") for title in ("Coding Agent", "FSQ UI")} + for title, section in sections.items(): + details = re.findall(r"
\s*[^<]+(.*?)
", section, re.DOTALL) + assert len(details) == 1 + assert section[: section.index("
")].strip(), "Keep a concise entry before setup details" + assert "Quick Test" in section + anchor = "coding-agent" if title == "Coding Agent" else "fsq-ui" + assert re.search(rf"\]\(docs/getting-started(?:\.zh-CN)?\.md#{anchor}\)", section) + agent = sections["Coding Agent"] + assert agent.index("https://github.com/FSQ-lab/FSQ") < agent.index("fsq init --agent skill") < agent.index(reload) + assert approval in agent + assert setup_boundary in agent + assert "Skill" in agent + assert "Web" in agent + ui = sections["FSQ UI"] + ui_details = re.search(r"
(.*?)
", ui, re.DOTALL) + assert ui_details is not None + assert "python -m pip install --upgrade fsq-agent" in ui_details[1] + assert "fsq ui" in ui_details[1] + assert all(marker in ui_details[1] for marker in ("Settings", "Provider", "Web", "Start Quick Test", "Open report")) + + goals = re.findall(r"```text\n(.*?)\n```", quick_start, re.DOTALL) + assert len(goals) == 1 + goal = goals[0] + assert goal.index("FSQ-lab/FSQ") < goal.index("About") < goal.index("Describe a goal.") < goal.index("Test the real app.") + assert "[describe the outcome]" not in goal + assert "[描述期望结果]" not in goal + if name == "README.md": + assert all( + marker in goal + for marker in ("search", "selected browser", "website link", "captured page evidence", "Only browse public pages", "Do not sign in", "submit forms", "Stop and report a blocker") + ) + else: + assert all(marker in goal for marker in ("搜索", "已选择的浏览器", "官网链接", "页面证据", "只浏览公开页面", "不登录", "不提交表单", "停止并报告阻碍")) + guide_name = "getting-started.zh-CN.md" if "zh-CN" in name else "getting-started.md" + guide = (ROOT / "docs" / guide_name).read_text(encoding="utf-8") + assert goal in guide, "README and detailed guide must share the complete first goal" def test_bilingual_getting_started_guides_follow_the_guided_skill_protocol() -> None: - contracts = { - "en": { - "path": ROOT / "docs" / "getting-started.md", - "reload": "Reload or start a new Coding Agent session", - "guided": "## Guided AI Explore", - "discovery": "exact opened project root", - "doctor_first": "before asking", - "scope": "platform and exact current target", - "workspace": "exact `/fsq_workspace`", - "goal": "smallest cohesive group", - "permissions": "permissions", - "provider": "Provider handoff", - "result": "result first", - "returned": "exact returned Case", - "optional_review": "only if you ask", - "static": "fsq case format", - "confirmation": "second explicit confirmation", - "fallback": "Provider-free Web troubleshooting fallback", - }, - "zh": { - "path": ROOT / "docs" / "getting-started.zh-CN.md", - "reload": "重新加载或启动新的 Coding Agent session", - "guided": "## 引导式 AI Explore", - "discovery": "准确打开的项目根目录", - "doctor_first": "先运行 Doctor", - "scope": "平台和准确的当前目标", - "workspace": "准确的 `/fsq_workspace`", - "goal": "最小且内聚的一组", - "permissions": "权限", - "provider": "Provider 交接", - "result": "先总结结果", - "returned": "准确返回的 Case", - "optional_review": "仅在你提出请求时", - "static": "fsq case format", - "confirmation": "第二次明确确认", - "fallback": "无需 Provider 的 Web 故障排查后备流程", - }, - } - - for contract in contracts.values(): - guide = " ".join(contract["path"].read_text(encoding="utf-8").split()) - install_markers = ( - "python -m pip install --upgrade fsq-agent", - "fsq init --agent skill", - contract["reload"], - ) - install_positions = [guide.index(marker) for marker in install_markers] - assert install_positions == sorted(install_positions) - - protocol = guide[guide.index(contract["guided"]) :] - assert protocol.index(contract["discovery"]) < protocol.index(contract["doctor_first"]) - assert protocol.index(contract["doctor_first"]) < protocol.index(contract["scope"]) - scope_markers = ( - contract["scope"], - contract["workspace"], - contract["goal"], - contract["permissions"], - ) - scope_positions = [protocol.index(marker) for marker in scope_markers] - ordered_markers = ( - contract["provider"], - "case create", - contract["result"], - contract["returned"], - contract["optional_review"], - contract["static"], - contract["confirmation"], - "case test", - contract["fallback"], - ) - positions = [protocol.index(marker) for marker in ordered_markers] - assert positions == sorted(positions) - assert max(scope_positions) < positions[0] - assert "assets/getting-started.web.fsq.yaml" in protocol[positions[-1] :] - - for obsolete_claim in ( - "installed Skill cannot", - "Skill cannot configure", - "cannot run Explore", - "relative Control Plane route", - "安装后的 Skill 不可以", - "Skill 也不能配置", - "运行 Explore/suggestion", - "相对 Control Plane 路由", - ): - assert obsolete_claim not in guide + contracts = ( + ( + "getting-started.md", + "## Review results before choosing advanced workflows", + "## Provider-free Web troubleshooting fallback", + "No registered project Workspace, manual project initialization, or project Doctor is required", + "authorizes one normal Quick Test without another FSQ-layer run confirmation", + "never automatically resubmits", + "Only your explicit confirmation after that disclosure", + "not the primary experience or a prerequisite", + ), + ( + "getting-started.zh-CN.md", + "## 检查结果\uff0c再选择进阶操作", + "## 无需 Provider 的 Web 故障排查后备流程", + "无需注册或初始化项目 Workspace\uff0c也无需先运行项目 Doctor", + "授权一次正常 Quick Test\uff0c无需另一层 FSQ 执行确认", + "不会自动重提", + "你明确确认后才执行一次 Strict Replay", + "不是主要体验\uff0c也不是用户 Goal 的前置步骤", + ), + ) + for name, review_heading, fallback_heading, no_project_setup, one_request, no_retry, replay_confirmation, fallback_boundary in contracts: + guide = (ROOT / "docs" / name).read_text(encoding="utf-8") + first_use = guide[: guide.index(review_heading)] + assert "~/.fsq/quick-test/" in first_use + assert no_project_setup in first_use + assert "/fsq_workspace" not in first_use + agent = _markdown_section(guide, "## Coding Agent") + assert agent.index("fsq init --agent skill") < agent.index(".agents/skills/fsq/") + assert agent.index("quick-test context") < agent.index("quick-test run") + assert one_request in agent + assert no_retry in agent + assert all(marker in agent for marker in ("JSONL", "Settings", "Provider")) + assert "case create" not in agent + assert "case test" not in agent + if name == "getting-started.md": + assert all( + marker in agent + for marker in ( + "Never put keys, tokens, or secrets in chat", + "failure, timeout, interruption, lost output, or malformed output", + "authoritative execution status", + "whole-goal verification", + ) + ) + else: + assert all(marker in agent for marker in ("不要将 Key、token 或 secret 放进聊天", "失败、超时、中断、输出丢失或格式错误", "权威执行状态", "整体验证结论")) + ui = _markdown_section(guide, "## FSQ UI") + assert all(marker in ui for marker in ("python -m pip install --upgrade fsq-agent", "fsq ui", "Quick Test", "Settings", "Start Quick Test", "Open report")) + advanced = _markdown_section(guide, review_heading) + assert "FSQ Case Manager" in advanced + assert "Doctor" in advanced + assert replay_confirmation in advanced + fallback = " ".join(_markdown_section(guide, fallback_heading).split()) + assert "assets/getting-started.web.fsq.yaml" in fallback + assert fallback_boundary in fallback + assert guide.index(review_heading) < guide.index(fallback_heading) < guide.index("case create") def test_changelog_keeps_released_skill_behavior_historical() -> None: @@ -504,68 +502,76 @@ def test_ci_verifies_all_runtime_package_resources() -> None: assert "uv run --no-sync python -m pytest" in workflow -def test_readme_uses_current_installation_and_cli_contract() -> None: - readme = (ROOT / "README.md").read_text(encoding="utf-8") - - headline = "Describe what to test. Let AI help to test the user flow." +@pytest.mark.parametrize( + ("name", "headline", "quick_heading", "advanced_heading", "feedback_heading", "safety_markers"), + [ + ( + "README.md", + "Describe a goal. Test the real app.", + "## Quick start", + "## Go further when you need to", + "## Help shape FSQ", + ( + "supported configured Provider and explicitly selected model", + "never paste them into Coding Agent chat", + "no manual project Workspace setup", + "does not save a formal Case", + "does not perform Strict Replay", + "Evidence is stored locally", + "may contain private app data", + "review it before sharing", + "alpha", + ), + ), + ( + "README.zh-CN.md", + "描述测试目标\uff0c让 AI 测试真实应用。", + "## 快速开始", + "## 需要时再进阶", + "## 一起完善 FSQ", + ( + "已配置的受支持 Provider 与明确选择的模型", + "不要发送到 Coding Agent 聊天", + "无需手动初始化项目 Workspace", + "不保存正式 Case", + "不执行 Strict Replay", + "证据保存在本地", + "可能包含私有应用数据", + "审查", + "Alpha", + ), + ), + ], + ids=("english", "chinese"), +) +def test_readme_uses_current_installation_and_cli_contract(name: str, headline: str, quick_heading: str, advanced_heading: str, feedback_heading: str, safety_markers: tuple[str, ...]) -> None: + readme = (ROOT / name).read_text(encoding="utf-8") assert readme.index(headline) < readme.index("FSQ Lab") - assert "python -m pip install --upgrade fsq-agent" in readme - assert "## Quick start" in readme - assert "### 1. Coding Agent" in readme - assert "### 2. FSQ UI" in readme - assert "fsq init --agent skill" in readme - assert "does not initialize a Workspace, configure a Provider, or run a test" in readme - assert "Use FSQ to test this user goal" in readme - assert all(detail in readme for detail in ("Microsoft Edge Stable", "Bing", "`FSQ GitHub`", "organic search results", "current position/ranking")) - assert "facts or user-owned decisions" in readme - assert "[describe the outcome]" not in readme - assert "README.zh-CN.md" in readme - assert "docs/getting-started.zh-CN.md" in readme - assert "![FSQ Test Runner for describing a test goal](docs/media/01-describe-goal.png)" in readme - assert "https://github.com/user-attachments/assets/aa9d0a12-2f93-4894-8349-52a013424939" in readme - assert "https://youtu.be/QqCahxGDdS0" in readme - assert "Autoplay demo page" not in readme - assert "English captions" not in readme - assert "docs/demo.html" not in readme - assert "Bilibili" not in readme - assert "fsq ui" in readme - for obsolete in ( - "fsq-agent run", - "run --strict", - "fsq-agent control-plane", - "fsq-agent report", - "fsq-agent playground", - "init --name", - "fsq-agent[web]", - "fsq-agent[android]", - "fsq-agent[windows]", - "fsq-agent[macos]", + assert "Python 3.11+" in _markdown_section(readme, quick_heading) + assert all(marker in readme for marker in safety_markers) + advanced = _markdown_section(readme, advanced_heading) + assert "FSQ Case Manager" in advanced + assert "Strict Replay" in advanced + if name == "README.md": + assert "A run may produce a Case" in advanced + assert "separately request Strict Replay" in advanced + assert "Quick Test never runs it automatically" in advanced + else: + assert "Run 可能产生 Case" in advanced + assert "单独请求 Strict Replay" in advanced + assert "Quick Test 不会自动重放" in advanced + assert readme.index(quick_heading) < readme.index(advanced_heading) < readme.index(feedback_heading) + assert "https://github.com/FSQ-lab/FSQ/issues/new/choose" in _markdown_section(readme, feedback_heading) + for link in ( + "README.md" if "zh-CN" in name else "README.zh-CN.md", + "docs/getting-started.zh-CN.md", + "docs/platform-prerequisites.md", + "docs/support-and-stability.md", + "SECURITY.md", + "CONTRIBUTING.md", + "LICENSE", ): - assert obsolete not in readme - - -def test_chinese_readme_uses_current_installation_and_cli_contract() -> None: - readme = (ROOT / "README.zh-CN.md").read_text(encoding="utf-8") - - headline = "描述你要测试什么\uff0c让 AI 帮你测试用户流程。" - assert readme.index(headline) < readme.index("FSQ Lab") - assert "python -m pip install --upgrade fsq-agent" in readme - assert "## 快速开始" in readme - assert "### 1. Coding Agent" in readme - assert "### 2. FSQ UI" in readme - assert "fsq init --agent skill" in readme - assert "不会初始化 Workspace、配置 Provider 或运行测试" in readme - assert "测试这个用户目标" in readme - assert all(detail in readme for detail in ("Microsoft Edge Stable", "Bing", "`FSQ GitHub`", "自然搜索结果", "当前所在的位置/排名")) - assert "事实或由用户决定的事项" in readme - assert "[描述期望结果]" not in readme - assert "![用于描述测试目标的 FSQ Test Runner](docs/media/01-describe-goal.png)" in readme - assert "https://github.com/user-attachments/assets/aa9d0a12-2f93-4894-8349-52a013424939" in readme - assert "https://youtu.be/QqCahxGDdS0" in readme - assert "自动播放演示页" not in readme - assert "中文字幕" not in readme - assert "Bilibili" not in readme - assert "fsq ui" in readme + assert re.search(rf"(?:\]\(|href=\"){re.escape(link)}(?:\)|\")", readme) for obsolete in ( "fsq-agent run", "run --strict",