diff --git a/AGENTS.md b/AGENTS.md index 100fdd1..48db227 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,9 +22,10 @@ agent, planner, model adapter, or robotics stack. An **Operational Surface** is any bounded interface that can be observed, acted on through declared runtime capabilities, and verified by a fresh observation. It may be a software GUI, browser page, mobile screen, native app window, device panel, appliance control surface, or a -simple embodied-control surface. This is the protocol's **L0**: it defines what kind of object can -enter the visual-agent loop. The P0 contract covers the surface operation layer, not full robotics, -navigation, manipulation, or world modeling. +simple embodied-control surface. This is the protocol layer's **L0**, downstream of the +product-intent L0 in `README.md`, `README.zh-CN.md`, and `PRD.md`: it defines what kind of object +can enter the visual-agent loop. The P0 contract covers the surface operation layer, not full +robotics, navigation, manipulation, or world modeling. ## Narrative Rule @@ -80,13 +81,37 @@ contract. ## Source Of Truth -The current documentation authority chain has exactly three files: +The current documentation authority chain has exactly four files: 1. `AGENTS.md` - build charter for agents. 2. `README.md` - public English thesis and product narrative. 3. `README.zh-CN.md` - public Chinese thesis and product narrative. +4. `PRD.md` - equivalent formal L0 projection, P0 scope, non-goals, downstream chain and owner boundary. No other file is currently authoritative for public positioning, architecture, or product scope. +Protocol layer documents, schemas, fixtures, runtimes, examples and adapters are downstream +contract/manual assets under this root authority; they are not independent product authorities. + +## Formal Development Flow + +Use this in-repo L0-L4 chain as the portable development contract. Do not +drive durable protocol or runtime changes from repo-external process notes alone. +The protocol layer has its own L0-L3 terms, but those terms are downstream of +root product-intent L0: + +```text +README.md / README.zh-CN.md / PRD.md + -> AGENTS.md + -> protocol layer documents + -> schemas + fixtures + conformance checks + -> runtimes/desktop-gui + adapters + examples + -> scripts/verify-local.sh evidence +``` + +- Product L0: `README.md`, `README.zh-CN.md`, and `PRD.md` define why visual-agent exists, P0 scope, non-goals, and owner boundary. +- Protocol L1/L2: protocol documents, schemas, fixtures, and conformance checks define the verifiable surface-operation contract. +- L3: runtime code, adapters, examples, manifests, and repo commands realize the protocol under the product boundary. +- L4: protocol conformance, Swift typecheck, shell syntax, smoke checks, and `VERIFY_LOCAL_OK` provide validation evidence; `ActionResult.ok == true` and passing local checks do not prove task success or product acceptance. ## Open-Source Surface @@ -155,5 +180,5 @@ Needs confirmation: - ``` -Do not expand P1/P2 into P0. Do not rewrite the three-file root contract without explicit user +Do not expand P1/P2 into P0. Do not rewrite the four-file root contract without explicit user approval. diff --git a/PRD.md b/PRD.md new file mode 100644 index 0000000..a0d8a14 --- /dev/null +++ b/PRD.md @@ -0,0 +1,70 @@ +# visual-agent PRD + +`PRD.md` is the fixed formal L0 projection of the product narrative in `README.md` +and `README.zh-CN.md` for `visual-agent`. Together they define product intent +before protocol layer documents, schemas, fixtures, runtime code, adapters, or +integration guides are changed. + +It is intentionally thin. It does not replace the public product narrative, and +it is not a fragmented spec set, implementation plan, or runtime manual. + +## L0 Problem + +Visual models and hosts can understand visible interfaces, but safe operation +requires a project-owned boundary around which surfaces are eligible, what a +runtime may execute, how surface-specific knowledge is represented, how task +intents expand into actions, and how success is verified by fresh observation. + +`visual-agent` exists to provide that protocol-first boundary without becoming +a computer-use agent, planner, model adapter, browser automation framework, MCP +server, or robotics stack. + +## P0 Scope + +P0 is the protocol-first surface operation system: + +- Operational Surface eligibility and boundary. +- Runtime capability declaration and machine-readable action results. +- Surface manuals for concrete operational surfaces. +- Scenario requirements and manifests that expand into safe action plans. +- Fresh-observation verification after declared runtime actions. + +The current desktop GUI runtime is a reference implementation of this product +boundary, not the product boundary itself. + +## Non-Goals + +- Do not use per-layer protocol documents as independent product PRDs. +- Do not let runtime code redefine product scope or layer semantics. +- Do not hide surface eligibility inside runtime, L1, L2, or adapter code. +- Do not add app-specific, site-specific, or device-specific workflows to + generic runtime code. +- Do not treat `ActionResult.ok == true` as task success. + +## Downstream Chain + +Formal development flows from the co-equal public narrative and PRD L0 assets +into protocol and runtime assets: + +```text +README.md / README.zh-CN.md / PRD.md + -> AGENTS.md + -> protocol layer documents + -> schemas + fixtures + conformance checks + -> runtime code + adapters + examples + -> verification evidence +``` + +Protocol layer documents are downstream contract/manual assets. They can define +protocol-layer L0/L1/L2/L3 behavior, but they must remain aligned with this PRD +and the public product narrative. + +## Owner Boundary + +Human owner or architect owns product intent, layer semantics, and P0 scope. +Agents and engineers may maintain protocol documents, schemas, fixtures, and +runtime code only under that frozen boundary. + +Return to public narrative / PRD review when a change would alter product +scope, layer ownership, safety semantics, adapter boundaries, or verification +meaning. diff --git a/README.md b/README.md index 76f4fc0..11824b8 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,12 @@ what a concrete surface means, which actions are safe, and how success is verifi That structure is **visual-agent**. +This `README.md` and [`README.zh-CN.md`](./README.zh-CN.md) are the public +product narrative. [`PRD.md`](./PRD.md) is the fixed equivalent formal L0 projection: +it compresses the same product intent and P0 scope before protocol layer +documents, schemas, fixtures, runtime code, adapters, or integration guides are +changed. + ## The Problem Most computer-use, GUI automation, device-control, and embodied-control projects start from an @@ -37,7 +43,8 @@ panels, appliance controls, instrument displays, and simple embodied-control sur protocol is about operating the surface. Full robotics, navigation, grasping, force control, and world modeling are outside P0. -This is the protocol's **L0: Operational Surface Contract**. L0 defines what kind of visible +Within the protocol layer, this is the **L0: Operational Surface Contract**. It is downstream of +the product-intent L0 in the public narrative and `PRD.md`, and defines what kind of visible interface can enter the visual-agent loop before any runtime action, surface manual, or scenario manifest is considered. diff --git a/README.zh-CN.md b/README.zh-CN.md index 65e5d05..9eac056 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -6,6 +6,10 @@ 这套结构就是 **visual-agent**。 +本 `README.zh-CN.md` 和 [`README.md`](./README.md) 是面向人的公开产品叙事。[`PRD.md`](./PRD.md) +是固定的等价形式化 L0 投影：它压缩同一组产品意图和 P0 范围，再由协议层文档、schema、fixture、 +运行时代码、adapter 和集成说明承接下游实现。 + ## 问题 很多计算机使用、图形界面自动化、设备控制和简单具身控制项目，都会从智能体循环、沙箱、浏览器、控制器或机器人栈开始。这些都重要，但它们中间缺了一层：从视觉理解到可信操作之间，需要协议和交互界面说明体系。 @@ -29,7 +33,7 @@ 它可以是软件 GUI、浏览器页面、移动界面、原生应用窗口、打印机或 ATM 面板、家电控制面板、仪表显示，也可以是简单具身控制中的按钮、旋钮、指示灯和设备屏幕组合。核心协议处理的是“可见交互界面的操作”这一层；完整机器人、导航、抓取、力控和世界建模不属于 P0。 -这就是协议的 **L0：Operational Surface Contract**。L0 定义什么样的可见交互界面可以进入 visual-agent 循环，先于任何运行时动作、交互界面说明书或场景清单。 +在协议层内部，这就是 **L0：Operational Surface Contract**。它位于产品叙事和 `PRD.md` 的产品意图 L0 下游，用来定义什么样的可见交互界面可以进入 visual-agent 循环，先于任何运行时动作、交互界面说明书或场景清单。 ## 运行模型 diff --git a/integration/host-embedding.md b/integration/host-embedding.md index 04fab75..c6aea5c 100644 --- a/integration/host-embedding.md +++ b/integration/host-embedding.md @@ -1,6 +1,6 @@ # Runtime Integration - Host Embedding Guide -- Kind: Engineering integration spec (NOT a protocol layer) +- Kind: Engineering integration guide (NOT a protocol layer) - Version: 1 - Status: Normative for integration - Builds on: the visual-agent L0/L1/L2/L3 contract in `../protocol/` and diff --git a/protocol/README.md b/protocol/README.md index 6466acb..26c66b7 100644 --- a/protocol/README.md +++ b/protocol/README.md @@ -3,9 +3,10 @@ - Audience: AI agents (planners, code editors, reviewers). For humans, start at `../README.md`. - Status: Normative map for the current tracked public surface. - Reading mode: this file fuses prose with YAML so agents can orient quickly. -- Authority rule: the root files (`AGENTS.md`, `README.md`, `README.zh-CN.md`) define public - positioning and P0 scope. Per-layer protocol files define current desktop GUI reference-runtime - behavior. +- Authority rule: the root files (`AGENTS.md`, `README.md`, `README.zh-CN.md`, `PRD.md`) define + product intent. `README.md` and `README.zh-CN.md` carry the public product narrative, while + `PRD.md` carries the equivalent formal L0 projection. Per-layer protocol files define current + desktop GUI reference-runtime behavior. ## 0. What This Document Is For @@ -21,10 +22,11 @@ not make the desktop GUI runtime the product boundary. documents: root_contract: - AGENTS.md + - PRD.md - README.md - README.zh-CN.md overview: protocol/README.md - protocol_specs: + protocol_layer_docs: L0: protocol/L0-operational-surface-contract.md L1: protocol/L1-runtime-capability-manual.md L2: protocol/L2-surface-manual.md @@ -46,7 +48,7 @@ adapters: reference_rule: direction: runtime --> protocols on_scope_conflict: "root contract wins for product boundary and P0 scope" - on_runtime_behavior_conflict: "protocol spec wins over code; update spec first, then code" + on_runtime_behavior_conflict: "update the relevant protocol layer document before changing observable runtime behavior" ``` ## 1. L0-L3 Layer Map @@ -78,7 +80,7 @@ layers: - task intent expansion - robotics navigation/grasping/force/world modeling authority: root_contract - spec: protocol/L0-operational-surface-contract.md + layer_doc: protocol/L0-operational-surface-contract.md L1: name: Runtime Capability Manual manual: runtime-capability-manual @@ -92,7 +94,7 @@ layers: - operational surface semantics - business/task judgement - scenario state - current_reference_spec: L1 + current_reference_doc: L1 L2: name: Surface Manual manual: surface-manual @@ -107,7 +109,7 @@ layers: - reimplementing L1 mechanics - task aggregation/reporting - raw event dispatch - current_reference_spec: L2 + current_reference_doc: L2 L3: name: Scenario Requirements manual: scenario-requirements @@ -122,7 +124,7 @@ layers: - raw pointer/device/actuator events - runtime coordinate arithmetic as task logic - deciding task success without fresh observation - current_reference_spec: L3 + current_reference_doc: L3 ``` ## 2. Desktop GUI Reference Runtime Bridge @@ -251,7 +253,7 @@ for semantics, layer boundaries, and future extension rules. ## 5. Cross-Cutting Invariants -These invariants bind the root contract, current reference specs, and runtime files. +These invariants bind the root contract, current reference layer documents, and runtime files. ```yaml invariants: @@ -320,7 +322,7 @@ playbooks: - "if collecting data: add normalization + anonymization" - "validate every intent with task_flow --manifest preview-json" edit_runtime_code: - before: "read the file's AGENT BINDING BLOCK and locate cited spec sections" + before: "read the file's AGENT BINDING BLOCK and locate cited protocol sections" rule: "if a change alters observable behavior, update the protocol section first, then code" never: "make the protocol merely describe current code" review_change: diff --git a/runtimes/desktop-gui/README.md b/runtimes/desktop-gui/README.md index d08b818..e31b822 100644 --- a/runtimes/desktop-gui/README.md +++ b/runtimes/desktop-gui/README.md @@ -99,7 +99,7 @@ runtimes/desktop-gui/ ## Agent-Facing Documents - `../../protocol/README.md` is the agent-facing protocol map. It connects the root contract to - L0/L1/L2/L3 specs, runtime files, examples, integration docs, adapters, and invariants. + L0/L1/L2/L3 protocol documents, runtime files, examples, integration docs, adapters, and invariants. - Each `src/*.swift` file has an `AGENT BINDING BLOCK` that points code symbols to protocol sections. Runtime files cite protocol sections; the protocol is not derived from code. diff --git a/runtimes/desktop-gui/src/action_executor.swift b/runtimes/desktop-gui/src/action_executor.swift index 5a0928f..7b2e8d0 100644 --- a/runtimes/desktop-gui/src/action_executor.swift +++ b/runtimes/desktop-gui/src/action_executor.swift @@ -18,15 +18,15 @@ // l0: ../../../protocol/L0-operational-surface-contract.md // overview: ../../../protocol/README.md // implements: -// - { symbol: TargetApp, spec_section: "L1 section 3 Desktop GUI Target Binding" } -// - { symbol: appObservation, spec_section: "L1 section 4 AppObservation (desktop GUI reference)" } -// - { symbol: JSONActionPlan/runPlan, spec_section: "L1 section 5 ActionPlan, section 11 ActionResult" } -// - { symbol: executeAction, spec_section: "L1 section 6 Action Set" } -// - { symbol: eventPoint, spec_section: "L1 section 7 Coordinate Spaces" } -// - { symbol: ensurePointerSafe/runPlan, spec_section: "L1 section 8 Safety Invariants" } -// - { symbol: postMouseSequence/postEsc/postScroll, spec_section: "L1 section 9 Backends" } -// - { symbol: visualStats/imageUsableForVision/captureTargetWindowImage, spec_section: "L1 section 10 Capture Usability" } -// - { symbol: "diagnose/observe/window/screens", spec_section: "L1 section 13 Diagnostics" } +// - { symbol: TargetApp, protocol_section: "L1 section 3 Desktop GUI Target Binding" } +// - { symbol: appObservation, protocol_section: "L1 section 4 AppObservation (desktop GUI reference)" } +// - { symbol: JSONActionPlan/runPlan, protocol_section: "L1 section 5 ActionPlan, section 11 ActionResult" } +// - { symbol: executeAction, protocol_section: "L1 section 6 Action Set" } +// - { symbol: eventPoint, protocol_section: "L1 section 7 Coordinate Spaces" } +// - { symbol: ensurePointerSafe/runPlan, protocol_section: "L1 section 8 Safety Invariants" } +// - { symbol: postMouseSequence/postEsc/postScroll, protocol_section: "L1 section 9 Backends" } +// - { symbol: visualStats/imageUsableForVision/captureTargetWindowImage, protocol_section: "L1 section 10 Capture Usability" } +// - { symbol: "diagnose/observe/window/screens", protocol_section: "L1 section 13 Diagnostics" } // invariants: # see L1 section 14; violating these breaks the protocol // - actions_executed_in_order // - coordinates_default_windowTopLeft diff --git a/runtimes/desktop-gui/src/scene_runner.swift b/runtimes/desktop-gui/src/scene_runner.swift index 39eb3e4..1758d2c 100644 --- a/runtimes/desktop-gui/src/scene_runner.swift +++ b/runtimes/desktop-gui/src/scene_runner.swift @@ -18,8 +18,8 @@ // l0: ../../../protocol/L0-operational-surface-contract.md // overview: ../../../protocol/README.md // implements: -// - { symbol: loadProfile + --profile handling, spec_section: "L2 section 3 Surface Profile Binding (E1)" } -// - { symbol: process-delegation, spec_section: "L2 section 2 Layer Boundaries (no L1 re-impl)" } +// - { symbol: loadProfile + --profile handling, protocol_section: "L2 section 3 Surface Profile Binding (E1)" } +// - { symbol: process-delegation, protocol_section: "L2 section 2 Layer Boundaries (no L1 re-impl)" } // delegates_to: ./action_executor.swift # L1 executor; resolved as a sibling file // externalized_to_surface_data: # these L2 extension points live in profiles/manifests/docs, not here // - "L2 section 4 Observed States (E2)" diff --git a/runtimes/desktop-gui/src/task_flow.swift b/runtimes/desktop-gui/src/task_flow.swift index d3b7499..b126569 100644 --- a/runtimes/desktop-gui/src/task_flow.swift +++ b/runtimes/desktop-gui/src/task_flow.swift @@ -18,11 +18,11 @@ // l0: ../../../protocol/L0-operational-surface-contract.md // overview: ../../../protocol/README.md // implements: -// - { symbol: loadManifest, spec_section: "L3 section 7 Scenario Manifest (E4)" } -// - { symbol: resolvedIntentFields, spec_section: "L3 section 6 State Persistence and Merge Rules" } -// - { symbol: "substitute/buildActions", spec_section: "L3 section 4 Action-Step Templates (E2)" } -// - { symbol: actionPlan, spec_section: "L3 section 3 Intent Set, section 10 Invariants" } -// - { symbol: executePlan, spec_section: "L3 section 2 Layer Boundary (delegate, no raw events)" } +// - { symbol: loadManifest, protocol_section: "L3 section 7 Scenario Manifest (E4)" } +// - { symbol: resolvedIntentFields, protocol_section: "L3 section 6 State Persistence and Merge Rules" } +// - { symbol: "substitute/buildActions", protocol_section: "L3 section 4 Action-Step Templates (E2)" } +// - { symbol: actionPlan, protocol_section: "L3 section 3 Intent Set, section 10 Invariants" } +// - { symbol: executePlan, protocol_section: "L3 section 2 Layer Boundary (delegate, no raw events)" } // delegates_to: ./action_executor.swift # L1 executor; resolved as a sibling file // not_implemented_here: # conditional L3 extension points realized by the scenario, not the engine // - "L3 section 5 Field/Normalization Conventions (E3)" # data-collecting scenarios add their own tooling diff --git a/scripts/verify-local.sh b/scripts/verify-local.sh index 558eda6..2c9caab 100755 --- a/scripts/verify-local.sh +++ b/scripts/verify-local.sh @@ -8,6 +8,59 @@ cd "$ROOT" echo "== diff hygiene ==" git diff --check +require_contains() { + local file="$1" + local expected="$2" + grep -Fq -- "$expected" "$file" || { + echo "missing root authority marker in $file: $expected" >&2 + exit 1 + } +} + +reject_contains() { + local file="$1" + local forbidden="$2" + if grep -Fq -- "$forbidden" "$file"; then + echo "forbidden source-method leakage marker in $file: $forbidden" >&2 + exit 1 + fi +} + +echo "== root authority conformance ==" +require_contains AGENTS.md "current documentation authority chain has exactly four files" +require_contains AGENTS.md "README.md / README.zh-CN.md / PRD.md" +require_contains AGENTS.md "Product L0" +require_contains AGENTS.md "Protocol L1/L2" +require_contains AGENTS.md "VERIFY_LOCAL_OK" +require_contains README.md "fixed equivalent formal L0 projection" +require_contains README.md '[`PRD.md`](./PRD.md)' +require_contains README.zh-CN.md "PRD.md" +require_contains PRD.md "fixed formal L0 projection" +require_contains PRD.md "## P0 Scope" +require_contains PRD.md "## Downstream Chain" +require_contains PRD.md "## Owner Boundary" +require_contains PRD.md "README.md / README.zh-CN.md / PRD.md" +require_contains protocol/README.md "AGENTS.md" +require_contains protocol/README.md "PRD.md" + +echo "== source-method leakage scan ==" +for file in AGENTS.md README.md README.zh-CN.md PRD.md protocol/README.md integration/host-embedding.md; do + reject_contains "$file" "source-side method" + reject_contains "$file" "execution-context detail" + reject_contains "$file" "agent-host" + reject_contains "$file" "source-function" + reject_contains "$file" "selected workflow" + reject_contains "$file" "selected skill" + reject_contains "$file" "target runtime external method" + reject_contains "$file" "agent runtime external method" + reject_contains "$file" "runtime dependency on source" + reject_contains "$file" "devops dependency on source" + reject_contains "$file" "l0_refs" + reject_contains "$file" "l1_l2_refs" + reject_contains "$file" "l3_refs" + reject_contains "$file" "l4_validation" +done + echo "== protocol conformance ==" node --check scripts/verify-protocol-conformance.mjs node scripts/verify-protocol-conformance.mjs