diff --git a/.agents/commands/autopilot-playtest.md b/.agents/commands/autopilot-playtest.md new file mode 100644 index 0000000..c725e93 --- /dev/null +++ b/.agents/commands/autopilot-playtest.md @@ -0,0 +1,8 @@ +--- +name: autopilot-playtest +description: "Run an automated playtest of the open Godot game with Autopilot" +skill: autopilot-playtest +category: qa +--- + +Invokes `/autopilot-playtest` skill. diff --git a/.agents/modules/art/mcp/aseprite.json b/.agents/modules/art/mcp/aseprite.json index 892a08f..fc8d117 100644 --- a/.agents/modules/art/mcp/aseprite.json +++ b/.agents/modules/art/mcp/aseprite.json @@ -9,7 +9,7 @@ "-m", "aseprite_mcp" ], - "enabled": true, + "codemode": false, "environment": { "ASEPRITE_PATH": "{env:ASEPRITE_PATH}" } diff --git a/.agents/modules/core/skills/setup-engine/SKILL.md b/.agents/modules/core/skills/setup-engine/SKILL.md index 64a5665..45b6e48 100644 --- a/.agents/modules/core/skills/setup-engine/SKILL.md +++ b/.agents/modules/core/skills/setup-engine/SKILL.md @@ -470,23 +470,23 @@ After the package imports, MCP for Unity opens a **setup wizard** automatically: #### Manual MCP config (reliable fallback) -Add this block to the `mcp` object in your harness config (`opencode.json` for OpenCode, `pi.json` for Pi): +Add this block to `mcp.servers` in your harness config (`opencode.json` for OpenCode, `pi.json` for Pi): ```json "unity": { - "type": "local", + "type": "remote", "url": "http://localhost:8080/mcp", - "enabled": false + "disabled": true } ``` -`enabled: false` is the default. Flip to `true` once Unity Editor is running and you want OCGS agents to use unity-mcp tools. +`disabled: true` keeps it off (servers connect automatically otherwise). Set `disabled` to `false` once Unity Editor is running and you want OCGS agents to use unity-mcp tools. > ⚠ **Unity Editor must be running** before OCGS agents can use unity-mcp tools. If the Editor is closed, MCP calls will fail with a connection error. Open your project in Unity, then continue. #### Verify -In a session where `enabled: true`, call any unity-mcp tool (e.g. `list_scenes`) via OCGS. Expect a list of scenes from your open Unity project. +In a session where `disabled` is `false`, call any unity-mcp tool (e.g. `list_scenes`) via OCGS. Expect a list of scenes from your open Unity project. #### Troubleshooting @@ -497,6 +497,84 @@ In a session where `enabled: true`, call any unity-mcp tool (e.g. `list_scenes`) --- +### 7.5. Configure Autopilot Playtesting (Optional — Godot 4.7+, OpenCode, Pi or omp) + +> **What it is:** Autopilot, by the developers of Aura (Apache-2.0), plays the game open in the Godot editor and reports what it finds. Its results are automated evidence, never a human playtest. See [github.com/ramenvr/autopilot](https://github.com/ramenvr/autopilot) and the `/autopilot-playtest` skill. + +**Gate:** Offer this step only if all three conditions hold. If any fails, skip the step, report "Autopilot: skipped (needs Godot 4.7+ and OpenCode, Pi or omp)" in the Output Summary, and do not offer it. +- The engine chosen in this setup is Godot. If it is Unity, Unreal, or another engine, this step does not apply, even when `docs/engine-reference/godot/VERSION.md` exists. +- The Godot version is 4.7 or later. Use the version chosen in this setup, or else the pin in `docs/engine-reference/godot/VERSION.md`. Autopilot's Godot addon documents Godot 4.7 or later and has no runtime version check. +- The project's harness is OpenCode, Pi or omp, asked below. Autopilot's MCP wiring is verified on OpenCode 2.2.5, Pi 1.x and omp 18.16.0. + +Ask with `question`: +- **Prompt**: "Which agent harness do you use for this project?" +- **Options**: `OpenCode` / `Pi` / `omp` / `Other` + +If the answer is `Other`, skip as above. Other harnesses are not verified. + +Ask with `question`: +- **Prompt**: "Would you like to set up Autopilot, an optional automated playtester for the open Godot editor?" +- **Options**: `No, skip for now (Recommended)` / `Yes, set it up` + +If the user picks No, report "Autopilot: not set up" in the Output Summary and write nothing. + +If the user picks Yes: + +1. **Tell the user what they must do themselves.** Setup cannot install these, and keys must never be pasted into this chat: + - Install the addon from Autopilot's download into the game project's `addons/autopilot/`. Do not copy it into this repository. Enable it in Project > Project Settings > Plugins, then commit the `project.godot` changes it makes. + - Install Autopilot's agent service, which puts the `autopilot` command on PATH, and Bun. + - Put your own keys in Autopilot's `autopilot.env`. + - In the dock's Privacy step, turn sharing off. + - Exclude `addons/autopilot/*` from export presets. + - The full steps are in the Setup section of `/autopilot-playtest`. +2. **Wire the server.** Setup writes only inside this repository. Show the block for the chosen harness: + - **OpenCode:** check `opencode.json` for `mcp.servers.autopilot`. If it is present, leave it alone. If it is missing, show this block and ask: "May I add this `autopilot` server to `opencode.json`? It ships disabled." Write it only after the user says yes: + +```json +"autopilot": { + "type": "local", + "command": ["autopilot", "mcp"], + "disabled": true, + "codemode": false +} +``` + + - **Pi:** Pi ignores a project `.pi/mcp.json` until the project is trusted, and this server is per machine. Show this block and ask the user to add it to their user-level `~/.pi/agent/mcp.json`. Do not write it yourself: + +```json +{ + "mcpServers": { + "autopilot": { + "command": "autopilot", + "args": ["mcp"], + "enabled": false, + "exposure": "direct" + } + } +} +``` + + Keep the entry disabled. `exposure: "direct"` is required: the default `codemode` does not show the tools to the model. + + - **omp:** omp reads a project `.omp/mcp.json`. If it already has `mcpServers.autopilot`, leave it alone. If it is missing, show this block and ask: "May I add this `autopilot` server to `.omp/mcp.json`? It ships disabled." Write it only after the user says yes: + +```json +{ + "mcpServers": { + "autopilot": { + "type": "stdio", + "command": "autopilot", + "args": ["mcp"], + "enabled": false + } + } +} +``` + + Keep the entry disabled. Restart omp after enabling it. +3. Do not start a run from this step. +4. In the Output Summary, report "Autopilot: set up; addon, agent service, and keys still to do". + ## 8. Update AGENTS.md Import Ask: "May I update the `@` import in `AGENTS.md` to point to the new engine reference?" @@ -666,6 +744,7 @@ Reference Docs: [created/skipped] AGENTS.md: [updated] Tech Prefs: [created/updated] Agent Config: [verified] +Autopilot: [set up, steps pending | not set up | skipped (needs Godot 4.7+ and OpenCode, Pi or omp)] Next Steps: 1. Review docs/engine-reference//VERSION.md diff --git a/.agents/modules/installed.json b/.agents/modules/installed.json index 231f21d..fb410ce 100644 --- a/.agents/modules/installed.json +++ b/.agents/modules/installed.json @@ -227,6 +227,7 @@ "agents/qa-lead.md", "agents/qa-tester.md", "agents/security-engineer.md", + "commands/autopilot-playtest.md", "commands/bug-report.md", "commands/bug-triage.md", "commands/qa-plan.md", @@ -242,6 +243,7 @@ "commands/test-setup.md", "rules/test-standards.md", "skills/automated-smoke-test/SKILL.md", + "skills/autopilot-playtest/SKILL.md", "skills/bug-report/SKILL.md", "skills/bug-triage/SKILL.md", "skills/perf-profile/SKILL.md", diff --git a/.agents/modules/qa/commands/autopilot-playtest.md b/.agents/modules/qa/commands/autopilot-playtest.md new file mode 100644 index 0000000..c725e93 --- /dev/null +++ b/.agents/modules/qa/commands/autopilot-playtest.md @@ -0,0 +1,8 @@ +--- +name: autopilot-playtest +description: "Run an automated playtest of the open Godot game with Autopilot" +skill: autopilot-playtest +category: qa +--- + +Invokes `/autopilot-playtest` skill. diff --git a/.agents/modules/qa/modulefile.yaml b/.agents/modules/qa/modulefile.yaml index b7e8d30..67b3ca8 100644 --- a/.agents/modules/qa/modulefile.yaml +++ b/.agents/modules/qa/modulefile.yaml @@ -7,10 +7,10 @@ provides: skills: [team-qa, qa-plan, smoke-check, soak-test, regression-suite, test-setup, test-helpers, test-flakiness, test-evidence-review, automated-smoke-test, bug-report, bug-triage, security-audit, perf-profile, team-polish, - skill-test, skill-improve] + skill-test, skill-improve, autopilot-playtest] commands: [team-qa, qa-plan, smoke-check, soak-test, regression-suite, test-setup, test-helpers, test-flakiness, test-evidence-review, bug-report, bug-triage, - security-audit, team-polish] + security-audit, team-polish, autopilot-playtest] rules: [test-standards] plugged-into: engines: [] diff --git a/.agents/modules/qa/skills/autopilot-playtest/SKILL.md b/.agents/modules/qa/skills/autopilot-playtest/SKILL.md new file mode 100644 index 0000000..ada8459 --- /dev/null +++ b/.agents/modules/qa/skills/autopilot-playtest/SKILL.md @@ -0,0 +1,91 @@ +--- +name: autopilot-playtest +description: "Playtest the game open in the Godot editor with Autopilot's MCP server: explore for bugs, test a level, verify a claim, or reach a goal in play. Files the result to production/qa/playtests/. Use when asked for an automated playtest or to check gameplay with Autopilot." +argument-hint: "[explore|test|verify|goal] [request]" +user-invocable: true +allowed-tools: Read, Glob, Grep, Write, question +--- + +# Automated Playtest with Autopilot + +Autopilot plays the game open in the Godot editor. You write the harness for this request, run it through Autopilot's MCP tools, and file what play showed. + +Autopilot's own instructions (`INSTRUCTIONS.md`, named by `start_run`) are the detailed reference. This skill is the framework's wrapper around them. + +**The result is automated evidence. It is never a human playtest.** + +--- + +## Setup (once per machine and project) + +Rationale and verified behaviour: `docs/framework/autopilot-integration.md`. + +- **Harness.** OpenCode 2.2.5, Pi 1.x and omp 18.16.0 are verified. Tool names differ by harness. In this skill, a bare name such as `start_run` means Autopilot's MCP tool of that name, shown as `autopilot_start_run` in OpenCode, `mcp__autopilot__start_run` in Pi and `mcp__autopilot_start_run` in omp. Other harnesses: stop and say so, and do not edit their config. +- **Godot 4.7 or later.** Autopilot's addon README states this. The addon is tested on 4.7.2, which this framework's Godot reference is pinned to. The addon has no runtime version check, so Godot 4.6 is untested. +- **Addon.** Install `addons/autopilot/` into the game project from Autopilot's download. Enable it in Project Settings > Plugins, then commit the `project.godot` changes it makes (autoload `AutopilotPlaytest`, and `autopilot/project_id`). Do not copy the addon into this repository. +- **Agent service and Bun.** Install Autopilot's agent service, which provides the `autopilot` command that `opencode.json` runs. Install Bun, which Autopilot's default harness agent needs. +- **Keys.** Bring your own keys: a TypeSafe decision key and an LLM key, both in Autopilot's `autopilot.env`. Do not sign in to Aura. +- **Sharing.** In the dock's Privacy step, turn sharing off. Autopilot shares recordings, screenshots, traces and memories by default. +- **Releases.** Exclude `addons/autopilot/*` from export presets. + +--- + +## Phase 1: Preflight + +1. Confirm the server is enabled for this session. In OpenCode, `opencode.json` → `mcp.servers.autopilot` must have `disabled` set to `false`. In Pi, the `autopilot` entry in your `mcp.json` must have `enabled` set to `true` and `exposure` set to `direct`. Pi's own MCP view can enable the server, and after editing the file, reconnect the server there or restart Pi. In omp, the `autopilot` entry in `.omp/mcp.json` must have `enabled` set to `true`; restart omp after editing it. If it is off, ask the user whether to enable it for this session, then stop until they answer. If the entry is missing, stop and tell the user to add it from the Autopilot section of `/setup-engine`. Do not edit config files yourself. +2. Readiness. The Autopilot tools join your tool list only after the MCP server starts. That took about 5 seconds in testing and can take longer on a cold start. Check that `autopilot_start_run` (OpenCode), `mcp__autopilot__start_run` (Pi) or `mcp__autopilot_start_run` (omp) is among your tools. If it is not, make one cheap read-only call (for example `read` on `AGENTS.md`) so the next turn sees the tools, then check again. Make at most 20 such checks. If the tool is still missing, stop and tell the user: "Autopilot's MCP server did not start. Check that the `autopilot` command is on PATH, then run `opencode mcp list` (OpenCode), `pi mcp list` (Pi) or `/mcp list` (omp). Send the request again once `autopilot` shows connected." +3. Confirm the game's project is open in the Godot editor with the Autopilot dock connected. If `start_run` says the agent service isn't running, relay its message and stop. +4. Keys: never ask the user to paste a key into this chat. If a run asks for a key, tell the user where it goes (see Setup). +5. Sharing: confirm the dock's Privacy step is off (see Setup). If the user has not confirmed this, ask before the first run. + +## Phase 2: Choose the kind and start the run + +Map the request to a kind: + +- `verify`: a claim to check, e.g. "the door opens with the key". Result: CONFIRMED, DENIED or INCONCLUSIVE. +- `goal`: an outcome to reach in play, e.g. "sell three crops". Result: achieved or not achieved. +- `explore`: look for bugs, or test a level or system, e.g. "go through the level and tell me what looks wrong". A test is an exploration. + +Call `start_run` with `request` (the user's words, verbatim) and `kind`. Add `minutes` (explorations; default 10), `time_limit_minutes` (default 30) and `level` only when the user gives them. + +Start one run per request unless the user asks for more. Each run costs model tokens. + +## Phase 3: Build the harness + +1. Read the `INSTRUCTIONS.md` that `start_run` names, in full, with the MCP `read_file` tool. It is long: read it in parts. Follow it. +2. Write the harness and notes with `write_file` or `edit_file`. Those tools reach only this run's folder, `notes.md` and `lib.py` in Autopilot's memory folder for the project. They cannot touch this repository. +3. Read the game's scripts from this repository with your read and search tools as needed (`read`, `glob` and `grep` in OpenCode and omp; in Pi, `read` and `bash`). Autopilot's own copy of the code is in its memory folder. +4. Never try to change game files. Autopilot acts on the game through play only. + +## Phase 4: Run and wait + +- A call that answers "still going" has not finished. Call `wait` until it returns the result. Do not call other game tools in the meantime. +- The user can watch, stop or steer the run in the Autopilot dock. Steering text arrives with a later tool result: follow it. +- Pictures: `describe_frames` needs a vision model. Autopilot's guide assumes a frame-describer subagent, which only Claude Code and Codex get from `autopilot connect`. Call `describe_frames` only if this session's model reads images. Otherwise say the run could not check visuals, and base the report on state data. + +## Phase 5: Report + +When `report` says "Reported", the run is over. Ask: + +> May I write this playtest report to `production/qa/playtests/autopilot-[date]-[slug].md`? + +Then write the report with: + +- Kind, request (verbatim), level, time limit used, and run id. +- Outcome: the verdict or achieved state, with Autopilot's reason and evidence. +- Findings, each marked **Unverified (Autopilot)**. Confirm a bug in the game before filing it with `/bug-report`. +- Recording location (the dock's run folder, `user://playtest/