Skip to content

feat(qa): add autopilot-playtest and migrate OpenCode config to V2 - #114

Merged
striderZA merged 14 commits into
masterfrom
feat/autopilot-playtest
Nov 4, 2026
Merged

feat(qa): add autopilot-playtest and migrate OpenCode config to V2#114
striderZA merged 14 commits into
masterfrom
feat/autopilot-playtest

Conversation

@striderZA

@striderZA striderZA commented Nov 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

  • Migrates the OpenCode config to the V2 schema. Under V2 the V1 MCP and permission shapes load without error and are then ignored. The migrated deny rules take effect: git reset --hard HEAD is denied under the migrated config and ran under the V1 config. The other migrated deny rules are not tested individually.
  • Adds autopilot-playtest, an optional MCP-driven playtest skill and /autopilot-playtest command. The server is disabled by default.
  • Adds the decision record docs/framework/autopilot-integration.md.

Commits

Six commits. Each passes the pre-commit and commit-msg hooks.

  1. caac7bd fix(config): load OpenCode MCP and permissions under the V2 schema
  2. b43fbab docs(opencode): describe V2 plugin SDK and MCP config shape
  3. fd97772 feat(qa): add autopilot-playtest skill and command
  4. dbe21d0 docs(framework): record the autopilot integration decision
  5. 71dabb1 fix(qa): install autopilot-playtest from the OpenCode module tree
  6. 2e3c56c docs(framework): list /autopilot-playtest in the QA command tables

Review commit 1 first. It changes behaviour: the migrated deny rules take effect. The tested case is git reset --hard HEAD; the other rules are not tested individually.

Verification

  • Hooks passed on every commit. Hook output: framework validator and workflow integrity ran for commits 1 and 3; plugin tests ran for commit 2 (177/177).
  • Framework validator on the final tree: 209/209.
  • Installer, in a scratch copy with the .opencode stubs replaced by junctions: before 71dabb1, install.mjs add qa --force left the autopilot skill and command missing. After it, add qa --force restores both, and the restored skill is byte-identical to the committed copy.
  • OpenCode 2.2.5, in a scratch copy and not this repo:
    • The migrated config loads.
    • git reset --hard HEAD returns Permission denied under the migrated config. Under the previous V1 config it was not blocked.
    • With the autopilot server enabled, the model is offered 14 autopilot_* tools, and autopilot_start_run reaches the real MCP server.
  • Startup timing: the autopilot_* tools appeared about 5 s after the run started, with a warm venv. The skill waits for autopilot_start_run before it starts a run.

Not verified

  • A live Godot run. The autopilot addon needs Godot 4.7+, and the engine reference is pinned to 4.6.
  • Slash-command expansion under V2. opencode run passes /autopilot-playtest to the model as plain text. The skill is the verified entry point.
  • Bare autopilot on PATH. Tests used absolute paths.
  • Model judgment. The model in the tests was scripted, so only tool access and plumbing were checked.

Follow-ups (not in this PR)

  • .gitignore line 17 (framework/) matches docs/framework/, so new files there are silently skipped. The decision doc was force-added. The rule's comment says it was meant for Superpowers docs. Fixing it changes which files are ignored repo-wide, so it's a separate change.
  • Two module trees, .opencode/modules/ (read by the installer) and .agents/modules/. Consolidating them is a separate decision.
  • Playtest folder paths disagree: production/qa/playtests/ vs production/playtests/. The gate artifacts live under the second.
  • UPGRADING.md still documents the V1 mcp.unity steps. Kept as release history.
  • Release export presets should exclude addons/autopilot/*. This needs a game project.

Notes

  • Pushed over HTTPS. The SSH key for github.com was rejected in this environment. The remote config is unchanged.
  • Refs Plugins don't work with OpenCode v2 API #107 (OpenCode V2 plugin API). No issue is closed by this PR.
  • Autopilot is a third-party tool (Aura, Apache-2.0). It is not vendored.

@striderZA
fix(config): load OpenCode MCP and permissions under the V2 schema
caac7bd 
OpenCode V2 reads `mcp.servers`, an ordered `permissions` array, and
`commands`. The V1 shapes (`mcp` with direct children, `enabled`, a
`permission` object, `command`) load without error and are then ignored.
Under V2 the repo's MCP servers and deny rules were not in effect,
including the deny rules for `rm -rf`, force pushes, and `sudo`.

- opencode.json: move MCP servers under `mcp.servers` using `disabled`;
  rename `command` to `commands`; translate the permission rules to the
  ordered array. `plugins` is unchanged.
- install.mjs: merge and remove MCP fragments under `mcp.servers`.
- art and setup-engine MCP instructions: use the V2 shape.

Verified in a scratch copy on OpenCode 2.2.5: under the migrated config
`git reset --hard HEAD` returns Permission denied. Under the previous V1
config the same command was not blocked.

Refs: #107
@striderZA
docs(opencode): describe V2 plugin SDK and MCP config shape
b43fbab 
- .opencode/plugins/README.md: hook examples use the V2 tool name
  `shell`; remove the V1-only hook and logging comparisons.
- docs/CONTRIBUTING.md: plugins default-export a `Plugin.define` from
  `@opencode/plugin`, matching the V2 build guide.
- docs/framework/quick-start.md, setup-requirements.md: the Unity MCP
  block uses `mcp.servers` with the remote type and `disabled`.

Refs: #107
@striderZA
feat(qa): add autopilot-playtest skill and command
fd97772 
Adds an optional automated playtest path. The autopilot-playtest skill
runs Autopilot against the open Godot project and writes a report to
production/qa/playtests/. It stays disabled until the user opts in.

- opencode.json: add `mcp.servers.autopilot`, disabled by default.
- qa module: skill, command, modulefile, and installed manifest.
- Mirror the skill and command to .agents/skills and .agents/commands.
- skill-testing-framework/catalog.yaml: register the skill (invariant I10).
- AGENTS.md and reference docs: skill and command counts 77 -> 78; add
  /autopilot-playtest to the QA row.

The skill checks for autopilot_start_run before starting a run. The MCP
tools appear a few seconds after the server starts, so the first model
turn can run before they exist.

Refs: #107
@striderZA
docs(framework): record the autopilot integration decision
dbe21d0 
Records why Autopilot is an optional MCP tool rather than a vendored
addon, the OpenCode V2 behaviour verified in a scratch copy, and the
open items: the live Godot 4.7 run, the bare `autopilot` command on
PATH, the duplicate module trees, and the playtest folder paths.

This file is force-added. `.gitignore` line 17 (`framework/`) matches
docs/framework/ and silently skips new files there. Fixing that rule is
a separate change.

Refs: #107
@striderZA
fix(qa): install autopilot-playtest from the OpenCode module tree
71dabb1 
The OpenCode installer reads .opencode/modules by default, but the
autopilot skill and command were added only to .agents/modules/qa, so
`install.mjs add qa --force` did not restore them. Mirror the skill,
command, modulefile entries, and installed.json entries into the
OpenCode tree. The two qa trees now have identical file lists.

Verified in a scratch copy: after deleting the root copies, `add qa
--force` restores both files, and the restored skill is byte-identical
to the committed copy.

Refs: #114
@striderZA
docs(framework): list /autopilot-playtest in the QA command tables
2e3c56c 
skills-reference.md and quick-start.md list QA commands by name. Add
the autopilot row beside automated-smoke-test. The skills-reference row
states that automated runs never count as human playtest sessions.

Refs: #114
@striderZA
docs(engine-reference): pin the Godot reference to 4.7.2
f7ebf45 
Refresh docs/engine-reference/godot to the 4.7.2 stable patch. Adds the 4.6 to 4.7
migration notes, the 4.7 breaking changes, five 4.7 renamed or removed APIs, and the
4.7 row in the post-cutoff timeline.

Sources: the official 4.7 migration guide and release notes, read on 2026-11-04. This
repository has no Godot code and no project.godot, so no source migration applies.

Refs: #114
@striderZA
feat(setup-engine): offer Autopilot playtesting for Godot 4.7+ on Ope…
9c36731 
…nCode and Pi

Add an optional Autopilot step to /setup-engine: section 7.5 in the module copies and
7.6 in the root copy. It is offered only when the chosen engine is Godot, the Godot
version is 4.7 or later, and the harness is OpenCode or Pi. Other engines, versions
and harnesses skip it, with a reason in the Output Summary.

Nothing is written without a yes. The OpenCode block ships disabled. The Pi block is
shown for the user-level mcp.json, not written, since Pi ignores an untrusted project
config. It sets exposure "direct", because the default codemode exposure does not show
the tools to the model. Keys and the dock's sharing setting stay with the user.

Refs: #114
@striderZA
feat(qa): harness-neutral tool names in autopilot-playtest
09bf757 
The skill now states one naming rule: a bare name such as start_run is the Autopilot
tool of that name, shown as autopilot_start_run in OpenCode and mcp__autopilot__start_run
in Pi. The readiness fallback uses read, which both harnesses offer. Pi does not offer
glob, so the old fallback would fail there.

Phase 1 adds the Pi preflight (enabled true, exposure direct). Phase 3 names the
read and search tools per harness. Phase 4 no longer limits the frame check to OpenCode.
The skill also states the Godot 4.7+ requirement from the addon README.

Checked: a scripted Pi 1.x run offered 14 mcp__autopilot__ tools under direct exposure,
and the model's start_run call reached the real server. Not checked: a real model
following this skill text.

Refs: #114
@striderZA
docs(framework): document Autopilot setup for OpenCode and Pi
8cb359d 
Record the verified scope: Godot 4.7+ with OpenCode 2.2.5 or Pi 1.x. Other harnesses are
not verified. Adds quick-start step 4c and a setup-requirements section, both pointing to
the /setup-engine offer and the playtest skill.

Known gap 4 now records the Godot 4.7.2 pin and that the addon has no runtime version
check. Known gap 7 records the Pi scripted-model result: 14 mcp__autopilot__ tools under
direct exposure, and a start_run call that reached the real server. A real model run
under Pi is not tested.

Design doc: docs/framework/autopilot-integration.md. Refs: #114
@striderZA
feat(omp): add the disabled autopilot MCP entry for oh-my-pi
cc2c85e 
Add .omp/mcp.json with mcpServers.autopilot, disabled. It matches the disabled default
in opencode.json. omp reads this project file natively. A session enables it by setting
enabled to true and restarting omp. The bare autopilot command must be on PATH.

Document the omp wiring in docs/omp-compatibility.md. Verified with omp's default agent
directory and model: enabled true reached the real Autopilot server, and enabled false
offered no Autopilot tool. Runs with an isolated PI_CODING_AGENT_DIR connected the server
but offered no MCP tools, so they are not used as evidence.

Refs: #114
@striderZA
feat(qa): name omp's Autopilot tools in autopilot-playtest
ddc5e99 
The naming rule now covers omp: a bare name such as start_run is mcp__autopilot_start_run
in omp. Preflight checks enabled in .omp/mcp.json for omp, and the readiness step names
the omp tool and /mcp list. Phase 3 names omp's read, glob and grep built-ins, which omp
offers.

Checked: with omp's default model, mcp__autopilot_start_run reached the real server.
Not checked: a full playtest, which needs a game open in the editor.

Refs: #114
@striderZA
feat(setup-engine): offer Autopilot wiring for oh-my-pi
a70b272 
Add omp to the optional Autopilot step in /setup-engine: 7.5 in the module copies and 7.6
in the root copy. The harness question offers OpenCode, Pi, omp or Other. For omp, the
step checks .omp/mcp.json for mcpServers.autopilot. If it is missing, it shows the
disabled block and asks before writing it. The heading, the gate, and the Output Summary
line name all three harnesses.

Refs: #114
@striderZA
docs(framework): record omp for the Autopilot integration
a26975c 
Record omp 18.16.0 as a verified harness in the decision record (gap 8 and Files changed),
in setup-requirements, and in quick-start step 4c. The engine-version gate now names
OpenCode, Pi or omp.

Design doc: docs/framework/autopilot-integration.md. Refs: #114
@striderZA
striderZA merged commit 4da8f99 into master Nov 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@striderZA