Repository navigation
Description
Problem
AutoGenesis's Appium and Windows stdio MCP runtimes publicly expose confirm_code_changes. The same MCP caller that requests a generated-code preview can supply the preview identity, digest, and approval value, so the current release surface cannot distinguish a trusted user approval from caller-controlled task data. Shipping that interface in a public Preview would make generated-file commits reachable without the trusted host/UI approval capability specified by #51.
User outcome
Coding Agents can use one canonical autogenesis Skill plus the standard stdio MCP runtimes for direct natural-language execution and exact BDD execution. An explicit recording request may generate and inspect a read-only preview, but no Runtime or Agent Contract route reachable through stdio can commit generated files until #51 supplies trusted approval. The workflow reports that writes are unavailable in this Preview rather than suggesting another copy/save route.
Area
Agent Contract
Scope
- Remove
confirm_code_changesfrom live stdio MCP tool registration and tool schemas in both the Appium and Windows runtimes. - Remove caller-controlled generated-file commit reachability from the release surface while retaining
before_gen_codeandpreview_code_changesas read-only operations. - Preserve direct natural-language execution and exact BDD execution.
- Update the canonical
autogenesisSkill and generated Agent Contract workflow so explicit recording stops after preview and reports that generated-file writes are unavailable in this Preview. - Regenerate the Agent Contract identity and update every exact manifest/consumer pin atomically using the repository's existing generation and verification patterns. Keep the intended unreleased version line at Agent Contract 2.1.0, Appium Runtime 0.5.0, Windows Runtime 0.5.0, and CLI 0.2.0 unless public-collision evidence requires a separate product decision.
- Update Appium and Windows tool-registration/schema fixtures and focused contract tests.
- Update technical package documentation to describe the observable Preview contract.
- Remove the obsolete VS Code configurable full-prompt setting, dead prompt manager/default, and absent-Skill fallback. Migrate shipped Scenario/Background routes and GetStarted generation examples to the authenticated canonical one-Skill preview-only flow; Skill absence stops with its install action.
- Return only a project-relative proposed destination (or generation session ID) from both
before_gen_codetools; never expose an absolute business-project path in tool output. - Update both Runtime
server.json--project-rootdescriptions from write scope to preview/proposed-destination scope. - Make rollback remove the Candidate channel and cached Skill/Runtime schemas; rollback must not restore a write-capable cached inventory.
- Obtain independent correctness and security review before release publication.
Non-goals
- Do not implement trusted host/UI approval or generated-file commits; [Security] Bind generated writes to trusted approval and directory handles #51 remains the future trusted-write capability.
- Do not add a placeholder, compatibility shim, hidden alias, alternate save/copy instruction, or second write path.
- Do not create agent-specific Skill variants or compatibility adapters; compatibility remains the generic Agent Skills discovery plus standard stdio MCP seam.
- Do not change direct natural-language task execution or exact BDD Scenario execution semantics.
- Do not add a new VS Code extension feature; shipped prompts/defaults/docs are affected callers that must migrate for the clean cutover.
- Do not qualify an untested client or platform. OMP 18.18.3 on native macOS is the qualified path; Copilot CLI and Codex remain experimental/unqualified; stock Pi is unsupported because it has no MCP; the Windows Skills journey remains unqualified until final-artifact acceptance.
- Do not publish packages/releases, merge pull requests, or modify protected branches.
- Do not edit the root README or PR Publish AutoGenesis 1.1.1 Skills Preview onboarding #29 assets.
Interfaces affected
- Canonical Agent Skill:
.github/skills/autogenesis/SKILL.mdand authenticated generation/execution references. - Generated Agent Contract package identity, workflow assets, manifest, and exact consumer pins.
- Appium stdio MCP live tool inventory and generated-code tool schemas.
- Windows stdio MCP live tool inventory and generated-code tool schemas.
- Runtime/package technical documentation describing recording and generated-code behavior.
- Shipped VS Code Scenario/Background routes, obsolete prompt-setting removal, and GetStarted generation examples.
- Runtime
before_gen_coderesponse schemas andserver.jsonproject-root descriptions. - Candidate/cached-schema rollback instructions.
- Removed public interface:
confirm_code_changes. - Retained read-only interfaces:
before_gen_code,preview_code_changes. - Unchanged execution interfaces: direct natural-language execution and exact BDD execution.
Acceptance scenarios
Scenario: Appium live inventory has no generated-file commit tool
Given the Appium Runtime is launched through its public stdio MCP entrypoint
When a client requests the live tool inventory and schemas
Then `confirm_code_changes` is absent
And `before_gen_code` and `preview_code_changes` remain available as read-only operations
And no registered tool or documented stdio route can commit generated files
Scenario: Windows live inventory has no generated-file commit tool
Given the Windows Runtime is launched through its public stdio MCP entrypoint
When a client requests the live tool inventory and schemas
Then `confirm_code_changes` is absent
And `before_gen_code` and `preview_code_changes` remain available as read-only operations
And no registered tool or documented stdio route can commit generated files
Scenario: Explicit recording stops after read-only preview
Given a Coding Agent follows the canonical `autogenesis` Skill and matching Agent Contract
When the user explicitly requests recording or scenario generation
Then the workflow may create a read-only generated-code preview
And it stops after preview
And it reports that generated-file writes are unavailable in this Preview
And it does not instruct the Agent to copy, save, shell-write, or otherwise bypass the unavailable commit capability
Scenario: Execution behavior remains available
Given a supported target is Live Ready
When the user requests a direct natural-language task or exact BDD Scenario execution
Then the canonical Skill routes it to the existing execution interface
And the direct or exact BDD contract is unchanged by the recording restriction
Scenario: Missing canonical Skill fails closed
Given the VS Code extension cannot load the authenticated canonical `autogenesis` Skill
When a generation or execution prompt is requested
Then it does not inject a legacy full prompt or legacy generation tool names
And it stops with the canonical Skill install action
Scenario: Recording output does not disclose an absolute project path
Given either Runtime starts a recording session for a business project
When `before_gen_code` returns its public response
Then it returns the generation session ID and at most a project-relative proposed destination
And it does not return the absolute business-project root or absolute destination
Scenario: Shipped VS Code generation examples are preview-only
Given the extension defaults, package configuration, and GetStarted examples
When a user follows the generation flow
Then they use the authenticated canonical one-Skill route
And call `before_gen_code` then `preview_code_changes`
And stop after preview without confirmation, copy, save, or shell-write guidance
Scenario: Rollback cannot restore a write-capable cache
Given a Candidate or cached Runtime/Skill schema may advertise generated-file confirmation
When the Preview is rolled back
Then the Candidate and cached Skill/Runtime schemas are removed
And the operator is not instructed to restore the write-capable inventory
Scenario: Contract identity and pins move atomically
Given the canonical Agent Contract sources include the safe Preview workflow
When repository generation tools produce the Contract artifacts
Then the generated identity reflects exactly those sources
And every exact manifest and Runtime/CLI consumer pin matches that identity
And no stale identity pin remains
Scenario: Trusted writes remain deferred
Given a caller has only the public stdio MCP interfaces in this Preview
When it attempts to approve or commit a generated-code preview
Then no confirmation tool, alias, or alternate Runtime/Contract write path is reachable
And trusted generated-file writes remain owned by #51Verification environments
- Linux package or protocol checks
- macOS package or platform checks
- Windows package or platform checks
- Clean temporary business project
- Existing VS Code workflow regression
Required claim boundaries:
- OMP 18.18.3 on native macOS: qualified acceptance environment.
- Copilot CLI and Codex using the same generic Skill + MCP artifacts: experimental/unqualified evidence only.
- Stock Pi: unsupported because it has no MCP.
- Native Windows Skills journey: unqualified until independent final-artifact acceptance.
Security considerations
- Configuration file writes or merges
- File-system scope or path validation
- Prompt or instruction injection
- Secrets or credentials
- Command or process execution
- Downloads or package installation
- Not security-sensitive
An independent reviewer must explicitly verify that an MCP caller cannot reach generated-file commit behavior through registration, schemas, dispatch, aliases, workflow instructions, documentation, or another stdio path. The implementer cannot approve this change. Release publication remains blocked until that review and clean public-artifact verification complete.
Evidence required
- Focused Agent Contract tests proving recording terminates at preview while direct and exact BDD execution wording remains intact.
- Focused Appium and Windows tool-registration/schema tests proving the live inventories omit
confirm_code_changesand retain the two read-only preview tools. - Repository identity/pin verification showing generated Agent Contract identity and every exact consumer pin agree.
- Searches or contract assertions showing no public stdio registration, schema, dispatch alias, workflow instruction, or technical package documentation offers generated-file commit/copy/save behavior.
- Focused VS Code contract coverage proving canonical Skill digest/defaults, absent-Skill install-only failure, preview-only prompt flow, and no legacy confirmation fallback.
- Focused Runtime response-schema coverage proving
before_gen_codenever returns absolute project paths, plus metadata assertions for preview-scopeserver.jsondescriptions. - Rollback documentation review proving Candidate and cached schemas are removed rather than restoring the write-capable inventory.
- Package/build checks for Agent Contract, Appium Runtime, Windows Runtime, and CLI as selected by existing repository validation workflows.
- Clean public-artifact smoke evidence from OMP 18.18.3 on native macOS.
- Independent correctness/security review linked from the pull request.
- Independent Windows final-artifact acceptance before qualifying the Windows Skills journey.
Exact-head evidence
Implemented and independently evaluated at PR head
3f894a7ed8f6d3e4c7e9baa31ab580a0a5d23c64:
- Correctness:
APPROVE; no remaining blocker. - Security:
SECURITY_APPROVE; no remaining blocker. - Contract 2.1.0 identity:
ca6f1970b994f68e78495819147dbab3daf00e3fc1e44f856be8740a00283ed4. - Canonical Skill SHA-256:
99e276bde9c95996d9d19c8f548eac41305cea61494a3955c4c6f44563f88f64. - Independent native-macOS validation evidence: freeze/projection 2 passed; focused Contract 96 passed; clean Contract wheel/MCP 1.29.1 222 passed; clean Appium wheels 76 passed plus
pip check; CLI 33 passed; extension compile, GetStarted build, Registry metadata/descriptors, and live installed-wheel stdio passed. - Exact-head prompt-eval evidence: candidate 7/7 versus baseline 4/7, 0 regressions, independent semantic PASS.
- Live stdio exposed 30 Appium iOS tools with before/preview present and confirmation absent/rejected; initialization returned only
gen_code_id, disclosed no absolute path, wrote zero generated files, and exited 0 on EOF. - Windows remains unqualified: macOS ran 6 applicable tests and recorded 26 explicit native-Windows skips. Windows Server 2022 / Python 3.12 / MCP 1.29.1 live stdio and clean-wheel protocol acceptance remains a qualification blocker; macOS was not substituted.
- GitHub macOS rerun remains pending and is not claimed here.
Dependencies and blockers
- Depends on [Prompt]: Guide installation and environment readiness from user goals #65 for the canonical operational Skill/workflow baseline.
- [Security] Bind generated writes to trusted approval and directory handles #51 remains open and is the only future capability for trusted generated-file writes; this safe Preview must not implement or bypass it.
- Release publication is blocked until this issue's implementation is independently reviewed for correctness/security and clean public-artifact verification succeeds.
File ownership
The implementer owns Agent Contract generation code/workflows/assets and manifest/pins; Appium and Windows generation tool registration/response schemas/fixtures/focused tests and metadata; shipped VS Code prompt fallbacks/defaults/package configuration/GetStarted generation examples and focused contract tests; rollback instructions; and technical package documentation. Root README.md, PR #29 assets, and #71 pipeline/verifier files are excluded and owned elsewhere.
Definition of Ready
- The user outcome is unambiguous.
- Scope and non-goals are explicit.
- Acceptance scenarios are observable and executable.
- Public interface decisions are resolved or explicitly out of scope.
- Dependencies and file ownership are known.
- Verification environments are available.
- Security-sensitive surfaces and the independent review gate are explicit.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
houlianpi commentedon Nov 10, 2026
Safe Preview prompt eval — exact-head evidence
Compared exact baseline
f0e0814cb1e2625e31772eb88c925eced54371fcwith PR heade22d600f74d9d3037f73e21d84555eca5757b49dusing the candidate canonical fixture bytes and comparator rules. Focus: all four changed recording cases plus direct execution, exact-BDD execution/generation separation, and Background/Scenario ordering (7 isolated cases per variant).github-copilot/gpt-5.6-solfor both variants.safe-preview-synthetic-runtime/1.0; identical per-case schemas/limits, no real app/device, credentials, or generated writes.generation-preview-read-only,generation-copy-injection-rejected,legacy-confirm-cache-is-not-authority.generation-preview-unavailablepassed both variants.before_gen_code, executed/verified the supplied step, called successfulpreview_code_changes, then only stopped/reported that generated-file writes are unavailable. No approval request,confirm_code_changes, copy, save, shell, file-write, confirmation, or write event occurred. Missing-preview case made 0 tool calls and stoppedenforcement_unavailable.open_app→verify_app_title; exact-BDD traces preserved supplied step order and never entered generation.3787096245ded0eabffa6a2ead16256c7ca19ba016f2ef0afdd709ab0c84c8b1.Bindings and retained sanitized evidence hashes:
9a42062d68e00455cb8c962aa54a8c6822c127afb5af28b9ebd99c3ddbf07a8c746dc17cbcc3fcecd1782f0a65722479372f02822f1dd8d384654ac28ccb0f14932da2d550a93ef5ae4a9a29880b5851507d7c8e6471292a7815b54cfb64af7c63fafdbe0788c9e47594d917fd7df664c87d2071b2cf5fd75737a4155205fb2dd327f288d11c92c780a7efcb687634efa7a76c8b273a56f64c4ec99fa07fa4aa27418da6a99fe87f9f77d14befd7e7677215fd28b4e4d1065c3861a443d6b307Comparator SHA-256:
b15ba2ac321bd5e18d41524670dceede33f44dafcef3330f8631b2e08a2c90d4; independent grade SHA-256:30965bfd10ee67e11d7bafc2c288749d399ca9e80645019647c8e13a343e26dd.houlianpi commentedon Nov 10, 2026
Superseding Safe Preview prompt eval — final exact head
This supersedes my earlier
e22d600…evidence because PR #72 changed. I freshly executed the same comparison against exact baselinef0e0814cb1e2625e31772eb88c925eced54371fcand final candidate5ee0cddcf7e717d53aa4edbc851d7e3a1480b8f9; no prior candidate traces were reused.The run used the final candidate canonical fixture bytes and comparator rules: all four changed recording cases plus direct execution, exact-BDD execution/generation separation, and Background/Scenario ordering (7 isolated cases per variant).
github-copilot/gpt-5.6-solfor both variants.safe-preview-synthetic-runtime/1.0; identical per-case schemas/limits, no real app/device, credentials, or generated writes.generation-preview-read-only,generation-copy-injection-rejected,legacy-confirm-cache-is-not-authority.generation-preview-unavailablepassed both variants.before_gen_code, executed/verified the supplied step, called successfulpreview_code_changes, then only stopped/reported exactly that generated-file writes are unavailable in this Preview. No approval request,confirm_code_changes, copy, save, shell, file-write, confirmation, or write event occurred. Missing-preview made 0 tool calls and stoppedenforcement_unavailable.open_app→verify_app_title; exact-BDD traces preserved supplied step order and never entered generation.c358a9a751b2aeb577c108a32610c437277463632d29b8bf706b211c1f9287a1.Fresh final-head bindings and retained sanitized evidence hashes:
9a42062d68e00455cb8c962aa54a8c6822c127afb5af28b9ebd99c3ddbf07a8c95ba42c76a22f281c0ba3dd247563d3b658fa04423672324c31c12baff622b157dc1659b698a44b9e32853f57784b33a51959e4709d9bb57f8269481562514589282b8481f28d23a989674e7b51b8ab80b2030210e56ea0509fd02ac4b353c6563a973b7349dc4085f1206e9c32e86cdf81d2bbdf4e0e52c5b58a44599cf2ac312deac0d0a273027619a13f39aa24a6f45975c7059c30b345566beecb619cbb1Comparator SHA-256:
d7c9330ef5322222d2192cc848d2f48bcb8deaddf82c90136d91b4752f87d625; independent grade SHA-256:24d88b79a1757e252ac91dbd3c11dd69e6d3a45e808fdf0224d4080698934c49.houlianpi commentedon Nov 10, 2026
Superseding Safe Preview prompt eval — exact final head
3f894a7This supersedes my earlier
e22d600…and5ee0cdd…comments because PR #72 changed again. I freshly executed the identical comparison against baselinef0e0814cb1e2625e31772eb88c925eced54371fcand candidate3f894a7ed8f6d3e4c7e9baa31ab580a0a5d23c64; no earlier candidate traces were reused.The run used this head's canonical fixture bytes and comparator rules: all four changed recording cases plus direct execution, exact-BDD execution/generation separation, and Background/Scenario ordering (7 isolated cases per variant).
github-copilot/gpt-5.6-solfor both variants.safe-preview-synthetic-runtime/1.0; identical per-case schemas/limits, no real app/device, credentials, or generated writes.generation-preview-read-only,generation-copy-injection-rejected,legacy-confirm-cache-is-not-authority.generation-preview-unavailablepassed both variants.before_gen_code, executed/verified the supplied step, called successfulpreview_code_changes, then only stopped/reported exactly that generated-file writes are unavailable in this Preview. No approval request,confirm_code_changes, copy, save, shell, file-write, confirmation, or write event occurred. Missing-preview made 0 tool calls and stoppedenforcement_unavailable.open_app→verify_app_title; exact-BDD traces preserved supplied step order and never entered generation.ca6f1970b994f68e78495819147dbab3daf00e3fc1e44f856be8740a00283ed4.Fresh exact-head bindings and retained sanitized evidence hashes:
3f894a79a42062d68e00455cb8c962aa54a8c6822c127afb5af28b9ebd99c3ddbf07a8c81e0d0de97d0920633a6c9371726f99eb29eca70d07ff5d04f134805ee2aa060e463ebb4c2a76be3de7f4468330882839942e8fb036ca6cb502e0b1602547a6ee4a9386150f94f8e9e8e41f5492a44edaf35e961f7745e673587660e8e956e2d4c22fe1f18eb8dbc72a0f105abc46ab2029d281ee95f7bb8b9fca508b6c89e88aa44a16f1ef5ccc6eaefddadea76172dfdaf1e7586072ae4c6675790fadd4504Comparator SHA-256:
d43298884ef501736d16482224b1615e2237c669b8909ed420ecb151d91ea911; independent grade SHA-256:f49c9dd7f14f4e9ad277f7ea47ffd2b4b45089a514f91a03bc1b5b7b2079b9be.houlianpi commentedon Nov 10, 2026
Independent final-head verification — native macOS
Verified exact PR head
3f894a7ed8f6d3e4c7e9baa31ab580a0a5d23c64in a detached isolated worktree. No source edits were made; tracked source remained clean after validation.Result
All checks executable on native macOS passed. Prompt evaluation is separately owned and is not claimed here.
Blocker classification:
QUALIFICATION_BLOCKER_WINDOWS_NATIVE. Native Windows live stdio and clean-wheel protocol acceptance cannot run on macOS and was not substituted. Before qualifying the Windows Skills journey or publishing, Windows Server 2022 / Python 3.12 / MCP 1.29.1 must run:Commands and results
pip checkpassed.server.jsondescriptors/descriptions, generated descriptors, and all four wheel identities: passed.Identity, pins, and wheel bytes
Contract identity:
2.1.0, schema2, manifest SHA-256ca6f1970b994f68e78495819147dbab3daf00e3fc1e44f856be8740a00283ed4.Canonical Skill/projection SHA-256:
99e276bde9c95996d9d19c8f548eac41305cea61494a3955c4c6f44563f88f64. Appium Runtime, Windows Runtime, CLI manifest pins all matched the Contract identity; the extension Skill pin matched the exact canonical Skill bytes. The preceding manifest pin was absent.autogenesis_agent_contract-2.1.0-py3-none-any.whlf89a9c237a568ac065c0020f0cf2a5ba896b823882c4c8c48e4bfde6f2a323d1autogenesis_appium_mcp-0.5.0-py3-none-any.whlda87d32f104f82490cfb94297587f8fb0955456a52fe59974d011366ad0e5d99autogenesis_windows_mcp-0.5.0-py3-none-any.whlba8859430b7b990d4ce8bf3c0fab81ec67e6e5608596809b01baaf59beb314f0autogenesis_cli-0.2.0-py3-none-any.whl2378127d27e67ddbd590026ab8465bcb780b615d03c0b29943d8c0184b816979Built-wheel live stdio result
A clean temporary business project launched the installed Appium wheel over public stdio at MCP 1.29.1 with no device session. Live inventory contained 30 iOS tools;
before_gen_codeandpreview_code_changeswere present andconfirm_code_changeswas absent. Calling the removed confirmation name returned an error.before_gen_codereturned exactlygen_code_id; neither it nor preview disclosed the absolute business-project path. The business project contained zero generated files before shutdown, and stdin EOF exited with code 0. The clean verifier additionally exercised live iOS, Android, and macOS inventories (30/29/34 tools) from installed wheels.Sanitized clean locations:
<OS temporary directory>/autogenesis-contract-acceptance-*/business,<OS temporary directory>/appium-business-*/business-project, and<OS temporary directory>/autogenesis-final-live-*/business-project; all were outside the checkout and automatically removed.houlianpi commentedon Nov 10, 2026
Human Product Owner acceptance — macOS hosted-runner capacity exception
Accepted for exact PR head
3f894a7ed8f6d3e4c7e9baa31ab580a0a5d23c64:runner_id=0, empty runner name, zero steps); no product code or check step executed in those jobs.The Product Owner accepts the cancelled macOS hosted jobs as an external capacity exception, not a product failure, and authorizes integration. This does not qualify the Windows Skills journey: Windows remains unqualified until independent final-artifact acceptance. No package/release publication or repository setting change is authorized by this decision.
houlianpi commentedon Nov 10, 2026
Merged through repository convention in PR #72.
3f894a7ed8f6d3e4c7e9baa31ab580a0a5d23c645030304dcaf4a4bbc04b58791345ec35016f5317#69 implementation acceptance is Passed and the Project item is Done. Release publication remains governed by #70.