Add: AI-assisted payload generation to the Ability Explorer - Test Ability Screen

[Arkenon]

What?

Adds AI-assisted payload generation to the Ability Test Runner. Users can describe a test scenario in natural language and automatically generate a valid JSON payload based on the selected ability's input schema.

Why?

Some abilities expose complex or deeply nested input schemas that are difficult to populate manually. Creating valid test payloads often requires understanding schema structures, enum values, nested objects, and validation constraints.

This makes testing abilities slower and more error-prone, especially for contributors who are unfamiliar with a specific ability's schema.

How?

ability_test_generate_with_ai.mp4

This PR introduces a new Generate with AI workflow in the Ability Test Runner:

• Added a Generate with AI button to the test interface.
• Added a modal that accepts a natural-language description of the desired test input.
• Sends the ability's input schema and the user's description to the configured AI provider using wp_ai_client_prompt().
• Parses the AI response, removes markdown code fencing if present, validates the JSON, and inserts the generated payload into the test input field.
• The feature is only available when:
  • An AI provider is configured.
  • The selected ability exposes an input schema.

Implementation Details

• Admin_Page.php
  • Added ajax_generate_payload() AJAX handler.
  • Retrieves the ability schema and user prompt.
  • Calls the configured AI provider via wp_ai_client_prompt().
  • Sanitizes and parses the response before returning JSON.
• Abilities_Explorer.php
  • Added generateNonce (ai_ability_explorer_generate_payload) for AJAX authentication.
• GeneratePayloadModal.jsx
  • New modal component for collecting user instructions and displaying errors.
• index.js
  • Added modal mounting/unmounting logic and integration with the Ability Test Runner UI.
• index.scss
  • Added styling for the modal footer and error notice area.

Use of AI Tools

AI assistance: Yes

Tool(s): ChatGPT

Model(s): GPT-5.5

Used for: Brainstorming the feature design, refining prompts, and drafting portions of documentation. Final implementation, review, testing, and code decisions were completed by me.

Testing Instructions

1. Navigate to Tools → Abilities Explorer.
2. Open an ability that exposes a non-trivial input schema.
3. Click Test Ability.
4. Click Generate with AI.
5. Enter a natural-language instruction such as:
   • "Use the default values"
   • "Search for posts with the title 'Hello'"
   • "Query only the site URL information"
6. Click Generate.
7. Verify that the payload textarea is populated with valid JSON matching the schema.
8. Click Invoke Ability.
9. Verify that the ability executes successfully with the generated payload.

Note: An AI provider must be connected in the plugin settings for this feature to be available.

Screenshots or screencast

Before | After -- | -- N/A | Generate with AI button and payload generation modal

Changelog Entry

Added - AI-assisted payload generation for the Ability Test Runner.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: Arkenon <arkenon@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.