feat(sdk): expose the AI pre-write validators as PipefyClient methods - #701
Conversation
Add PipefyClient.validate_ai_agent_behaviors and PipefyClient.validate_ai_automation_prompt. Both delegate to the pipefy_sdk.ai_preflight functions and use the MCP tool names and parameters, so an agent that builds its tools from PipefyClient can validate before it writes. The MCP tools and CLI commands now call the new methods. Move the PipefyClient imports in ai_preflight and ai_phase_transition_validation under TYPE_CHECKING, so client.py can import ai_preflight without an import cycle. Closes #694 Signed-off-by: mocha06 <52426811+mocha06@users.noreply.github.com>
adriannoes
left a comment
There was a problem hiding this comment.
Reviewed at 5c22ee7d: local gates, the targeted contract tests, and a read-only live pass of both validators from a worktree server and the worktree CLI.
Verdict: merge with notes into dev.
Nothing here should hold the merge. The client methods delegate to the existing validators, MCP and CLI call those methods, and the live envelopes matched.
Decisions I would make
test_agent_validate_behaviors_jsonstill asserts onlysuccesswhile that fixture isvalid: false. The weak assert predates this PR. I would not block on it. The MCP twin already requiresvalidtrue.- PRs 657 and 687 also edit
CHANGELOG.md. PR 657 also editsclient.pyanddocs/sdk/README.md. I would rebase after whichever lands first and keep both Unreleased bullets.
Optional
Your call on all of these.
- The new facade tests do not lock the two pipe-read failure dicts. Production already returns them. I would add one agent case (
successfalse, reason inproblems) and one prompt case (onlysuccess,valid, anderror). - One smaller pointer is on its line in the diff.
What worked well
- The facade only delegates, and moving
PipefyClientunderTYPE_CHECKINGbreaks the import cycle. The module functions stay, which is what the issue requires. - The same inputs on MCP and CLI matched for a valid behavior, an unknown field, a missing knowledge base, a valid prompt, a bad event, and a bad field id.
Review path
Tier standard, trust trusted, reviewed at adf67c3a...5c22ee7d.
| Gate | Result |
|---|---|
CI on dev |
green (ci.yml, dco.yml) |
| Local ruff + pytest sdk/mcp/cli | 3785 passed |
| Targeted contracts | 190 passed |
| Live smoke | pass, PipeClaw pipe 306996636, sibling stdio, no mutations |
| Judge | merge with notes, APPROVE |
Not covered: live agent create and ai-automation create. No mutation consent. Unit tests cover the preflight gate. Skill example bugs stay with issues 696 through 700.
…_prompt Answers the review comment on client.py: the docstring listed only the success keys. Signed-off-by: mocha06 <52426811+mocha06@users.noreply.github.com>
Answers the review note that the facade tests did not cover the two failure dicts. Signed-off-by: mocha06 <52426811+mocha06@users.noreply.github.com>
|
adriannoes
left a comment
There was a problem hiding this comment.
Since 5c22ee7d, two commits landed. 401b3cb9 names the pipe-read failure shape on validate_ai_automation_prompt. d4aeab77 locks both failure dicts in the facade tests. The earlier approval was dismissed when this head moved.
I re-ran packages/sdk/tests/test_ai_preflight_facade.py at d4aeab77: 6 passed. Inverting the two new assertions in a scratch copy failed those two tests, so they lock the live dicts. The docstring now names {success, valid, problems, warnings, field_map} and says a failed pipe read returns only {success, valid, error}, with success false. GitHub CI on this head is green. The old CLI test that asserts only success was not touched.
Verdict: merge with notes into dev. Nothing from the last review is still open.
Thanks, @mocha06, for naming that pipe-read failure shape and for locking both failure dicts in the facade tests.
Summary
PipefyClientmethods cannot validate before it writes. The two validators exist only asvalidate_ai_agent_behaviors_sdkandvalidate_ai_automation_prompt_sdkinpipefy_sdk.ai_preflight, but thepipefy-ai-agentsandpipefy-automationsskills tell the agent to validate first.PipefyClient.validate_ai_agent_behaviors(pipe_id, behaviors, *, strict_unknown_action_types=True, data_source_ids=None)andPipefyClient.validate_ai_automation_prompt(pipe_id, prompt, field_ids, event_id=None)delegate to those functions. The names and parameters match the MCP tools. Both methods are read-only. The module functions stay, so this change only adds methods.validate_ai_agent_behaviorsandvalidate_ai_automation_promptnow call the client methods. The CLI commandsagent create|update|validate-behaviorsandai-automation create|update|validate-promptcall them too, so every surface reaches the validators the same way. The CLI part goes beyond the issue text.client.pynow importsai_preflight. To prevent an import cycle,ai_preflight.pyandai_phase_transition_validation.pyimportPipefyClientonly underTYPE_CHECKING, asautomation_preflight.pyandai_pipe_validation.pyalready do.Contract. The change adds no GraphQL operation and no downstream call. The API calls, the auth they need, and the error text that reaches users stay the same.
Behavior boundaries. The result shape is unchanged.
validis true only whenproblemsis empty. If the pipe read fails, the agent result hassuccess: falsewith the reason inproblems. The prompt result then has onlysuccess,valid, anderror.docs/sdk/README.mddocuments both shapes.Test plan
uv run pytest -m "not integration": 4786 passed, 5 skipped, with everyPIPEFY_*variable unsetuv run ruff check . && uv run ruff format --check .cd packages/mcp && uv run lint-imports: 2 contracts kept.scripts/bump_version.py verify, the skill-ref linter, and the skill-frontmatter check pass.validate-*commands pass a realPipefyClientto the new methods. Unit tests cover the MCP tools through the same methods.Live check on a test pipe. Rows 1–6 compare the branch CLI with the MCP server that runs
main, given the same inputs.update_cardon a start-form fieldvalid: true, no problems, no warningsvalid: true,[],[]999999999valid: false, 1 problem that names the fieldvalid: false,fieldId "999999999" not found in pipe fields.data_source_ids=["kb-missing-694"]valid: true, 1 membership warningvalid: true,'kb-missing-694' is not a knowledge base of pipe …event_id=card_createdvalid: true,field_mapwith 2 entriesvalid: true, 2 entriesvalid: false, overlap problemvalid: false, "used both as a prompt input and as an output field"event_id=not_an_eventvalid: false, problem that lists the valid eventsvalid: false, 10 valid events listedsuccess: false, reason inproblems, no exceptionsuccess: false,problems: ["Failed to fetch pipe 1: Acesso negado"]success: false, onlysuccess,valid,error{"success": false, "valid": false, "error": "Failed to fetch pipe 1: Acesso negado"}Docs / skills
docs/parity.mdupdated when MCP ↔ CLI coverage changedskills/updated in this PR (or a paired PR)docs/sdk/README.mdgains a "Pre-write validation" section, andCHANGELOG.mdgains an Unreleased "Added" entry.docs/parity.mddoes not change, because no MCP tool or CLI command is added or renamed. The skills do not change: they already use the tool names that the client methods now carry.Legal / contributions
git commit -s)COMPLIANCE.mdwhen applicableCloses #694