Skip to content

feat(sdk): expose the AI pre-write validators as PipefyClient methods - #701

Merged
mocha06 merged 3 commits into
devfrom
rc-dev/feat/694-sdk-validate-client-methods
Sep 23, 2026
Merged

mocha06 merged 3 commits into
devfrom
rc-dev/feat/694-sdk-validate-client-methods

Conversation

@mocha06

@mocha06 mocha06 commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • An agent that builds its tool list from PipefyClient methods cannot validate before it writes. The two validators exist only as validate_ai_agent_behaviors_sdk and validate_ai_automation_prompt_sdk in pipefy_sdk.ai_preflight, but the pipefy-ai-agents and pipefy-automations skills tell the agent to validate first.
  • PipefyClient.validate_ai_agent_behaviors(pipe_id, behaviors, *, strict_unknown_action_types=True, data_source_ids=None) and PipefyClient.validate_ai_automation_prompt(pipe_id, prompt, field_ids, event_id=None) delegate to those functions. The names and parameters match the MCP tools. Both methods are read-only. The module functions stay, so this change only adds methods.
  • The MCP tools validate_ai_agent_behaviors and validate_ai_automation_prompt now call the client methods. The CLI commands agent create|update|validate-behaviors and ai-automation create|update|validate-prompt call them too, so every surface reaches the validators the same way. The CLI part goes beyond the issue text.
  • client.py now imports ai_preflight. To prevent an import cycle, ai_preflight.py and ai_phase_transition_validation.py import PipefyClient only under TYPE_CHECKING, as automation_preflight.py and ai_pipe_validation.py already do.

Contract. The change adds no GraphQL operation and no downstream call. The API calls, the auth they need, and the error text that reaches users stay the same.

Behavior boundaries. The result shape is unchanged. valid is true only when problems is empty. If the pipe read fails, the agent result has success: false with the reason in problems. The prompt result then has only success, valid, and error. docs/sdk/README.md documents both shapes.

Test plan

  • uv run pytest -m "not integration": 4786 passed, 5 skipped, with every PIPEFY_* variable unset
  • uv run ruff check . && uv run ruff format --check .
  • cd packages/mcp && uv run lint-imports: 2 contracts kept. scripts/bump_version.py verify, the skill-ref linter, and the skill-frontmatter check pass.
  • Manual smoke via Cursor MCP: not run. The live check below uses the branch's CLI, whose validate-* commands pass a real PipefyClient to the new methods. Unit tests cover the MCP tools through the same methods.

Live check on a test pipe. Rows 1–6 compare the branch CLI with the MCP server that runs main, given the same inputs.

# Scenario Expected Observed Verdict
1 Agent: valid update_card on a start-form field valid: true, no problems, no warnings valid: true, [], [] Pass
2 Agent: unknown field 999999999 valid: false, 1 problem that names the field valid: false, fieldId "999999999" not found in pipe fields. Pass
3 Agent: data_source_ids=["kb-missing-694"] valid: true, 1 membership warning valid: true, 'kb-missing-694' is not a knowledge base of pipe … Pass
4 Prompt: valid prompt, event_id=card_created valid: true, field_map with 2 entries valid: true, 2 entries Pass
5 Prompt: the same field as input and output valid: false, overlap problem valid: false, "used both as a prompt input and as an output field" Pass
6 Prompt: event_id=not_an_event valid: false, problem that lists the valid events valid: false, 10 valid events listed Pass
7 Agent: a pipe the user cannot read success: false, reason in problems, no exception success: false, problems: ["Failed to fetch pipe 1: Acesso negado"] Pass
8 Prompt: a pipe the user cannot read success: false, only success, valid, error {"success": false, "valid": false, "error": "Failed to fetch pipe 1: Acesso negado"} Pass

Docs / skills

  • docs/parity.md updated when MCP ↔ CLI coverage changed
  • Affected skills/ updated in this PR (or a paired PR)
  • No docs/skills update needed

docs/sdk/README.md gains a "Pre-write validation" section, and CHANGELOG.md gains an Unreleased "Added" entry. docs/parity.md does not change, because no MCP tool or CLI command is added or renamed. The skills do not change: they already use the tool names that the client methods now carry.

Legal / contributions

  • Commits include DCO sign-off (git commit -s)
  • Regulated-domain skills include COMPLIANCE.md when applicable

Closes #694

Add PipefyClient.validate_ai_agent_behaviors and
PipefyClient.validate_ai_automation_prompt. Both delegate to the
pipefy_sdk.ai_preflight functions and use the MCP tool names and
parameters, so an agent that builds its tools from PipefyClient can
validate before it writes. The MCP tools and CLI commands now call the
new methods.

Move the PipefyClient imports in ai_preflight and
ai_phase_transition_validation under TYPE_CHECKING, so client.py can
import ai_preflight without an import cycle.

Closes #694

Signed-off-by: mocha06 <52426811+mocha06@users.noreply.github.com>
@mocha06 mocha06 added the enhancement New capability or intentional behavior change (not a bugfix) label Sep 23, 2026
@mocha06
mocha06 requested a review from adriannoes September 23, 2026 00:45
adriannoes
adriannoes previously approved these changes Sep 23, 2026

@adriannoes adriannoes left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 5c22ee7d: local gates, the targeted contract tests, and a read-only live pass of both validators from a worktree server and the worktree CLI.

Verdict: merge with notes into dev.

Nothing here should hold the merge. The client methods delegate to the existing validators, MCP and CLI call those methods, and the live envelopes matched.

Decisions I would make

  • test_agent_validate_behaviors_json still asserts only success while that fixture is valid: false. The weak assert predates this PR. I would not block on it. The MCP twin already requires valid true.
  • PRs 657 and 687 also edit CHANGELOG.md. PR 657 also edits client.py and docs/sdk/README.md. I would rebase after whichever lands first and keep both Unreleased bullets.

Optional

Your call on all of these.

  • The new facade tests do not lock the two pipe-read failure dicts. Production already returns them. I would add one agent case (success false, reason in problems) and one prompt case (only success, valid, and error).
  • One smaller pointer is on its line in the diff.

What worked well

  • The facade only delegates, and moving PipefyClient under TYPE_CHECKING breaks the import cycle. The module functions stay, which is what the issue requires.
  • The same inputs on MCP and CLI matched for a valid behavior, an unknown field, a missing knowledge base, a valid prompt, a bad event, and a bad field id.
Review path

Tier standard, trust trusted, reviewed at adf67c3a...5c22ee7d.

Gate Result
CI on dev green (ci.yml, dco.yml)
Local ruff + pytest sdk/mcp/cli 3785 passed
Targeted contracts 190 passed
Live smoke pass, PipeClaw pipe 306996636, sibling stdio, no mutations
Judge merge with notes, APPROVE

Not covered: live agent create and ai-automation create. No mutation consent. Unit tests cover the preflight gate. Skill example bugs stay with issues 696 through 700.

Comment thread packages/sdk/src/pipefy_sdk/client.py Outdated
…_prompt

Answers the review comment on client.py: the docstring listed only the success keys.

Signed-off-by: mocha06 <52426811+mocha06@users.noreply.github.com>
Answers the review note that the facade tests did not cover the two failure dicts.

Signed-off-by: mocha06 <52426811+mocha06@users.noreply.github.com>
@mocha06

mocha06 commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator Author

@mocha06
mocha06 requested a review from adriannoes September 23, 2026 01:54

@adriannoes adriannoes left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since 5c22ee7d, two commits landed. 401b3cb9 names the pipe-read failure shape on validate_ai_automation_prompt. d4aeab77 locks both failure dicts in the facade tests. The earlier approval was dismissed when this head moved.

I re-ran packages/sdk/tests/test_ai_preflight_facade.py at d4aeab77: 6 passed. Inverting the two new assertions in a scratch copy failed those two tests, so they lock the live dicts. The docstring now names {success, valid, problems, warnings, field_map} and says a failed pipe read returns only {success, valid, error}, with success false. GitHub CI on this head is green. The old CLI test that asserts only success was not touched.

Verdict: merge with notes into dev. Nothing from the last review is still open.

Thanks, @mocha06, for naming that pipe-read failure shape and for locking both failure dicts in the facade tests.

@mocha06
mocha06 merged commit 4def1e2 into dev Sep 23, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New capability or intentional behavior change (not a bugfix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants