Skip to content

[api][plan][python] Add Python CEL Action condition filtering - #937

Closed
rosemarYuan wants to merge 6 commits into
apache:mainfrom
rosemarYuan:pr3/python-cel-implementation
Closed

[api][plan][python] Add Python CEL Action condition filtering #937
rosemarYuan wants to merge 6 commits into
apache:mainfrom
rosemarYuan:pr3/python-cel-implementation

Conversation

@rosemarYuan

Copy link
Copy Markdown
Contributor

Linked issue: #754

Stacked PR — this PR is based on #821. Please review the incremental diff, not the full diff against main.

Purpose of change

This is the third PR in the four-PR stack tracked by #754:

  1. [api][java][python] Introduce EventType constants and unify Action trigger entry #756 - API changes (under review)
  2. [runtime][java][python] Add CEL Action condition filtering #821 - Java CEL runtime
  3. This PR — Python CEL
  4. Documentation (planned)

It adds trigger_conditions to the Python @action decorator, Agent.add_action, and YAML. Python keeps the conditions in their original order as raw strings in AgentPlan JSON.

This implementation does not add a Python CEL library. AgentPlan.from_agent() checks only that each Action has a non-empty list of non-blank strings. For a remote job, apply() sends the full Plan JSON to AgentPlanPreflight through the existing PyFlink JVM. Java then checks the CEL rules before Python builds the output DataStream. An invalid Plan raises ValueError, while a JVM or bridge failure raises RuntimeError.

Java remains responsible for CEL validation, compilation, and Action matching. Python collects the raw conditions and runs only the Python Actions selected by Java.

Tests

  • Python API, Plan, and remote apply() tests: They cover @action, Agent.add_action, YAML aliases, raw condition order, basic data checks, Plan JSON, calls to Java preflight, user errors, bridge errors, and retry after a failed apply().
  • Java Plan tests: They cover valid Plans, invalid CEL with Action and condition details, invalid Plan shape, JSON deserialization, and Python/Java Plan snapshots.
  • Cross-language tests: They cover real Java preflight from Python, a Python Agent running a Java Action, and a Java Agent running a Python Action. The same Action runs only once when two conditions both match.

API

  • Python Action declarations and YAML now support ordered trigger_conditions.

  • Remote apply() can raise ValueError when the Plan is invalid, before the Flink job is build.

Notes

We also considered two ways to validate CEL fully in Python, but neither is a good fit for this PR.

Option Why not selected
Use the community cel-python library with Lark The Lark tree contains only the parsed structure, not the full CEL rules. We would need to write and maintain every CEL validation rule ourselves, which could get out of sync with Java.
Use the official cel-expr-python library backed by CEL C++ It does not support Python 3.10 yet, our question about support has not received a reply, and some APIs needed by this project are not exposed.

Calling Java from Python Plan validation adds a JVM dependency for detailed CEL checks, but this is acceptable because supported Python jobs already run through PyFlink and use the same JVM.

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant