[api][plan][python] Add Python CEL Action condition filtering - #937
Closed
rosemarYuan wants to merge 6 commits into
Closed
[api][plan][python] Add Python CEL Action condition filtering #937rosemarYuan wants to merge 6 commits into
rosemarYuan wants to merge 6 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #754
Purpose of change
This is the third PR in the four-PR stack tracked by #754:
It adds
trigger_conditionsto the Python@actiondecorator,Agent.add_action, and YAML. Python keeps the conditions in their original order as raw strings inAgentPlanJSON.This implementation does not add a Python CEL library.
AgentPlan.from_agent()checks only that each Action has a non-empty list of non-blank strings. For a remote job,apply()sends the full Plan JSON toAgentPlanPreflightthrough the existing PyFlink JVM. Java then checks the CEL rules before Python builds the outputDataStream. An invalid Plan raisesValueError, while a JVM or bridge failure raisesRuntimeError.Java remains responsible for CEL validation, compilation, and Action matching. Python collects the raw conditions and runs only the Python Actions selected by Java.
Tests
apply()tests: They cover@action,Agent.add_action, YAML aliases, raw condition order, basic data checks, Plan JSON, calls to Java preflight, user errors, bridge errors, and retry after a failedapply().API
Python Action declarations and YAML now support ordered
trigger_conditions.Remote
apply()can raiseValueErrorwhen the Plan is invalid, before the Flink job is build.Notes
We also considered two ways to validate CEL fully in Python, but neither is a good fit for this PR.
cel-pythonlibrary with Larkcel-expr-pythonlibrary backed by CEL C++Calling Java from Python Plan validation adds a JVM dependency for detailed CEL checks, but this is acceptable because supported Python jobs already run through PyFlink and use the same JVM.
Documentation
doc-neededdoc-not-neededdoc-included