Skip to content

feat(workflow-operator): bound Reservoir Sampling's k at 1 - #9563

Open
Matthew-DWM wants to merge 1 commit into
apache:mainfrom
Matthew-DWM:feat/reservoir-sampling-k-minimum
Open

Matthew-DWM wants to merge 1 commit into
apache:mainfrom
Matthew-DWM:feat/reservoir-sampling-k-minimum

Conversation

@Matthew-DWM

Copy link
Copy Markdown

What changes were proposed in this PR?

Reservoir Sampling's sample size k had no lower bound in its schema, so the property editor accepted 0 or a negative number and let the workflow run. The engine refuses a sample size below 1 at the first row, but the refusal pauses the worker, so the run stays Running until it is killed. k now declares minimum: 1, so the editor rejects such a value before the run starts.

Before:  k = 0 typed in  ->  form valid, Run enabled  ->  Reservoir Sampling stays Running
After:   k = 0 typed in  ->  "must be >= 1"            ->  Invalid Workflow, Run disabled

The check runs in the editor only. A value set through the API, an imported workflow or a loop variable still reaches the engine, which refuses it at the first row as before. Inside a loop, k can still hold a loop variable such as $i, since the bound applies only to numbers.

Any related issues, documentation, discussions?

Closes #9506

How was this PR tested?

A new test in ReservoirSamplingOpDescSpec checks k = -1, 0, 1 and 5 against the generated schema, using the same validator as ScanSourceOpDescSpec. It fails on main, where the schema accepts -1, and passes with this change.

The editor's validator (ajv 8.18.0, set up as in ValidationWorkflowService) was also run on the schema before and after:

  • k = 0 and k = -1 go from valid to "must be >= 1"; 1 and 5 stay valid.
  • A new operator with k left empty still shows "required".
sbt "WorkflowOperator/testOnly *ReservoirSampling*"                   # 19 passed
AMBER_TEST_FILTER=skip-integration sbt WorkflowOperator/jacoco         # 2,994 passed
AMBER_TEST_FILTER=integration-only sbt WorkflowOperator/test           # 85 passed
sbt "WorkflowExecutionService/testOnly *SystemMetadataResourceSpec"    # 2 passed
sbt OperatorReferenceGenerator/test                                    # 19 passed

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5.5)

Reservoir Sampling's sample size had no lower bound in its schema, so the
property editor accepted 0 or a negative number and let the run start. The
engine refuses such a sample size at the first row, but the refusal pauses
the worker, so the run stayed Running until it was killed. k now declares
minimum: 1, and the editor shows "Invalid Workflow" instead. A k from the
API, an imported workflow or a loop variable is still refused by the
executor.
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @kz930, @aglinxinyuan
    You can notify them by mentioning @kz930, @aglinxinyuan in a comment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reservoir Sampling's sample size accepts a value below 1

1 participant