Repository navigation
Conversation
A plugin can now run code before Core sends a workflow task completion and after the server answers. The hooks also run for the completions with no lang activation.
A hung after_complete could stall its run, and a task that waited on a busy run got a deadline past the server timeout.
1 of 3 tasks
Owner
Author
|
Closing as stale. This layer is folded into the v5 tip, #209, and the upstream review uses one fresh PR instead. Plan: https://www.notion.so/4048fc567738814990b7c796ac28fe3f |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
This PR adds a public
WorkflowTaskInterceptortrait and aWorkerExtensionfield onWorkerConfigthat carries it.Core awaits
before_complete(info)before everyRespondWorkflowTaskCompletedit sends for a run. That includes the local activity heartbeat completion and the completion of output buffered before an eviction, which have no lang activation. The hook site is inhandle_activation_completed_success, beforecomplete_workflow_task, which is the only production call of that RPC. An error, or a hook that missesinfo.deadline, fails the workflow task and evicts the run. Core reports the failure only for a first attempt, as it does today.after_complete(info, accepted)runs after the answer.acceptedis false on an RPC error and when the answer hasreset_history_event_id > 0. Core waits at most 5 s on it.on_task_failedruns when Core fails a task, andon_evictedruns when the run leaves the cache, afterafter_complete.The info has the run,
started_event_id, attempt,started_time(poll field 13, now kept onValidPollWFTQResponse),closes_runand a deadline at 90% of the workflow task timeout, measured from poll receipt. The hooks of a run block that run's next workflow task. The heartbeat completion runs inside the activation poll, so while its hooks run, that poll gives no activation to other runs. The replay worker clears the extension, so it never calls the hooks.Why?
Core has no hook before it sends a workflow task completion. So a plugin cannot write data for a task before the server sees it, or act once the server accepts it. The trait is generic and Core knows nothing about what the plugin writes.
How did you test it?
Link to a test plan if any -
Mock worker tests in
core_tests/workflow_task_interceptor.rslog the hooks and each completion the mock server gets, and check their order on each path: a normal completion, a hook error, a missed deadline, anUnhandledCommandrefusal, a dropped speculative task, the local activity heartbeat, an eviction with output, a hungafter_complete, and the replay worker.