Skip to content

Reduce repeated full-projection cost during workflow task execution #508

Description

@rmcdaniel

Problem

A small embedded workflow that repeats three parallel no-op activities followed by a timer spends increasing time on each wake as history grows. Profiling must distinguish workflow replay from synchronous monitoring projection work; do not bypass history checks or disable monitoring as a fix.

A local reproduction using published Workflow 2.0.12, PHP 8.4.25, an isolated in-memory SQLite database, a one-CPU/1-GB container limit, and mocked activity effects completes three rounds (nine activities, 36 history events). The two middle timer wakes took approximately 1.03 and 1.54 seconds. SQL execution accounted for approximately 87 and 112 milliseconds, but the wakes executed 1,268 and 1,596 queries. These are diagnostic measurements, not production throughput claims.

The query trace shows 1,221 individual timeline-row SELECTs and 432 wait-row SELECTs over the complete short run. They originate in full summary projection calling the timeline/wait projectors, which invoke IdempotentProjectionUpsert::upsert for every historical row. An initial longer diagnostic showed further growth; a corrected harness now reloads the stub before checking terminal state.

Acceptance

  • Retain a small reproducible growing-history workload and measure the dominant PHP and SQL costs before selecting the optimization.
  • Eliminate unnecessary repeated work in the measured hot path without weakening replay identity, durable history authority, projection repair, configured models/connections, concurrent upsert handling, or stale-row cleanup.
  • Add regression tests for the affected projection behavior and bounded query behavior, plus relevant replay/concurrency tests.
  • Compare the same workload before/after with profiling disabled and verify completion and exactly-once recorded activity results.
  • Run normal repository checks and follow through to published artifacts and affected consumers before calling delivery complete.

Scope is the public workflow engine. Deployment-specific details and infrastructure experiments are not part of this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:defectA public product behavior is incorrectpriority:P1High-priority product or release risk

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions