Skip to content

A second cancel during the parallel flush's error cleanup leaves workers' output segments pinned #1745

Closed

Description

@OffgridwithJD

@jdatcmd Found by the #1741 fix (#1744) and left out of it. This analysis is from the source; I haven't measured it yet.

What. pflush_error_cleanup (src/columnar_write_state.c:2610 on main) runs under PG_ENSURE_ERROR_CLEANUP (:2844), inside the parallel flush's error handling. For each started worker it calls TerminateBackgroundWorker and then WaitForBackgroundWorkerShutdown, which waits on the latch and runs CHECK_FOR_INTERRUPTS. A second cancel during that wait raises out of the cleanup half done:

  • the remaining workers aren't waited for;
  • the second loop, which dsm_unpin_segments each finished worker's output segment, never runs.

Those segments stay pinned until postmaster restart. That's the leak the function's own header comment says it exists to prevent ("a statement cancel inside WaitForBackgroundWorkerShutdown would leak the pinned segments"). The error reported is also the second cancel, not the first.

Test idea: a parallel flush (#445 path) cancelled twice, the second cancel while the dispatcher waits for worker shutdown; slow the workers' exit if needed. Then require no pinned DSM segments left, e.g. through pg_get_shmem_allocations / the DSM registry count before vs after, or the dispatcher's own leak check. It should be red on main.

Fix direction: the same shape as #1744. Either run the unpin loop regardless of how the wait ends (unpin first, or a nested PG_TRY whose catch still unpins), or copy the handled error out and re-throw it after a cleanup that can't lose it. HOLD_INTERRUPTS around the wait is reasonable here, unlike for the store, because worker shutdown after SIGTERM is bounded.

🤖 Generated with Claude Code

Activity

  1. OffgridwithJD commented on Oct 22, 2026

    @OffgridwithJD
    CollaboratorAuthor

    @jdatcmd Measured and fixed in #1746. Two corrections to what I wrote above:

    1. Reproduced. I counted segment files in the cluster's pg_dynshmem on PG17 main. A single cancel (2 s statement_timeout) leaves 0. Adding a pg_cancel_backend at 3.5 s, while a worker is still encoding a zstd-19 column, leaves 4, and the error reported is the second cancel's.
    2. Wrong in my issue: "worker shutdown after SIGTERM is bounded" was too strong. The encode path has no interrupt check, so a terminated worker finishes its current column first (about 15 s in that run). A single cancel already waits that long, so HOLD_INTERRUPTS around the terminate-and-wait adds no latency. It only stops a second cancel from cutting the cleanup short.
    3. Also left out of fix: a second cancel during the parallel flush's error cleanup frees what the workers pinned (#1745) #1746: on the normal (no-cancel) path, a worker that pinned its output and then failed is never unpinned. That comes from reading the code, not a measurement yet; I'll open an issue once it's measured.

    🤖 Generated with Claude Code

  2. added 2 commits that reference this issue on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions