You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A second cancel during the parallel flush's error cleanup leaves workers' output segments pinned #1745
@jdatcmd Found by the #1741 fix (#1744) and left out of it. This analysis is from the source; I haven't measured it yet.
What.pflush_error_cleanup (src/columnar_write_state.c:2610 on main) runs under PG_ENSURE_ERROR_CLEANUP (:2844), inside the parallel flush's error handling. For each started worker it calls TerminateBackgroundWorker and then WaitForBackgroundWorkerShutdown, which waits on the latch and runs CHECK_FOR_INTERRUPTS. A second cancel during that wait raises out of the cleanup half done:
the remaining workers aren't waited for;
the second loop, which dsm_unpin_segments each finished worker's output segment, never runs.
Those segments stay pinned until postmaster restart. That's the leak the function's own header comment says it exists to prevent ("a statement cancel inside WaitForBackgroundWorkerShutdown would leak the pinned segments"). The error reported is also the second cancel, not the first.
Test idea: a parallel flush (#445 path) cancelled twice, the second cancel while the dispatcher waits for worker shutdown; slow the workers' exit if needed. Then require no pinned DSM segments left, e.g. through pg_get_shmem_allocations / the DSM registry count before vs after, or the dispatcher's own leak check. It should be red on main.
Fix direction: the same shape as #1744. Either run the unpin loop regardless of how the wait ends (unpin first, or a nested PG_TRY whose catch still unpins), or copy the handled error out and re-throw it after a cleanup that can't lose it. HOLD_INTERRUPTS around the wait is reasonable here, unlike for the store, because worker shutdown after SIGTERM is bounded.
@jdatcmd Measured and fixed in #1746. Two corrections to what I wrote above:
Reproduced. I counted segment files in the cluster's pg_dynshmem on PG17 main. A single cancel (2 s statement_timeout) leaves 0. Adding a pg_cancel_backend at 3.5 s, while a worker is still encoding a zstd-19 column, leaves 4, and the error reported is the second cancel's.
Wrong in my issue: "worker shutdown after SIGTERM is bounded" was too strong. The encode path has no interrupt check, so a terminated worker finishes its current column first (about 15 s in that run). A single cancel already waits that long, so HOLD_INTERRUPTS around the terminate-and-wait adds no latency. It only stops a second cancel from cutting the cleanup short.
Activity
OffgridwithJD commentedon Oct 22, 2026
@jdatcmd Measured and fixed in #1746. Two corrections to what I wrote above:
pg_dynshmemon PG17 main. A single cancel (2 sstatement_timeout) leaves 0. Adding apg_cancel_backendat 3.5 s, while a worker is still encoding a zstd-19 column, leaves 4, and the error reported is the second cancel's.HOLD_INTERRUPTSaround the terminate-and-wait adds no latency. It only stops a second cancel from cutting the cleanup short.🤖 Generated with Claude Code
fix: a second cancel during the parallel flush's error cleanup frees …
fix: a second cancel during the parallel flush's error cleanup frees …
Merge pull request #1746 from commandprompt/fix/1745-pflush-double-ca…
fix: a cancelled remote parallel export aborts the uploads its worker…