Skip to content

A parallel flush worker that fails after pinning its output leaks the segment on the normal path #1747

Closed

Description

@OffgridwithJD

@jdatcmd Found while fixing #1745 (#1746): a parallel flush worker that pins its output and then fails leaks that segment on the normal path, until postmaster restart.

What. pgcolumnar_parallel_flush_worker (src/columnar_write_state.c) pins its OUTPUT segment, then runs table_close, PopActiveSnapshot and CommitTransactionCommand, and only then writes PFLUSH_DONE. An ERROR anywhere after the pin reaches the worker's PG_CATCH, which writes PFLUSH_FAILED. The dispatcher's collect loop attaches and unpins only DONE slots; for a FAILED one it logs the WARNING and completes the columns serially, so nothing ever unpins the segment. pflush_error_cleanup doesn't run, because the dispatcher raised nothing.

Measured on PG17, main 9be30954, with a measurement-only ereport(ERROR) right after dsm_pin_segment, gated by an environment variable given only to the test postmaster and NOT committed anywhere. A 300000-row, 8-column parallel flush (two stripes, 4 workers each):

WARNINGs parallel_flush worker N failed, completing its columns serially for N = 0..3, twice
rows 300000, correct (serial fall-back)
pg_dynshmem 2 → 10: 8 segments left pinned

Reachability. In the worker, CommitTransactionCommand has nothing to flush: pgcolumnar's own pre-commit callbacks find no pending writes. So the window raises only through an out-of-memory error or another preloaded library's transaction callback. That's rare, but each occurrence keeps a segment pinned until restart, and nothing reports it.

Fix, stacked on #1746 (same unpin code):

  1. Nothing that can raise after the pin. The worker commits first, keeping its record in TopMemoryContext, then records the handle (as fix: a second cancel during the parallel flush's error cleanup frees what the workers pinned (#1745) #1746 does), pins, detaches and writes DONE.
  2. Belt and braces. The collect loop also frees any slot it didn't collect that holds a valid handle.

The test is a structure check in both harnesses: in the worker, nothing but dsm_detach sits between dsm_pin_segment and the DONE write. It's red on main and runs in CI. The local injection measurement goes in the PR body.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions