You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A cancelled parallel_export_parquet to s3:// leaves its workers' multipart uploads open #1722
@jdatcmd Found while porting objstore_sink_write (#1721). Measured there on PG17; the check is red on head, so #1721 leaves it out.
What happens. When a parallel_export_parquet to an s3:// prefix is cancelled, the dispatcher's pexport_cleanup (src/columnar_parallel_export.c) calls TerminateBackgroundWorker on each worker and then pexport_remove_outputs. Each worker gets SIGTERM and exits through ereport(FATAL) ("terminating background worker … due to administrator command"). FATAL calls proc_exit directly and does not longjmp. So the PG_CATCH in columnar_parquet.c that calls PgColumnarSinkAbort (and through it objstore_sink_abort's DELETE ?uploadId=) never runs. The dispatcher's cleanup then DELETEs only the plain keys and _SUCCESS, which does nothing to an upload that is still in progress.
Measured. On the #1721 fixture server (it now lists open uploads), after a cancelled 4-worker export to cx/, 4 multipart uploads were still open 10 s later. On real S3 those parts are billed until a lifecycle rule reaps them.
Possible fixes.
(a) In the worker, register a before_shmem_exit/on_proc_exit callback around the sink that aborts the open upload.
(b) In the dispatcher cleanup, list the open uploads under the prefix (?uploads&prefix=) and abort each one.
(a) is local but runs network I/O in an exit callback. (b) also covers a worker killed with SIGKILL.
Test.#1721's fixture can already serve this check: cancel: the server holds no open upload under the prefix. It should land with the fix, red-first on main.
Activity
fix: a cancelled remote parallel export aborts its workers' uploads (#…
fix: a cancelled remote parallel export aborts its workers' uploads (#…
fix: an object-store wait slice times out, so a silent store cannot h…
fix: a cancelled remote parallel export aborts its workers' uploads (#…
fix: an object-store wait slice times out, so a silent store cannot h…