Skip to content

Move AioS3StreamedFile writes onto the multipart write session #1228

Closed

Description

@laughingman7743

Sub-issue of #1211, which records the design decisions. Depends on the session added for S3File.

Use case

AioS3StreamedFile duplicates the part tracking of S3File with its own differences: _discard() only logs an abort failure, _committing guards a queued commit, and __del__ decides from implicit state whether to abort.

Proposed change

Move AioS3StreamedFile onto _S3WriteSession:

  • Writes await the session's readiness with asyncio.wrap_future; a cancelled task cancels only its own readiness future.
  • The commit reserves completion with begin_complete() before run_shielded() queues it, replacing _committing; the abort and __del__ use the session state.
  • The single-request path, abort-failure handling, and state errors match S3File.

Behavior changes are listed in the PR for the 4.2.0 release notes.

Validation plan (if implementing)

  • The existing AioS3StreamedFile tests, including cancellation, loop shutdown, garbage collection, and one-worker pools.
  • Tests that a cancellation while the commit is queued does not abort it, and that failed part futures and bodies are released after the error is consumed.
  • The filesystem integration tests against the CI bucket.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions