Repository navigation
feat(file_storage): direct, resumable multipart uploads (#454) - #483
Merged
Merged
Conversation
Backend capability (create/presign/list/complete/abort/head) implemented for
S3 (presigned upload_part straight to the bucket, s3_public_endpoint_url for
the browser-facing host, CSP connect-src provider) and the filesystem
(parts relayed through PUT /uploads/{id}/parts/{n}, concatenated on
complete). Sessions are owner-scoped and tenant-scoped; the backend's part
listing is the source of truth for resume. checksum_sha256 becomes nullable;
a compute_checksum task and an hourly reaper ship in tasks.py, loaded only
when background_tasks is installed.
Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…hardening (#454) Browser: files above direct_upload_threshold_bytes upload via sessions, 4 parts at a time with per-part retry/backoff, localStorage resume keyed by name+size+lastModified and a Resume upload / Start over prompt. Hardening from security review: open sessions reserve their declared size against the quota; complete re-checks max_direct_upload_bytes and the quota against the stored object's size (head), deleting it on violation; S3 part URLs sign ContentLength; part numbers are limited to 1..total_parts. Docs and changelog (checksum_sha256 is now nullable). Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…tant, share upload error builders Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
Lock the session row in complete/abort, commit rejected completes, scope the S3 orphan sweep to this module's keys and untracked uploads, serialise quota reservations on Postgres, tolerate a deleted object in compute_checksum, write FS part etags atomically. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
… follow-up) Purge finished upload sessions past upload_session_retention_days, release a session whose create was cancelled in flight, persist the abort of an expired session, and cover the Postgres quota/complete locks with concurrency tests. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
Deploying simple-module-python with
|
| Latest commit: |
7515d2f
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://9dd07d8c.simple-module-python.pages.dev |
| Branch Preview URL: | https://feat-multipart-uploads-454.simple-module-python.pages.dev |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…s-454 # Conflicts: # CHANGELOG.md
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the #447-#459 enhancement batch (the design spec lands with #481).
Changes
DirectUploadBackendcapability (separate protocol, implemented by the S3 and filesystem backends) so large uploads go in parts instead of one request.ContentLength,s3_public_endpoint_urlfor browser-facing endpoints, and a CSPconnect-srcprovider so the browser can PUT straight to the bucket.PUT .../parts/{n}), assembled with a temp file and atomic rename.upload_sessiontable + Alembic migration; endpoints to create a session, get part URLs, PUT a part, complete, abort and resume.created=False;FileUploadedis published once.compute_checksumtask whenbackground_tasksis installed.Behaviour changes
checksum_sha256is now nullable (computed asynchronously after a multipart upload).quota_bytesis now enforced for the session path.ETag) for the app origin.Security review
FileUploaded). Fixed withFOR UPDATE+ status re-check.compute_checksumretried forever on a deleted object. Fixed.Known limitations
Spec deviations
DirectUploadBackendprotocol rather than extending the base backend.Verification
make lint: pass.make test-js: 73 files, 570 tests passed.make test-py: 4135 passed, 14 skipped, 1 xfailed.pytest -p no:anyio modules/file_storage274 passed, 1 skipped (the opt-in real-S3 test).make migrations-roundtrip-pg: upgrade, downgrade base, upgrade,alembic checkclean.Content-Disposition(pre-existing, saves under the storage key).Closes #454
https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4