Skip to content

feat(file_storage): public files, anonymous serving, list search/sort, thumbnails - #410

Merged
antosubash merged 15 commits into
mainfrom
feat/352-353-file-storage-public-and-picker
Oct 7, 2026
Merged

antosubash merged 15 commits into
mainfrom
feat/352-353-file-storage-public-and-picker

Conversation

@antosubash

Copy link
Copy Markdown
Owner

Fixes #353
Fixes #352

Design

Public files (#353). StoredFile.public (default false; migration e5a8c1f27b94 uses sa.false(), Postgres-portable). Set via public=true on upload or PATCH /files/{id} (file_storage.upload, tenant-scoped like every other authenticated route). StoredFileOut gains public and public_url. Anonymous GET /api/file-storage/public/{id}[/{filename}] and /public/{id}/thumbnail are exempted via register_public_routes (GET-only regex). ETag/304, Cache-Control: public, S3 backends redirect to the presigned URL (cached for at most half its TTL).

Search/filter/sort/thumbnails (#352). GET /files takes q (case-insensitive substring, LIKE wildcards escaped), content_type (exact or image/ prefix) and sort (created_at, -created_at default, name, -name, size, -size; invalid is 422; ties break on id). GET /files/{id}/thumbnail?w= (and the public equivalent) returns WebP, aspect preserved, never enlarged, width clamped to 32-1024 and snapped to 64/128/256/512/1024. Variants are cached in the storage backend beside the original ({key}.w{width}.webp): durable, shared across workers, bounded to 5 per file, tenant-prefixed, deleted with the file. Browse screen gets a Public badge and a make public/private action (i18n'd). Docs in docs/modules/file_storage.md.

Security

  • Access control. Public routes serve only public=True, non-deleted rows; unknown, private and deleted are one identical 404 (no existence/metadata leak). all_tenants() is used only on that single by-UUID lookup, which also requires public=True (publishing is the owner's explicit cross-tenant choice). PATCH, authenticated thumbnail and download go through the tenant-filtered get(), so another tenant's id is a 404; thumbnail/download need file_storage.download, PATCH needs file_storage.upload. Anonymous callers cannot reach /files/*.
  • Stored XSS. Every public response carries nosniff and Content-Security-Policy: default-src 'none'; sandbox. HTML/XHTML/SVG/XML/JS are forced to Content-Disposition: attachment and always streamed (never redirected). SecurityHeadersMiddleware now keeps a CSP a response already set instead of overwriting it (framework change; the app-wide CSP stays the default).
  • Pillow attack surface. Image.open(formats=[JPEG, PNG, WEBP, GIF]) only; the sniffed format must match the declared content type (a mislabelled SVG/PSD/BMP is a 422); SVG and every other type is a 404. First frame only; output is re-encoded from pixels, so EXIF/XMP/ICC are dropped.
  • Resource exhaustion. Header-only pixel budget (25 MP) checked before any decode, without touching Pillow's process-global MAX_IMAGE_PIXELS; source capped at 20 MB while streaming (no unbounded buffer); width snapped to a 5-value whitelist; decodes bounded by a per-event-loop semaphore (2). Decode runs as a shielded single-flight task per variant, so a client that disconnects cancels only its wait: the slot is held until the worker thread returns and retries join the in-flight decode instead of multiplying them.
  • Tests cover each of the above (cross-tenant PATCH/thumbnail/download, private-vs-unknown identical responses, format mismatch, allowlist, animated GIF, EXIF stripping, pixel/source caps, cancellation holding the slot, global cap untouched).

Verification

make lint green; make test-py run to completion (3601 passed; the only failures were two test_module_css_build tests, which failed because the suite's own pnpm step clobbered node_modules in the worktree and pass after npm install); make test-js file_storage vitest (44 passed), make ci-check-untranslated OK; migration applies on SQLite and alembic check shows no file_storage drift. Not run: Postgres suite, S3 redirect against a real bucket (covered by unit logic only).

https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4

…sort, thumbnails

Fixes #353, fixes #352.

- StoredFile.public (default false; migration uses sa.false()), set on upload
  (public=true) or PATCH /files/{id}; StoredFileOut gains public/public_url.
- Anonymous GET /public/{id}[/{filename}] and /public/{id}/thumbnail, exempted
  via register_public_routes (GET only). Only public, non-deleted rows resolve;
  other cases are one 404. Cross-tenant lookup by id uses all_tenants().
  ETag/304, Cache-Control public, nosniff, sandbox CSP; active content (HTML,
  SVG, JS) forced to attachment and streamed.
- SecurityHeadersMiddleware keeps a CSP the response already set.
- GET /files: q, content_type (exact or "image/" prefix), sort.
- GET /files/{id}/thumbnail?w=: Pillow WebP, width snapped to a whitelist,
  pixel-budget guard, variants cached in the storage backend and dropped with
  the file.
- Browse screen: Public badge and make public/private action.

Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…ps, concurrency) and add tenant/permission tests

Pillow only opens JPEG/PNG/WEBP/GIF, sniffed format must match the declared
type, DecompressionBombWarning is an error, source bytes are capped while
reading, decodes are bounded by a semaphore, first frame only, metadata not
carried over.

Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…hes; stop mutating Pillow's global pixel cap

Decode runs as a shielded single-flight task per variant, so a cancelled
request neither frees its slot early nor lets retries multiply decodes. The
semaphore is per event loop. The explicit header-only pixel budget replaces the
racy process-global MAX_IMAGE_PIXELS toggle.

Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ✅ Completed 2026-10-05T12:58:28.672155Z adbcbf9 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Deploying simple-module-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: 66dd945
Status: ✅  Deploy successful!
Preview URL: https://f6bc7d72.simple-module-python.pages.dev
Branch Preview URL: https://feat-352-353-file-storage-pu.simple-module-python.pages.dev

View logs

@antosubash

Copy link
Copy Markdown
Owner Author

Merge-order note: this PR's migration e5a8c1f27b94 and #406's e5f2a8c1d7b3 both descend from c7f2d9a41e83. Whichever PR merges second must repoint its down_revision to the other's revision (otherwise main gets two mainline heads).

https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4

@antosubash

Copy link
Copy Markdown
Owner Author

/ship results (optimize, review, QA, local CI)

Pushed 5 commits (adbcbf9..da03d56). Report: https://claude.ai/artifact/KGrv9YGs1YZhQ73NkDunWk

Optimize: net -38 lines (shared not_found/headers helpers, dead code, one-use wrappers, width snapped once).

Code review (3 passes, security-heavy): fixed uniform 404 when bytes are missing, atomic thumbnail writes, CSP kept only when it carries sandbox, Pillow SyntaxError/struct.error -> 422, Content-Disposition control chars, DB connection released before thumbnail generation (anonymous burst could exhaust the pool), variants deleted even if the original delete fails. Accepted: PATCH public needs the upload permission (same as upload with public=true), failed decodes not negatively cached (bounded by 2 decode slots), public max-age=3600 lag after unpublish.

QA (browser + HTTP agents, app on API 8100 / Vite 5250, SQLite): uploaded PNG/JPG/PDF/HTML/SVG, toggled public/private in Browse, logged-out context 200 when public and identical 404 when private/unknown/deleted/soft-deleted, HTML/SVG/XML/JS always attachment + nosniff + sandbox CSP (no inline render, no dialogs), LIKE wildcards literal, search/content_type/sort, thumbnails at several widths (snapped to 5 sizes), non-image 404, 40000x40000 PNG refused (422, server responsive), 30 concurrent cold thumbnails all 200. Bugs found and fixed: anonymous public responses set a 14-day session cookie next to Cache-Control: public (P2), PATCH public accepted "yes"/1 (P3), 16-bit grayscale PNG failed at w=64 (P3). Retest of fixes and regression: 0 failures. Not covered: multi-tenant mode in the browser (existing tenancy tests pass). Known, left alone: created_at lacks a timezone suffix (not from this PR); Browse table does not show thumbnails yet; no sort control in the UI.

Local CI: make lint ok, make test-py 3619 passed / 9 skipped, make test-js ok, make ci-check-untranslated ok, make doctor ok. Migration e5a8c1f27b94 untouched (down_revision unchanged; merge-order note with #406 stands).

Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4

@antosubash
antosubash merged commit b3bdfa5 into main Oct 7, 2026
14 checks passed
@antosubash
antosubash deleted the feat/352-353-file-storage-public-and-picker branch October 8, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant