Add file_storage module with filesystem and S3 backends

[antosubash]

Summary

Introduces a complete file storage module with pluggable backends, REST API, and web UI for managing uploaded files. Supports both local filesystem and S3-compatible object storage (AWS S3, MinIO, Cloudflare R2, etc.).

Key Changes

Core Service & Orchestration

• FileStorageService: Orchestrates upload validation, stream-based SHA256 hashing, backend I/O, and database lifecycle management
• Implements compensation logic: rolls back backend uploads if database writes fail
• Validates file size limits and content-type allowlists before persistence
• Supports both streaming downloads (proxied through app) and presigned URL redirects (backend-dependent)

Storage Backends

• FilesystemBackend: Writes objects to local filesystem with automatic sharding (first 2 chars) to prevent directory bloat
• S3Backend: S3-compatible storage with lazy aioboto3 import; supports presigned URLs for efficient downloads
• Backend Registry: Extensible registration system allowing third-party providers to self-register via decorator
• Unified StorageBackend protocol with capability flags (supports_presigned_url) for capability-based dispatch

REST API

• POST /api/file-storage/upload: Upload with validation, returns metadata + checksum
• GET /api/file-storage/files: Paginated file listing
• GET /api/file-storage/files/{id}: Fetch single file metadata
• GET /api/file-storage/files/{id}/download: Download (streams or redirects based on backend)
• DELETE /api/file-storage/files/{id}: Delete file and metadata
• Proper HTTP status codes (201 Created, 413 Payload Too Large, 415 Unsupported Media Type, 404 Not Found)

Web UI

• Browse.tsx: React component with file listing, upload dropzone, delete confirmation dialog
• UploadDropzone.tsx: Drag-and-drop file upload with progress feedback
• Responsive table with filename, size, type, uploader, and action buttons
• Toast notifications for success/error feedback
• Permission-based UI visibility (upload/delete buttons hidden if lacking permissions)

Database & Models

• StoredFile SQLModel with audit trail (created_by, created_at, updated_by, updated_at) and soft-delete support
• Tracks: key, filename, content_type, size_bytes, backend_id, checksum_sha256
• Alembic migration creates file_storage schema (PostgreSQL) with indexed columns
• Supports multi-backend scenarios (backend column records which provider holds the bytes)

Configuration & Settings

• Environment-based configuration via SM_FILE_STORAGE_* prefix
• Configurable max file size, allowed content types, S3 endpoint/credentials
• Defaults: filesystem backend writing to ./uploads/
• S3 presigned URL TTL configurable (default 3600s)

Events & Integration

• Domain events: FileUploaded, FileDeleted published to event bus
• Module registration with permission groups (UPLOAD, DOWNLOAD, DELETE, MANAGE)
• i18n support with English locale and generated TypeScript keys
• Proper error codes and user-facing messages

Testing

• Comprehensive test suite: service validation, backend round-trips, compensation logic, API integration
• S3 backend tests against real moto HTTP server (not mocked)
• Filesystem backend path traversal protection tests
• Backend registry extension tests

Notable Implementation Details

• Stream-based hashing: SHA256 computed during upload without buffering entire file
• Lazy imports: S3 backend imports aioboto3 only if selected, with clear error if missing
• Spool-to-disk: S3 backend spools large uploads to temp file (configurable threshold) before put_object
• Sharded filesystem layout: Prevents millions of files in single directory
• Presigned URL support: Backends declare capability; service dispatches to redirect or stream accordingly
• Audit trail: All files track creator and timestamps via AuditMixin
• Soft deletes: Files marked deleted but retained for audit/recovery

https://claude.ai/code/session_01PS5Dk1UvtZaf8f5CPWqo7e