From 24631efac3a272a918a995a421b156d90e08f6e6 Mon Sep 17 00:00:00 2001 From: MrChengLen Date: Tue, 29 Sep 2026 09:20:49 +0200 Subject: [PATCH 1/5] docs(selfhost): systemd unit, own-database overlay, licences and release facts match the repo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The self-hosting, installation, development, security and licensing docs had drifted from the code they describe: - systemd unit: /usr/bin on PATH (ffmpeg, gs and soffice were invisible), an optional root-owned EnvironmentFile for the variables the app reads only from the process environment (DATABASE_URL, FORWARDED_ALLOW_IPS, FILEMORPH_IMAGE_MAX_MEGAPIXELS, JWT_SECRET), a 127.0.0.1 bind, and one process: limiter and concurrency caps are per process, so extra workers or service instances multiply every limit. - Cloud overlay: a DATABASE_URL in .env loses to the overlay's own environment: value; to use your own database, edit the overlay and remove the postgres service and its depends_on. POSTGRES_PASSWORD falls back to "changeme"; JWT_SECRET has no fallback. - installation.md: the container user is a system user with an unpinned UID (look it up instead of chown 1000:1000); /ready checks DB and tempdir, not ffmpeg; APP_PORT is read only by run.py; python3 instead of python3.11. - development.md: the parity list names the test that pins each place a new format must appear. - security-overview.md: the shipped tax-retained deletion path instead of "Stripe accounts get 409", the webhook events, PGP wording aligned with SECURITY.md and release-signing.md, accepted advisories listed. - threat-model.md: rate limits key on request.client.host with FORWARDED_ALLOW_IPS, not X-Forwarded-For. - third-party-licenses.md: Ghostscript (AGPL-3.0, used unmodified as a separate program), LibreOffice (MPL-2.0) and fonts in the office image, pillow-avif-plugin with its codecs, Chart.js and the Tailwind bundle; httpx dropped; the v1.1.0 SBOM caveat. - patch-policy.md: the actual two-track release model (continuous latest/sha-* images, tags at the maintainer's discretion — one so far). - email-setup.md: no SMTP means 200 plus a log line, not 503. Full suite 1471 passed / 72 skipped; gitleaks and scope-guard clean; security-auditor and code-reviewer findings addressed. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 27 +++++ docs/development.md | 99 ++++++++++++--- docs/email-setup.md | 36 ++++-- docs/incident-response.md | 7 +- docs/installation.md | 107 ++++++++++++---- docs/patch-policy.md | 47 +++++--- docs/release-signing.md | 4 +- docs/security-overview.md | 94 ++++++++++----- docs/security-pentest-report.md | 14 ++- docs/self-hosting.md | 208 +++++++++++++++++++++++++++++--- docs/third-party-licenses.md | 100 +++++++++++---- docs/threat-model.md | 17 +-- 12 files changed, 604 insertions(+), 156 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7998f19..fad6285 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,33 @@ Versions follow [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Fixed — self-hosting and security docs: systemd unit, licences and release facts + +The self-hosting, installation, development, security and licensing docs had +drifted from the code they describe. The systemd unit was missing `/usr/bin` +on `PATH` (ffmpeg/Ghostscript/LibreOffice invisible to a non-Docker install), +never loaded `.env` into the process environment (`DATABASE_URL`, +`FORWARDED_ALLOW_IPS` and `FILEMORPH_IMAGE_MAX_MEGAPIXELS` are read directly +from `os.environ`, not through the app's own `.env` parsing), bound +`0.0.0.0` instead of `127.0.0.1`, and ran multiple workers despite the rate +limiter and concurrency caps being per-process. Account deletion, the Stripe +webhook table and the audit-log hash paragraph described a pre-launch state +(Stripe-touched accounts refused with 409) instead of the shipped +tax-retained deletion path. `third-party-licenses.md` was missing +Ghostscript, LibreOffice, `pillow-avif-plugin` and the vendored Chart.js / +Tailwind assets, still listed `httpx` as a runtime dependency (it's dev-only), +and pointed at a scan/SBOM that predates the lockfile-parity and +SBOM-hardening work. `patch-policy.md` claimed every merge is tagged and a +release lands every 1–4 weeks; it now describes the actual two-track model +(continuous `latest`/`sha-*` builds on every merge, a `vX.Y.Z`/`X.Y.Z` tag at +the maintainer's discretion — one so far, `v1.1.0`). Smaller fixes: +`/ready` doesn't check ffmpeg (DB + tempdir only), `APP_PORT` is read only by +`run.py`, `python3.11` is no longer an installable apt package on current +Ubuntu/Debian, a Docker UID/GID mismatch on `./data` needs a `chown` not a +`chmod`, the email-setup guide's "no SMTP" behaviour is `200` + a log line +(not `503`), and the pentest report's Tailwind-CDN / inline-script claims +are corrected in place rather than silently rewritten. + ### Fixed — website texts match what the service does The public pages were checked claim by claim against the code: diff --git a/docs/development.md b/docs/development.md index 7414720..dbe7580 100644 --- a/docs/development.md +++ b/docs/development.md @@ -17,29 +17,42 @@ filemorph/ │ │ ├── convert.py # POST /api/v1/convert │ │ ├── compress.py # POST /api/v1/compress │ │ ├── formats.py # GET /api/v1/formats -│ │ └── health.py # GET /api/v1/health +│ │ ├── health.py # GET /api/v1/health, /ready +│ │ ├── auth.py # Cloud Edition: register / login / account +│ │ ├── keys.py # Cloud Edition: dashboard API-key CRUD +│ │ ├── billing.py # Cloud Edition: Stripe checkout + webhook +│ │ └── cockpit.py # Cloud Edition: admin routes │ ├── core/ │ │ ├── config.py # Settings loaded from .env via pydantic-settings -│ │ └── security.py # API key generation, hashing, validation +│ │ ├── security.py # API key generation, hashing, validation +│ │ ├── rate_limit.py # slowapi limiter + failed-key budget +│ │ ├── quotas.py # Per-tier size / concurrency / output caps +│ │ └── audit.py # Tamper-evident audit-log hash chain │ ├── converters/ -│ │ ├── base.py # AbstractConverter base class +│ │ ├── base.py # BaseConverter base class │ │ ├── registry.py # Converter registry (@register decorator) -│ │ ├── image.py # Image conversions (Pillow + pillow-heif) +│ │ ├── image.py # Image conversions (Pillow + pillow-heif + pillow-avif-plugin) │ │ ├── document.py # Document conversions (docx, pdf, txt, md) │ │ ├── video.py # Video conversions (ffmpeg-python) │ │ ├── audio.py # Audio conversions (ffmpeg-python) │ │ └── spreadsheet.py # Spreadsheet conversions (openpyxl, csv, json) │ ├── compressors/ -│ │ ├── image.py # Image quality compression (Pillow) -│ │ └── video.py # Video CRF compression (ffmpeg) +│ │ ├── image.py # Image quality / target-size compression (Pillow) +│ │ ├── video.py # Video CRF compression (ffmpeg) +│ │ └── pdf.py # PDF compression +│ ├── db/ # SQLAlchemy models + async engine (Cloud Edition) +│ ├── ee/ # Commercial-licensed add-ons (PII redaction) — inert by default │ ├── models/ │ │ └── schemas.py # Pydantic response schemas -│ ├── static/ # CSS and JavaScript +│ ├── static/ # CSS, JavaScript and vendored assets (Chart.js) │ └── templates/ # Jinja2 HTML templates +├── alembic/ # Cloud Edition schema migrations ├── tests/ ├── scripts/ │ ├── generate_api_key.py # CLI key generator -│ └── first_run.py # Called by Docker entrypoint on first start +│ ├── promote_admin.py # Promote a registered user to the admin role +│ ├── first_run.py # Called by Docker entrypoint on first start +│ └── build-tailwind.sh # Rebuilds the self-hosted Tailwind bundle ├── data/ │ └── api_keys.json # Hashed API keys (gitignored) ├── run.py # Entry point for direct Python runs @@ -48,7 +61,11 @@ filemorph/ ├── start.bat # Windows launcher: Docker mode ├── start.sh # Linux/macOS launcher: Docker mode ├── entrypoint.sh # Docker container entrypoint (first-run key setup) -└── docker-compose.yml +├── docker-compose.yml # Community Edition (default) +├── docker-compose.cloud.yml # Cloud Edition overlay (Postgres, JWT, billing) +├── docker-compose.office.yml # High-fidelity docx→pdf overlay (LibreOffice) +├── requirements.txt # Direct dependencies (source of truth for versions) +└── requirements.lock # Hash-pinned lockfile — what the image actually installs ``` --- @@ -67,13 +84,21 @@ cd FileMorph first run, and starts uvicorn with `--reload`. Code changes are picked up automatically without restarting the server. +`dev.ps1` installs only `requirements.txt` (the runtime dependencies). To run +tests or lint locally, also install the dev tools it doesn't cover +(`pytest`, `ruff`, `pip-audit`, …): + +```powershell +.venv\Scripts\pip.exe install -r requirements-dev.txt +``` + ### Linux / macOS ```bash git clone https://github.com/MrChengLen/FileMorph.git cd FileMorph -python3.11 -m venv .venv +python3 -m venv .venv # Python 3.11 or newer source .venv/bin/activate pip install -r requirements-dev.txt @@ -85,7 +110,7 @@ uvicorn app.main:app --reload ### Live reload -With `--reload`, uvicorn watches `Z:\Python\projects\filemorph` for file changes and +With `--reload`, uvicorn watches the project directory for file changes and restarts the server process automatically. No manual restart needed when editing Python files. --- @@ -101,7 +126,8 @@ Run a single test file: pytest tests/test_convert_image.py -v ``` -Run with coverage: +Run with coverage (optional — `pytest-cov` isn't in `requirements-dev.txt`, +install it separately: `pip install pytest-cov`): ```bash pytest tests/ --cov=app --cov-report=term-missing ``` @@ -248,9 +274,11 @@ def test_epub_to_txt(client, auth_headers, tmp_path): assert len(res.content) > 0 ``` -### Step 6 — Update format documentation +### Step 6 — Update the format lists -Add the new format to [docs/formats.md](formats.md). +Formats are also listed by hand in several places — [docs/formats.md](formats.md) is one — +and tests compare them with the registry, so the build fails until they agree. The full +list, with the test that pins each place, is under "Parity places" below. --- @@ -268,6 +296,33 @@ def compress_image(input_path: Path, output_path: Path, quality: int = 85) -> Pa Add the new format to the `_SUPPORTED_FORMATS` list in the relevant compressor file, and import + call it from `app/api/routes/compress.py`. +**Parity places to update in the same PR**, for either a new converter or a new +compressor format — each is pinned by a test, so a missed one fails the build: + +- `docs/formats.md` (the From → To tables and the Audio/Video lists), the README + (the drop-zone mockup and the "Supported Formats" table), the homepage FAQ + answer "Which file formats can I convert?" (EN + DE), the "FileMorph converts …" + sentence in `/llms.txt`, and the "Convert …" entries of the JSON-LD `featureList` + (`app/core/jsonld.py`) — `tests/test_format_lists_match_registry.py` compares + each with the registry. +- The homepage drop-zone captions `#supported-convert` and `#supported-compress` + (`app/templates/partials/convert_tool.html`, EN + DE) — + `tests/test_homepage_drop_zone_modes.py` compares them with `/api/v1/formats`. +- `_HOMEPAGE_ADVERTISED` in `tests/test_format_registry.py` — the test's own list of + the formats the homepage names. Add the format there; the test only checks that + every listed format is registered, so a missing entry goes unnoticed. +- `_FORMAT_CATEGORY` in `app/api/routes/pages.py` — `tests/test_formats_categories.py` + fails for a source format without a category (it would land in the "Other" + bucket on `/formats`). +- If the format supports exact-size compression, `TARGET_SIZE_FORMATS` in + `app/compressors/image.py` **and** the matching `TARGET_SIZE_FORMATS` array + in `app/static/js/app.js` — `tests/test_target_size_formats_parity.py` + fails the build if the two disagree, and also pins the hand-written claims about + which formats hit an exact target (homepage FAQ, `/formats`, `/llms.txt`, the + OpenAPI form-field docs, the `/tools` card); the `/compress` page copy is pinned + by `tests/test_compress_page.py`. +- A test per new format/pair (Step 5 above / the equivalent for compressors). + --- ## API key internals @@ -284,21 +339,25 @@ All logic is in `app/core/security.py`. ## Environment variables reference -Defined in `app/core/config.py` using pydantic-settings: +Defined in `app/core/config.py` using pydantic-settings — a representative +slice (the real class has ~40 fields: JWT, Stripe, SMTP, audit-log, +concurrency, AI-redaction and office-engine settings besides these): ```python class Settings(BaseSettings): app_host: str = "0.0.0.0" app_port: int = 8000 app_debug: bool = False - app_version: str = "1.0.0" - api_keys_file: str = "data/api_keys.json" + app_version: str = "1.1.0" + api_keys_file: str = "" # resolved to data/api_keys.json if left empty max_upload_size_mb: int = 100 - cors_origins: str = "*" + cors_origins: str = "http://localhost:8000" ``` -All settings can be overridden via environment variables or `.env` (uppercase, same names): -`APP_HOST`, `APP_PORT`, `APP_DEBUG`, `API_KEYS_FILE`, `MAX_UPLOAD_SIZE_MB`, `CORS_ORIGINS` +All settings can be overridden via environment variables or `.env` +(uppercase, same names). [`.env.example`](../.env.example) is the +source of truth for the full list — every variable there carries a +one-line description; this section only shows the shape. --- diff --git a/docs/email-setup.md b/docs/email-setup.md index aec12b8..61c57fc 100644 --- a/docs/email-setup.md +++ b/docs/email-setup.md @@ -29,10 +29,19 @@ is unset (NULL) the operator default `LANG_DEFAULT` applies. The dunning mail fires from a Stripe webhook with no HTTP request to derive a locale from, which is exactly why the column exists. -If `SMTP_HOST` is empty, every feature above degrades gracefully: - -- `/forgot-password` and `/resend-verification` return `503 Service Unavailable` - with a reason string the UI surfaces. +If `SMTP_HOST` is empty, every feature above degrades gracefully — all of +them return their normal success status regardless, and the outbound +send is silently skipped: + +- `/forgot-password` and `/resend-verification` still return `200`. + `send_email()` sees `SMTP_HOST` is empty, logs `send_email skipped — + SMTP not configured (to_domain=…, subject=…)` at WARNING and returns + without sending — no exception reaches the route, so there is no + error status to surface. `/forgot-password` is deliberately + enumeration-safe: it always returns the same generic response whether + the address exists, the user is inactive, or the email was skipped, + so the response alone never tells a caller which case occurred — check + the application log to see what actually happened. - `/register` still creates the user — the verification email is fire-and-forget. The user can request a fresh link via `/resend-verification` once SMTP is wired. - `/auth/account` deletes the user even if the confirmation email cannot be sent; @@ -168,12 +177,19 @@ curl -X POST https://your-domain.example.com/api/v1/auth/forgot-password \ # Expected: an email arrives at the inbox within seconds. ``` -If no mail arrives, check: - -1. **Application log** — the sender logs `send_email ok` (success) or - `send_email failed` (failure). The latter is logged at exception level - with full SMTP error details visible only in the server log; the HTTP - response stays generic so the SMTP details never leak to the client. +The request returns `200` either way (see "What needs SMTP" above), so a +successful-looking response does not by itself mean the email was sent — +check the application log for what actually happened. If no mail +arrives, check: + +1. **Application log first.** Three possible lines: `send_email skipped + — SMTP not configured` (WARNING — `SMTP_HOST` is empty; the container + may not have picked up an `.env` change, or the overlay/unit file + isn't passing it through), `send_email ok` (success — the message left + for the provider; a delivery problem from here on is provider-side), + or `send_email failed` (the exception is logged server-side with full + SMTP error details; the HTTP response stays generic so those details + never leak to the client). 2. **DNS / SPF / DKIM / DMARC** — for ESPs and mailbox providers, the sending domain must have valid SPF and DKIM records pointing at the provider, plus a DMARC policy. Without alignment, Gmail and Outlook diff --git a/docs/incident-response.md b/docs/incident-response.md index cef8bf8..ec04fa4 100644 --- a/docs/incident-response.md +++ b/docs/incident-response.md @@ -58,9 +58,10 @@ contact is added to the response thread before the advisory goes public. image is signed and pushed, and the advisory is published. Compliance- Edition customers on the security mailing list receive the advisory five working days before public disclosure when feasible. -6. **Post-mortem.** Within 30 days of disclosure the maintainer publishes - a short post-mortem in the project's `runbooks/` (where a runbook - directory exists) or as a follow-up release note. Format below. +6. **Post-mortem.** Within 30 days of disclosure the maintainer records a + short post-mortem in a private incident log and, where the details are + appropriate for a public audience, publishes a follow-up release note. + Format below. ## Post-mortem template diff --git a/docs/installation.md b/docs/installation.md index 9099c68..df2d199 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -154,18 +154,22 @@ cd FileMorph .\dev.ps1 ``` -On first run, `dev.ps1` automatically: +`dev.ps1` automates four steps on every start, then launches the server: | Step | What happens | |------|-------------| -| 1/4 | Creates `.venv` virtual environment | -| 2/4 | Installs all dependencies from `requirements.txt` | -| 3/4 | Copies `.env.example` to `.env` | -| 4/4 | Generates your API key (shown once — save it) | +| 1/4 | Creates `.venv` virtual environment (skipped once it exists) | +| 2/4 | Installs / verifies dependencies from `requirements.txt` | +| 3/4 | Copies `.env.example` to `.env` (skipped once `.env` exists) | +| 4/4 | Generates your API key (skipped once one exists — shown once, save it) | | Done | Starts uvicorn at `http://127.0.0.1:8000` with `--reload` | -On subsequent starts, all setup steps are skipped. The server starts in seconds, -with no internet connection required. +Steps 1, 3 and 4 are skipped once their target already exists, so a repeat +start is fast — but step 2 runs `pip install` **every time**, not just on +first run, so a `git pull` that added a dependency is picked up +automatically. That means every start needs network access to reach the +package index, even when nothing actually changed (a no-op check, but +not an offline one). ### Optional — Desktop shortcut @@ -177,6 +181,28 @@ Places a `FileMorph` shortcut on your Desktop. Double-clicking it starts the ser without opening a terminal manually. The window stays open so you can see server logs and any errors. +### Optional — PDF rendering support (WeasyPrint, Ghostscript) + +DOCX, Markdown, HTML and EML → PDF render through WeasyPrint, which needs +the Pango/GTK native libraries; TXT → PDF does not (it uses `reportlab`, +pure Python, no extra install). On Windows, the pinned WeasyPrint version +(`weasyprint>=69.0,<70`) looks for those libraries in +`C:\msys64\mingw64\bin` or `C:\Program Files\GTK3-Runtime Win64\bin` by +default (override with the `WEASYPRINT_DLL_DIRECTORIES` env var, `;`-separated) — +install either an MSYS2 `mingw64` environment with Pango, or the standalone +GTK3 Runtime Win64 installer. Full steps: +[doc.courtbouillon.org/weasyprint/stable/first_steps.html#installation](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#installation). +Without it, those four conversions fail at request time; everything else +(images, audio, video, spreadsheets, TXT → PDF) is unaffected. + +`pdf → pdfa` (full PDF/A-2b conformance) needs Ghostscript on PATH — +install it from [ghostscript.com](https://www.ghostscript.com/releases/) +and make sure its `bin` folder (containing `gswin64c.exe`) is on PATH. +Without it, `pdf → pdfa` still works but falls back to a markup-only +output that veraPDF rejects if the source has unembedded fonts — same +trade-off as the Docker image, see +[`docs/self-hosting.md`](self-hosting.md#pdfa-2b-conformance-optional-ghostscript). + ### Stopping the server Press `Ctrl+C` in the PowerShell window. This stops only the server process. @@ -198,13 +224,22 @@ git pull ```bash sudo apt update sudo apt install -y \ - python3.11 python3.11-venv python3-pip \ + python3 python3-venv python3-pip \ ffmpeg \ ghostscript \ libheif-dev \ libcairo2 libpangocairo-1.0-0 libgdk-pixbuf2.0-0 +python3 --version # must be 3.11 or newer ``` +`python3.11` as a specific apt package name is a moving target — current +Ubuntu (24.04+) and Debian (13+) ship a newer default `python3` (3.12 / +3.13) and no longer carry a `python3.11` package at all, so pinning that +exact name in the install command fails on a fresh system. Use the +distro's own `python3` and confirm the version is ≥ 3.11; if your distro's +default is older, add the [deadsnakes PPA](https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa) +(Ubuntu) or use `pyenv` instead of chasing a specific apt package name. + > `ghostscript` is optional but enables full PDF/A-2b conformance. Without it, > `pdf → pdfa` falls back to markup-only output. See > [docs/self-hosting.md](self-hosting.md) for the trade-off. @@ -215,7 +250,7 @@ sudo apt install -y \ git clone https://github.com/MrChengLen/FileMorph.git cd FileMorph -python3.11 -m venv .venv +python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt @@ -241,9 +276,12 @@ Expected response: ``` `/api/v1/health` is the unauthenticated liveness probe — minimal by design so it -discloses no version or codec details. To verify ffmpeg is on PATH (needed for video -and audio), call `GET /api/v1/ready`, which reports operational state without leaking -it to the public internet. +discloses no version or codec details. `GET /api/v1/ready` reports whether the app +can actually serve traffic: the database (Cloud Edition; reported `skipped` when +none is configured) and the tempdir are writable. It does **not** check ffmpeg — +if ffmpeg is missing, `/ready` still reports healthy, and the only signal is a +`ffmpeg not found on PATH` line in the startup log (video/audio conversion then +fails at request time instead). --- @@ -264,7 +302,7 @@ Audio and video conversion will not work until ffmpeg is installed. - **Linux:** `sudo apt install ffmpeg` - **Docker:** ffmpeg is bundled in the image — no action needed -### Cloud-mode 500s on `/auth/register` after `docker compose up` +### Cloud-mode 503s (`Database not configured.`) on `/auth/register` You started the default community-mode compose, which has no Postgres. Either use the community-mode flow (no accounts), or layer the Cloud @@ -282,13 +320,22 @@ On a slow connection, increase the timeout: ### Port 8000 already in use -Change the port in `.env`: - -```env -APP_PORT=8080 -``` - -Then restart the server. +`APP_PORT` in `.env` is read only by `run.py` (the PyInstaller / direct +`python run.py` entry point) — it does **not** change the port for the +other three installation methods, which all hardcode `8000`: + +- **Docker** (Methods 1 and 2): the container always listens on `8000` + internally (`entrypoint.sh`). Change the **host** side of the port + mapping in `docker-compose.yml` instead — e.g. `"8080:8000"` — and + reach the app at `http://localhost:8080`. +- **`dev.ps1`** (Method 3): also starts uvicorn on a hardcoded `8000`. + Edit the `--port 8000` argument in `dev.ps1` itself if you need a + different port. +- **Manual `uvicorn app.main:app`** (Method 4): pass `--port 8080` on + the command line; `APP_PORT` has no effect here either since + `uvicorn`'s CLI flags take precedence over anything in `.env`. +- **`python run.py`**: this is the one path that honours `APP_PORT` — + set it in `.env` and restart. ### "ModuleNotFoundError: No module named 'pillow_heif'" @@ -300,6 +347,24 @@ On Linux, also install: `sudo apt install libheif-dev` ### Permission denied on `data/api_keys.json` (Linux) +Usually a UID mismatch on the bind-mounted `./data` directory: the +Docker image runs as a non-root `appuser` (a system user whose UID the +Dockerfile does not pin, typically 999), so if the host +`./data` directory is owned by root or by your own user account, the +container can't create or update files inside it. Look up the UID and GID the container actually runs as, +then give `./data` to them: + ```bash -chmod 600 data/api_keys.json +docker compose run --rm --entrypoint id filemorph +# prints e.g. uid=999(appuser) gid=999(appuser) groups=999(appuser) + +sudo chown -R : ./data # the two numbers from that line +docker compose restart filemorph ``` + +If the container is already running, `docker compose exec filemorph id` +prints the same line. + +Running outside Docker (Method 4), the file is owned by whoever +generated it — `chmod 600 data/api_keys.json` restricts it to that +user if it was created with looser permissions. diff --git a/docs/patch-policy.md b/docs/patch-policy.md index 1568713..7c714be 100644 --- a/docs/patch-policy.md +++ b/docs/patch-policy.md @@ -8,15 +8,22 @@ compatible with their patch-management requirements. ## Release line -FileMorph uses a single `main` branch. Each merge to `main` that ships a -user-visible change is tagged `vX.Y.Z` and built into a Docker image -published to GitHub Container Registry under -`ghcr.io/mrchenglen/filemorph`. - -There is no long-term-support branch. Self-hosters track the latest -`main` tag, or pin to a specific `vX.Y.Z` and upgrade on their own -schedule. Pinning to a major version (e.g. `v1`) is supported and -follows the SemVer guarantee below. +FileMorph uses a single `main` branch. **Every merge to `main`** builds +and pushes the `latest` (slim) and `office` Docker images, plus a +`sha-` tag for each — this is the continuous path, and how +most fixes reach a self-hoster: pull `latest` / `office` again. +Separately, at the maintainer's discretion, a commit on `main` gets a +GPG-signed git tag `vX.Y.Z`; that tag build additionally pushes the +image tags `X.Y.Z` and `X.Y` — **no `v` prefix on the image tag, and no +bare-major tag** (pin `X.Y.Z` or `X.Y`, never just `X`). As of this +writing there has been one such tagged release, `v1.1.0` (2026-06-01) — +see [GitHub Releases](https://github.com/MrChengLen/FileMorph/releases) +for the current list. + +There is no long-term-support branch. Self-hosters either track `latest` +/ `office` for continuous fixes with no version pinning, or pin to a +specific `X.Y.Z` / `X.Y` image tag and upgrade deliberately on their own +schedule. ## Versioning @@ -45,8 +52,11 @@ uses (CVSS v3.x base score). The patch-release timelines below apply | Medium | 4.0 – 6.9 | next regular release | | Low | 0.1 – 3.9 | next regular release | -A *regular release* is the next planned `MINOR` or `PATCH` cut, which -historically lands every 1–4 weeks. +A *regular release* is the next tagged `vX.Y.Z` cut, made at the +maintainer's discretion rather than on a fixed cadence — see "Release +line" above. Independently of tagged releases, a merged fix reaches the +continuously built `latest` / `sha-*` images as soon as it lands on +`main`. For deployments behind an air-gap or with a fixed change-window, we publish patch-only branches on request — contact `security@filemorph.io` @@ -125,14 +135,17 @@ the new `MAJOR` along with the migration guide. Recommended cadence: -1. Pin to a `vX.Y` tag (e.g. `v1.0`). +1. Pin to a specific image tag (`X.Y.Z` or `X.Y`, e.g. `1.1.0`) for + controlled, deliberate upgrades — or track `latest` / `office` if you + want fixes as soon as they merge to `main`. 2. Subscribe to GitHub Releases on this repository (the *Watch → Custom → - Releases* setting). -3. Schedule a redeploy after every PATCH or MINOR release, or at minimum - monthly. + Releases* setting) to hear about tagged `vX.Y.Z` cuts. +3. If pinned to a version tag, redeploy when a new one ships. If tracking + `latest`, re-pull periodically — otherwise fixes that already merged + never reach your instance. 4. Subscribe to GitHub Security Advisories on this repository to be - notified of Critical and High issues out-of-band from the regular - release cycle. + notified of Critical and High issues out-of-band from tagged + releases. For deployments where each upgrade requires an internal change-window, the SBOM and signed image attestations let your security team diff --git a/docs/release-signing.md b/docs/release-signing.md index 593f4a5..d48a02b 100644 --- a/docs/release-signing.md +++ b/docs/release-signing.md @@ -23,8 +23,8 @@ signing is not enough — the published image must be tied to a verifiable identity. This document plus the cosign workflow cover both surfaces. -ISO 27001 A.14.2.4 ("System acceptance testing") and BSI APP.5.1 -("Container") both expect the signing claims to be reproducible +ISO 27001:2013 A.14.2.9 ("System acceptance testing") and BSI SYS.1.6 +("Containerisierung") both expect the signing claims to be reproducible *outside* the repository — i.e. a third-party auditor can verify without our help. Sigstore's transparency log (Rekor) and the public PGP keys below satisfy that expectation. diff --git a/docs/security-overview.md b/docs/security-overview.md index 9bbd24b..4fbcaaf 100644 --- a/docs/security-overview.md +++ b/docs/security-overview.md @@ -432,21 +432,27 @@ mechanically. - Email and bcrypt-hashed password live in Postgres (`users` table). - API keys live as SHA-256 hashes in `api_keys`. -- File-content hashes, original filenames, or upload metadata are - not persisted. The `usage_records` table records only an - operation type, byte counts, and a timestamp. +- Original filenames and upload metadata are not persisted. The + `usage_records` table records only an operation type, byte counts, + and a timestamp. The output SHA-256 of a single-file convert/compress + result *is* persisted, though — as `output_sha256` in the audit-log + payload (see "File data" above); that hash is the verification anchor + the audit trail is built on, not a record of the file's contents. - **Self-service account deletion** lives at `DELETE - /api/v1/auth/account` (Art. 17 GDPR). The free path is fully - self-service: three-field re-confirmation (`password`, - `confirm_email`, `confirm_word="DELETE"`), last-active-admin - guard returning 409, and a confirmation email after commit. - Cascade is hybrid: `api_keys` rows are removed, `file_jobs` and - `usage_records` actor IDs are nulled (analytics integrity - preserved), audit-event `actor_user_id` is nulled (the - `event_type` and payload survive). Accounts that have ever - touched Stripe are refused with 409 directing the user to the - operator support contact until the paid-path tax-retention - flow ships under HGB §257 / AO §147 — see + /api/v1/auth/account` (Art. 17 GDPR), fully self-service on both + paths: three-field re-confirmation (`password`, `confirm_email`, + `confirm_word="DELETE"`), a last-active-admin guard returning 409, + and a confirmation email after commit (204 on success). Free / + never-paid accounts get a full hard-delete: `api_keys` rows are + removed, and `file_jobs` and `usage_records` actor IDs are nulled + (analytics integrity preserved). + Accounts that have ever touched Stripe take the restricted + **tax-retained** path instead: any active subscription is + cancelled first (a Stripe error aborts the whole request with 500 + and leaves the account unchanged), then the `users` row is kept + with only `email` / `stripe_customer_id` / `tier` / `created_at` + surviving, for the HGB §257 / AO §147 ten-year retention window + (permitted under Art. 17(3)(b) GDPR) — see [`gdpr-account-deletion-design.md`](./gdpr-account-deletion-design.md) § 5.B for the design. @@ -559,11 +565,13 @@ Single-instance deployments are not affected. ### Stripe webhook coverage -The webhook handler currently dispatches on -`customer.subscription.*` and `checkout.session.completed`. -`invoice.payment_failed` and `invoice.payment_succeeded` are not -yet wired, so dunning state on a failed renewal will not flow -back to the application until the next subscription event. +The webhook handler dispatches on `customer.subscription.created`, +`customer.subscription.updated`, `customer.subscription.deleted` (tier +reverts to Free) and `invoice.payment_failed` (sends the debounced +dunning email). `checkout.session.completed` and +`invoice.payment_succeeded` are not handled — the subscription events +above already carry the tier change, so a successful checkout or +renewal doesn't need a separate handler to take effect. ### Email verification @@ -615,7 +623,10 @@ alert rules for uptime / error-rate / p95 latency) lives in the private `filemorph-ops` repo and is tracked as a follow-up. Until it is deployed there is no alerting on rate-limit hits, error rates, or latency percentiles. Treat the dashboard/alert layer as -required before public launch. +a prerequisite for running the metrics endpoint unattended in +production — wire your own scrape + alerting stack against the +metric families in [`self-hosting.md`](./self-hosting.md#monitoring--metrics) +until then. ### API key in browser localStorage (PT-010) @@ -631,10 +642,14 @@ across the files in a batch upload. A batch designed to scrape the per-file cap many times over could still produce a large total egress. An aggregate cap is on the backlog. -### No PGP key for security@ +### No published PGP key for encrypted reports -`security@filemorph.io` accepts plain email today. Publishing a -PGP key for encrypted reports is on the backlog. +`security@filemorph.io` accepts plain email; encrypted mail is +welcome on request — see [`SECURITY.md`](../SECURITY.md) for the +current disclosure channel. The maintainer public keys in +[`release-signing.md`](./release-signing.md) verify release signatures; +for an encrypted report, ask for a key at the same address, as +`SECURITY.md` says. ### No public bug-bounty programme @@ -647,8 +662,10 @@ but monetary rewards are not offered. ### Reporting a finding -Send email to `security@filemorph.io`. Plain email is acceptable; -encryption is not currently offered. +See [`SECURITY.md`](../SECURITY.md) for the current disclosure +channel, or send email directly to `security@filemorph.io`. Plain +email is acceptable; encrypted mail is welcome — request a PGP key +at the same address. In the report, please include: @@ -696,6 +713,20 @@ Self-hosters who fork this repository should recompile `pip-audit -r requirements.lock` — the lockfile, not `requirements.txt`, is what the image installs. +### Accepted advisories + +Two advisories are currently allow-listed in CI +(`--ignore-vuln` in `.github/workflows/ci.yml`) rather than blocking +the build, each with a documented reason and a re-evaluation trigger: + +| Advisory | Package | Why it's accepted | +|---|---|---| +| PYSEC-2026-1325 (CVE-2024-23342, GHSA-wj6h-64fc-37mp) | `ecdsa`, transitive via `python-jose` | Minerva timing side-channel in ECDSA sign/keygen/ECDH. No fixed version exists upstream. Not reachable here: FileMorph's JWTs are HS256-only (`app/core/tokens.py`), so no ECDSA code path ever runs. Added 2026-07-15; drop once the project migrates off `python-jose` or upstream ships a fix. | +| CVE-2026-55073 (GHSA-jf6q-chmf-3h3v), CVSS 6.2 | `weasyprint` < 70.0 | SSRF-protection bypass: two `write_pdf()` parameters (`xmp_metadata`, `stylesheets`, both accepting a URL) build a fresh default fetcher instead of honouring a custom `url_fetcher`. Not reachable here: every `write_pdf()` call site passes only the output path, never those parameters. The fix (70.0) reworks the `url_fetcher` contract in a way that would break `_deny_url_fetcher`; re-evaluate once that port happens. Added 2026-09-09. | + +The full reasoning (with exact code line references) lives in the +`ci.yml` comments next to the `pip-audit` step. + ### Update cadence - `pip-audit -r requirements.lock` runs in CI as a blocking gate @@ -732,9 +763,12 @@ For readers who want to jump directly to the code: --- -*Last revised 2026-05-06. The findings synthesised here are +*Last revised 2026-09-28. The findings synthesised here are sourced from the static code review dated 2026-04-19 and the -current state of the repository. The 2026-05-06 revision lands -the self-service account-deletion endpoint, the email-verification -flow, and the deployment-agnostic support contact (no FileMorph -SaaS addresses leak into self-hosted error messages).* +current state of the repository. The 2026-09-28 revision corrects +drift against the code: the account-deletion section now reflects +the shipped tax-retained deletion path (no longer a 409 refusal), +the Stripe webhook coverage table matches what is actually wired, +the audit-log paragraph acknowledges the persisted output-hash +attestation, and the accepted-advisories table lists the two +CVEs currently allow-listed in CI.* diff --git a/docs/security-pentest-report.md b/docs/security-pentest-report.md index bc85fec..f258eb3 100644 --- a/docs/security-pentest-report.md +++ b/docs/security-pentest-report.md @@ -41,12 +41,12 @@ The authoritative per-finding mapping to current code lives in | PT-002 | Critical | Non-constant-time API-key comparison | **Addressed** — `hmac.compare_digest` | `app/core/security.py::validate_api_key` | | PT-003 | High | CORS `*` with credentials | **Addressed** — `CORS_ORIGINS` allow-list, never `*` with credentials | `app/main.py::_build_csp_header`; `.env.example` | | PT-004 | High | Internal exception details leaked to clients | **Addressed** — global error handler returns a generic message; the stack trace is logged server-side only | `app/main.py` error handlers | -| PT-005 | High | Missing HTTP security headers / external CDN script | **Addressed** — `security_headers` middleware (HSTS, CSP, X-Frame-Options, Referrer-Policy, Permissions-Policy); Tailwind/fonts/Chart.js served from the deployment's own origin; the only inline script is the hash-pinned Tailwind config | `app/main.py::security_headers`; `tests/test_security_headers.py` | +| PT-005 | High | Missing HTTP security headers / external CDN script | **Addressed** — `security_headers` middleware (HSTS, CSP, X-Frame-Options, Referrer-Policy, Permissions-Policy); Tailwind/fonts/Chart.js served from the deployment's own origin; the only hash-pinned inline script is the JSON-LD structured-data block, not a Tailwind config | `app/main.py::security_headers`; `tests/test_security_headers.py` | | PT-006 | High | Rate-limit bypass via `X-Forwarded-For` / no per-endpoint throttle | **Addressed app-side** — per-endpoint `@limiter.limit(...)` on every API route except `/auth/refresh`, `/auth/me`, the Stripe webhook and `/metrics` (each exempt on purpose, see `api-reference.md` § Rate Limiting), plus a per-IP budget for rejected `X-API-Key` attempts; **residual:** the trust boundary is an operator-config step (`FORWARDED_ALLOW_IPS`), documented | `app/core/rate_limit.py`; `security-overview.md` § Operational Hardening (trusted-proxy) | | PT-007 | Medium | No magic-byte / content-type validation | **Addressed** — `BLOCKED_MAGIC` allow-list rejects PE/ELF/shell/PHP before any decoder runs | `app/core/processing.py` (`BLOCKED_MAGIC`); enforced in the upload routes | | PT-008 | Medium | WeasyPrint SSRF via HTML/CSS in Markdown | **Addressed** — every `weasyprint.HTML(...)` call passes `url_fetcher=_deny_url_fetcher`; WeasyPrint never opens a network connection | `app/converters/document.py` | | PT-009 | Medium | Temp-file orphan risk / world-readable temp files | **Addressed** — `fm_`-prefixed UUID dirs, `shutil.rmtree` in the request `finally`, plus a startup sweep and a periodic background sweep of dirs older than the configured max age | `security-overview.md` § Data Privacy | -| PT-010 | Medium | API key persisted in browser `localStorage` | **Accepted trade-off** — CSP is the primary mitigation (no external script origins; inline config hash-pinned); documented as a known limitation | `security-overview.md` § Known Limitations | +| PT-010 | Medium | API key persisted in browser `localStorage` | **Accepted trade-off** — CSP is the primary mitigation (no external script origins; the one hash-pinned inline script, the JSON-LD block, carries no logic); documented as a known limitation | `security-overview.md` § Known Limitations | | PT-011 | Low | `/health` discloses version + `ffmpeg_available` without auth | **Addressed** — `/health` now returns only `{"status":"ok"}`; the app version and codec availability are not exposed on an unauthenticated endpoint; `/ready` carries only operational state (db / tempdir reachable) | `app/api/routes/health.py`; `tests/test_readiness.py` | | PT-012 | Low | Docker container runs as root | **Addressed** — the image creates a system `appuser` and ends with `USER appuser` | `Dockerfile` | | PT-013 | Medium | Output filename not sanitised for `Content-Disposition` | **Addressed** — `safe_download_name()` strips control/bidi characters and RFC-5987-encodes non-ASCII | `app/core/utils.py::safe_download_name` | @@ -268,6 +268,16 @@ The application returns no security-relevant HTTP response headers. Verified by The Web UI loads Tailwind CSS from `https://cdn.tailwindcss.com` with no SRI (Subresource Integrity) hash. A compromised Tailwind CDN could inject malicious JavaScript. +> **Correction (2026-09-28).** Tailwind is self-hosted today, not loaded +> from `cdn.tailwindcss.com`: a purged, content-hashed bundle, committed +> to the repository (CI only checks that it is up to date), ships from +> `/static/css/tailwind..css` (see +> `docs/tailwind-build-setup.md`), so this finding's CDN-compromise and +> missing-SRI scenario no longer applies. The CSP's single hash-pinned +> inline script is the JSON-LD structured-data block +> (`app/core/jsonld.py`), not a Tailwind config — see PT-005 in the +> resolution table above. + **Impact:** Clickjacking attacks on the Web UI. MIME-type confusion attacks on downloaded files. XSS via CDN supply-chain compromise. Referrer header leaks API keys embedded in URLs to third-party analytics. diff --git a/docs/self-hosting.md b/docs/self-hosting.md index cf2e0a5..db6146d 100644 --- a/docs/self-hosting.md +++ b/docs/self-hosting.md @@ -20,13 +20,18 @@ reverse proxy setup (Caddy or nginx), HTTPS/SSL, and operational best practices. ```bash git clone https://github.com/MrChengLen/FileMorph.git -cd filemorph +cd FileMorph cp .env.example .env ``` Edit `.env` for production: ```env +# APP_HOST / APP_PORT below have no effect on this Docker setup — the +# container's entrypoint always starts uvicorn on 0.0.0.0:8000 regardless +# of what's in .env. They only matter for `python run.py` (the PyInstaller +# / direct-Python entry point). To change the port Docker publishes, edit +# the host side of `ports:` in docker-compose.yml instead (e.g. "8080:8000"). APP_HOST=0.0.0.0 APP_PORT=8000 APP_DEBUG=false @@ -41,6 +46,14 @@ MAX_UPLOAD_SIZE_MB=100 # Restrict to your own domain in production CORS_ORIGINS=https://yourapp.example.com,https://portal.example.com +# Public canonical URL — used in the sitemap, canonical/og:url tags, +# JSON-LD, and (Cloud Edition) the links built into transactional email. +APP_BASE_URL=https://yourapp.example.com + +# RFC 9116 security.txt / /security contact. Override this to your own +# disclosure address; the default points at the upstream project. +SECURITY_CONTACT_EMAIL=security@yourapp.example.com + # Optional: route heavy upload POSTs (convert/compress, single + batch) through # a separate subdomain. Empty string = same-origin (default, simplest). Set # only when the main site sits behind a proxy that caps request bodies and @@ -64,9 +77,13 @@ LANG_DEFAULT=de docker compose up -d ``` -This builds and runs the **slim** image (`filemorph:latest`, ~150 MB). -For Word documents with footnotes, headers, multi-section layout, or -table-of-contents, see the [office image variant](#image-variants) below. +This builds and runs the **slim** image locally (~150 MB). With no +`image:` key in `docker-compose.yml`, Compose names the built image +after your project directory (e.g. `filemorph-filemorph:latest` if you +cloned into `FileMorph/`) — run `docker compose images` if you need +the exact local tag. For Word documents with footnotes, headers, +multi-section layout, or table-of-contents, see the +[office image variant](#image-variants) below. ### 3. Generate API keys @@ -103,8 +120,9 @@ conversion stack. Use this image when your deployment: # docker-compose.yml — default, builds the slim image services: filemorph: - image: ghcr.io/mrchenglen/filemorph:latest - # or: build: { context: ., target: base } + build: { context: ., target: base } + # or, to skip the local build and pull the pre-built image instead: + # image: ghcr.io/mrchenglen/filemorph:latest ``` ### `filemorph:office` — high-fidelity DOCX → PDF @@ -139,6 +157,11 @@ in your `.env` (recommended in the office image when you never want the fallback — it makes a missing `soffice` fail loud instead of silently degrading). +`OFFICE_SUBPROCESS_TIMEOUT_SECONDS` (default `60`) bounds how long a +single `soffice --convert-to` call may run before it is killed — +raise it if you regularly convert long, complex Word documents on a +slower host. Ignored when `FILEMORPH_OFFICE_ENGINE=mammoth`. + ### Verifying signatures Both images are cosign-signed (keyless OIDC, no long-lived signing @@ -196,6 +219,61 @@ building anything: images are digest-addressed and signed with cosign. Pin the digest rather than a tag, and verify it as shown under [Verifying signatures](#verifying-signatures) above. +### Building without Compose + +`.dockerignore` keeps `.env*` files, the contents of `data/` (the local +API-key store) and other local-only files such as `.git` out of the build +context, so a self-built image doesn't bake those in. It is a deny-list, +though: anything else in the folder you build from — a private key, a +database dump — is still copied into the image (the runtime stage runs +`COPY . .`), so build from a clean checkout before you share an image. +Two consequences if you build and run without `docker compose`: + +- A plain `docker run` needs `--env-file .env` (or `-e` per variable) + and a volume for `./data:/app/data` — neither is baked into the + image the way it might have been from an image built before this + `.dockerignore` fix landed. +- If you previously built and distributed a custom image from an + older checkout, rotate any secrets that image might have baked in + (`.env` values, `data/api_keys.json`) — pulling the current + `Dockerfile` and rebuilding does not retroactively scrub an image + you already shared. + +## Cloud Edition overlay (accounts, billing, admin cockpit) + +The setup above is Community Edition: single container, no database, +no accounts. Layering `docker-compose.cloud.yml` on top adds a +Postgres service and switches the app into Cloud-Edition mode +(registration, JWT login, the admin cockpit, and — with `STRIPE_*` +set — billing): + +```bash +docker compose -f docker-compose.yml -f docker-compose.cloud.yml up -d +``` + +Two `.env` variables you must set explicitly before this goes anywhere +near production: + +- **`POSTGRES_PASSWORD`** — `docker-compose.cloud.yml` falls back to + the literal `changeme` when this is unset or empty. Set a strong + random value. +- **`JWT_SECRET`** — required for the account/login features. The + overlay has no fallback for it, and the app refuses to start with a + short or published value; see + [JWT secret (Cloud Edition)](#jwt-secret-cloud-edition) below. + +The overlay derives `DATABASE_URL` from `POSTGRES_PASSWORD` +automatically (`postgresql+asyncpg://filemorph:***@postgres/filemorph`); +you only need to set the password. To point at a database you run +yourself instead of the bundled container, edit the `DATABASE_URL` line +in the overlay (or in a copy of it) and remove both the `postgres` +service and the `depends_on` entry under `filemorph`. Setting +`DATABASE_URL` in `.env` does not work: the overlay's own +`environment:` value takes precedence over `env_file`. See +[`docs/installation.md`](installation.md) "Method 2" for the full +first-boot walkthrough (migrations, the legacy single-user key, what +`/register` and `/cockpit` need). + ## Reverse proxy (HTTPS) Place FileMorph behind a reverse proxy to handle SSL termination, domain routing, and request-body limits. @@ -378,11 +456,18 @@ If FileMorph should only be accessible within your organization (no public inter **Option A — Bind to internal IP only** -In `.env`: -```env -APP_HOST=192.168.1.50 # your server's internal IP +`APP_HOST` in `.env` has no effect under Docker (see the note under +"Edit `.env` for production" above) — bind the **host** side of the +port mapping in `docker-compose.yml` instead: + +```yaml +ports: + - "192.168.1.50:8000:8000" # your server's internal IP ``` +Running without Docker (`python run.py`), `APP_HOST` in `.env` does +apply directly. + **Option B — Use a firewall** ```bash @@ -397,7 +482,9 @@ sudo ufw deny 8000 # docker-compose.yml — no port exposed externally services: filemorph: - build: . + build: + context: . + target: base # omit this and Docker builds the last stage (office, ~280 MB larger) expose: - "8000" # accessible only within Docker network networks: @@ -431,7 +518,6 @@ route returns 404. |---|---|---|---| | `http_requests_total` | counter | `handler`, `method`, `status` | Throughput + error rate per route | | `http_request_duration_seconds` | histogram | `handler`, `method` | Latency percentiles (p50/p95/p99) | -| `http_request_size_bytes` / `http_response_size_bytes` | summary | `handler` | Upload / download volume | | `filemorph_conversions_total` | counter | `operation`, `src`, `tgt`, `status` | Per-format-pair conversion KPIs | The endpoint emits only aggregate counters and timings — never file @@ -474,7 +560,7 @@ scrape_configs: Grafana dashboards and alert rules (uptime, error-rate, p95 latency) are not bundled in this repo — wire your own against the metric families -above, or use the dashboards the Compliance Edition ships. +above. --- @@ -561,6 +647,22 @@ No restart required — keys are re-read on every request. 2. Update your application/service with the new key 3. Remove the old hash from `data/api_keys.json` +### Cloud Edition — dashboard keys + +Signed-in users manage their own keys from the dashboard instead of +the shared file, via `POST /api/v1/keys` (create, rate-limited +10/minute per account), `GET /api/v1/keys` (list) and `DELETE +/api/v1/keys/{key_id}` (revoke, 204). Each account can hold at most +**25 active keys**; `POST /api/v1/keys` past that cap returns `409` +with a message pointing at revoking an unused key first. A key is +shown once, at creation, and stored only as a SHA-256 hash — same +model as the Community Edition file. + +Any rejected `X-API-Key` — a wrong dashboard key or a wrong key from the +key file — normally answers `401`. Past **30 failed attempts per minute +per IP** it answers `429` with `Retry-After` instead — a valid key is +never affected by this budget, only guessing is throttled. + --- ## AI file operations (commercial add-on) @@ -607,6 +709,13 @@ Example with **uptime monitoring** (e.g. UptimeRobot, Gatus): - URL: `https://filemorph.example.com/api/v1/health` - Expected keyword: `"status":"ok"` +Both `/api/v1/health` and `/api/v1/ready` are rate-limited to 30 +requests/minute per IP, each counted separately — limits are set per +route by `@limiter.limit(...)` decorators, not globally. A monitor +polling more often than that will start seeing `429` instead of a real +health signal — keep the check interval at 5 seconds or slower (or +poll from more than one source IP). + ### Log access ```bash @@ -723,7 +832,7 @@ when the Cloud Edition is on (Postgres + SMTP configured): | `POST /api/v1/auth/register` | Sign up | Fires a verification email best-effort; SMTP failure does not block registration. | | `POST /api/v1/auth/verify-email` | Mark `users.email_verified_at` | Token bound to email-at-issuance (`eat` claim, 7-day TTL). Email rotation silently invalidates stale links. | | `POST /api/v1/auth/resend-verification` | New verify link | Auth-required (no spam vector). 200 no-op when already verified. | -| `DELETE /api/v1/auth/account` | Self-service delete | Three-field re-confirmation; last-active-admin guard returns 409; Stripe-touched accounts return 409 directing to your support contact. Confirmation email sent post-commit. | +| `DELETE /api/v1/auth/account` | Self-service delete | Three-field re-confirmation; last-active-admin guard returns 409. Free / never-paid accounts hard-delete; accounts that have touched Stripe cancel any active subscription first, then keep a restricted record (email, Stripe customer id, tier, created-at only) for the HGB §257 / AO §147 ten-year tax-retention window. Confirmation email sent post-commit. | All four endpoints write `auth.*` events to the audit-log hash chain. Outbound email uses the same `SMTP_*` configuration as @@ -735,6 +844,23 @@ identity. See [`docs/email-setup.md`](email-setup.md) for the SMTP walkthrough (provider options, port/TLS choice, sandbox-mode pitfalls, DSGVO sub-processor disclosure). +### Promoting an admin + +Phase 1 has no cockpit UI for promoting the first admin — registration +always creates a regular user. Run this on the server (or `docker +compose exec`) after registering the account that should have access +to `/cockpit`: + +```bash +docker compose exec filemorph python scripts/promote_admin.py you@example.com +``` + +It looks the user up by email and sets `role=admin`; running it again +on an already-promoted address is a no-op. This is also the recovery +path if the last admin was demoted — there is no other way back into +the cockpit. It only sets the role: it does not reactivate a +deactivated account. Requires `DATABASE_URL` (Cloud Edition). + ### Updating ```bash @@ -749,7 +875,9 @@ API keys in `./data/` are preserved across updates. ## systemd service (without Docker) -For running FileMorph directly as a Linux service: +For running FileMorph directly as a Linux service, behind the same +reverse proxy used above (Caddy/nginx on `127.0.0.1:8000` — see +"Reverse proxy (HTTPS)"): ```ini # /etc/systemd/system/filemorph.service @@ -762,11 +890,13 @@ After=network.target Type=simple User=filemorph WorkingDirectory=/opt/filemorph -Environment="PATH=/opt/filemorph/.venv/bin" -# Cloud Edition: DATABASE_URL and JWT_SECRET in a root-owned file, mode 600 -# (see "JWT secret (Cloud Edition)" above), never in an Environment= line. -# EnvironmentFile=/etc/filemorph/filemorph.env -ExecStart=/opt/filemorph/.venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 2 +Environment="PATH=/opt/filemorph/.venv/bin:/usr/bin" +# Variables the app reads only from the process environment (DATABASE_URL, +# FORWARDED_ALLOW_IPS, FILEMORPH_IMAGE_MAX_MEGAPIXELS) and, for the Cloud +# Edition, JWT_SECRET: in a root-owned file, mode 600 (see "JWT secret +# (Cloud Edition)" above), never in an Environment= line. +EnvironmentFile=-/etc/filemorph/filemorph.env +ExecStart=/opt/filemorph/.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000 Restart=on-failure RestartSec=5 @@ -774,6 +904,40 @@ RestartSec=5 WantedBy=multi-user.target ``` +The leading `-` in `EnvironmentFile=-` makes the file optional, so the +service also starts on a Community Edition install that has none. + +Three things this unit gets right that a naive copy of the Docker +setup would miss: + +- **`PATH` includes `/usr/bin`.** The venv's `bin/` only holds Python + entry points; `ffmpeg`, `gs` (Ghostscript) and `soffice` + (`filemorph:office`-equivalent installs) are system binaries under + `/usr/bin`. Without it on `PATH`, uvicorn starts fine but video/audio + conversion and the Ghostscript PDF/A re-render path silently fall back + or fail. +- **`EnvironmentFile=` puts these variables into the process environment.** + `DATABASE_URL`, `FORWARDED_ALLOW_IPS` and + `FILEMORPH_IMAGE_MAX_MEGAPIXELS` are read directly from the process + environment (`os.environ`), not through the application's own + `.env` parsing (that only covers the settings pydantic-settings + declares). Under Docker, `env_file:` in `docker-compose.yml` does + this for you; under systemd nothing does unless you add + `EnvironmentFile=` yourself — without it, a `.env` sitting in + `WorkingDirectory` is silently ignored for these three variables. Put + them in the same root-owned file as `JWT_SECRET`. +- **Bind `127.0.0.1`, not `0.0.0.0`.** Consistent with the reverse-proxy + guidance above: the proxy is the only thing that should be reachable + from outside, and binding the app to loopback makes that true at the + socket level instead of relying on a firewall rule. + +**Run one process** (the unit above has no `--workers` flag, which +defaults to 1). The rate limiter and the `/convert` + `/compress` +concurrency caps (`MAX_GLOBAL_CONCURRENCY` and friends, see "Capacity +tuning" above) are in-memory and per-process, so every additional +process gets its own set — `--workers N` and `N` separate `.service` +instances behind the proxy alike silently multiply those limits by `N`. + ```bash sudo systemctl daemon-reload sudo systemctl enable --now filemorph @@ -788,8 +952,12 @@ sudo systemctl status filemorph - [ ] Set `APP_DEBUG=false` in production - [ ] Cloud Edition: set `JWT_SECRET` to a generated value of at least 32 characters (see [JWT secret](#jwt-secret-cloud-edition)) - [ ] Keep `data/api_keys.json` out of version control (it is in `.gitignore`) -- [ ] Use HTTPS (see nginx + Certbot above) +- [ ] Use HTTPS (see Caddy or nginx above) +- [ ] Bind the published port to loopback (`ports: ["127.0.0.1:8000:8000"]`) once a reverse proxy is in front of it +- [ ] Set `FORWARDED_ALLOW_IPS` to your proxy's actual address, never `*` (see "HSTS behind Docker" above) - [ ] Set `MAX_UPLOAD_SIZE_MB` to a sensible limit for your use case - [ ] Restrict network access if the service is internal-only +- [ ] Restrict `/api/v1/metrics` at the reverse proxy — it is unauthenticated while `METRICS_ENABLED` is on, which is the default (see [Monitoring & metrics](#monitoring--metrics)) - [ ] Rotate API keys regularly +- [ ] Cloud Edition: set a strong `POSTGRES_PASSWORD` (the overlay falls back to `changeme`) - [ ] Monitor disk usage (temp files are cleaned up, but check `/tmp` if issues occur) diff --git a/docs/third-party-licenses.md b/docs/third-party-licenses.md index d3d7678..839de5d 100644 --- a/docs/third-party-licenses.md +++ b/docs/third-party-licenses.md @@ -27,11 +27,14 @@ Apache-2.0, ISC, Unlicense, PSF, MIT-CMU) or weak/file-level copyleft (MPL-2.0) — none is GPL/AGPL strong-copyleft *at the Python level*, so embedding the dependency tree in a closed-source product is unconstrained beyond preserving notices. The copyleft that exists lives in the **native layer** (the FFmpeg -binary, the HEVC libraries) and is reached only across a process boundary -(FFmpeg is invoked as a separate program) or a wrapper boundary (`libheif` via -`pillow-heif`), neither of which makes FileMorph a derivative work. Two items -warrant attention from anyone redistributing the artifact — `pillow-heif`'s -wheel metadata and the FFmpeg build in the Docker image — both detailed below. +and Ghostscript binaries, the HEVC libraries and — in the `office` image — +LibreOffice) and is reached only across a process boundary (FFmpeg, +Ghostscript and LibreOffice are invoked as separate programs) or a wrapper +boundary (`libheif` via `pillow-heif`), neither of which makes FileMorph a +derivative work. Four items warrant attention from anyone redistributing the +artifact — `pillow-heif`'s wheel metadata, the GPL FFmpeg build and the +AGPL-3.0 Ghostscript in the Docker image, and the MPL-2.0 LibreOffice in the +`office` image — all detailed below. ## FileMorph's own code @@ -49,7 +52,7 @@ GitHub release; feed that to your scanner. The summary by licence class: | Licence class | Examples | Implication | |---|---|---| -| **Permissive** — MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0, ISC, Unlicense/CC0, PSF-2.0, MIT-CMU (Pillow's HPND) | FastAPI/Starlette/Pydantic, Uvicorn, Jinja2, Pillow, pypdf, reportlab, WeasyPrint, Markdown, openpyxl, python-docx, mammoth, ffmpeg-python, SQLAlchemy/Alembic, asyncpg, python-jose, bcrypt, cryptography, stripe, Babel, slowapi, lxml, requests/httpx, … (the large majority) | No copyleft. Bundle, modify, redistribute closed-source freely; keep the copyright/notice text (each wheel ships its `LICENSE` file — those, plus the SBOM, are your notice manifest). | +| **Permissive** — MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0, ISC, Unlicense/CC0, PSF-2.0, MIT-CMU (Pillow's HPND) | FastAPI/Starlette/Pydantic, Uvicorn, Jinja2, Pillow, pypdf, reportlab, WeasyPrint, Markdown, openpyxl, python-docx, mammoth, ffmpeg-python, SQLAlchemy/Alembic, asyncpg, python-jose, bcrypt, cryptography, stripe, Babel, slowapi, lxml, pillow-avif-plugin, requests (transitive, via `stripe`), … (the large majority) | No copyleft. Bundle, modify, redistribute closed-source freely; keep the copyright/notice text (each wheel ships its `LICENSE` file — those, plus the SBOM, are your notice manifest). | | **Weak / file-level copyleft** — MPL-2.0 | `pikepdf` (PDF/A-2b output) — its wheels also bundle **qpdf**, which is Apache-2.0; `certifi` (CA bundle, transitive) | OK in a proprietary product: you must make the source of *the MPL-2.0 files* available (these are shipped unmodified, so pointing at the upstream sdist suffices) and you can't sublicense those files under other terms; the rest of your product is unaffected. | | **Tri-licensed, pick-one** — GPLv2+ / LGPLv2+ / MPL-1.1 | `pyphen` (hyphenation, transitive via WeasyPrint) | Choose the LGPLv2+ or MPL-1.1 arm; not a constraint. | | **Flagged for automated scanners** — wheel metadata declares GPLv2 | `pillow-heif` (HEIC input) | See the dedicated note below — an automated `pip-licenses`/SBOM scan **will** surface this; the explanation and mitigations matter. | @@ -73,6 +76,31 @@ system `libheif` built without `x265` (decode-only), at the cost of any future HEVC-encode capability — or drop HEIC input entirely. Either is a build-time choice with no code changes; raise it in the pilot conversation if it applies. +### `pillow-avif-plugin` — the bundled AV1 codec stack + +FileMorph uses `pillow-avif-plugin` for AVIF input **and** output — unlike +`pillow-heif`, whose bundled HEVC encoder FileMorph never calls, the plugin's +AV1 encoder is used. PyPI's package metadata classifies it **MIT**; the +`LICENSE` file the wheel ships +is worded as a standard BSD-2-Clause notice (same permissive family, +functionally interchangeable — the classifier is what an automated scanner +reports, so it's the label used here). That same file bundles the licences +of the native libraries the wheel embeds: + +- **libavif** (the AVIF container/codec wrapper) — BSD-2-Clause. +- **dav1d** (AV1 decoder; the `obu.c` file specifically) — BSD-2-Clause, + copyright VideoLAN and dav1d authors. + +libavif can additionally build against **aom** (the AV1 reference +encoder/decoder, BSD-2-Clause plus the Alliance for Open Media patent +licence), **rav1e** (BSD-2-Clause) and **SVT-AV1** (BSD-3-Clause-Clear) as +alternative codec backends — all permissive, none copyleft — but the +installed wheel's bundled licence file only itemises libavif and dav1d by +name, so treat the other three as "present if your platform's wheel links +them" rather than independently confirmed here. Verify against the wheel +you actually ship with `pip show pillow-avif-plugin` and the `LICENSE` +file in its `dist-info`. + ## Native / system libraries in the Docker image The image (`python:3.14-slim` base) adds, via `apt`, the native pieces the @@ -81,9 +109,21 @@ converters need: | Component | Licence | How FileMorph reaches it | Implication | |---|---|---|---| | **FFmpeg** (Debian package) | Debian builds FFmpeg with `--enable-gpl` (x264, x265, …) → effectively **GPL-2.0+** (GPL-3.0+ for `--enable-version3` parts) | Invoked as a **separate program** via `ffmpeg-python` subprocess calls — never linked into the FileMorph process | Calling a separate GPL program does not make the caller a derivative work, so **FileMorph's own licence is unaffected**. The *Docker image*, as a bundle, does contain GPL software — a redistributor of the image carries the GPL source-availability obligation for the FFmpeg component (Debian's source archive satisfies it). A "no GPL anywhere in the deployed artifact" requirement needs a custom image with an LGPL-only FFmpeg build (`--disable-gpl`, reduced codec set) — available on request. | -| **libheif** (`libheif1`) + HEVC backend | `libheif` LGPL-3.0; `libde265` (decode) LGPL-3.0; `x265` (encode) GPL-2.0+ | Used via the `pillow-heif` wheel's bundled copy for HEIC decode; the runtime image installs `libheif1` (no headers / no dev package — those live in the throwaway builder stage of the multi-stage Dockerfile, see P3-8) | Same as the `pillow-heif` note above — decode path is LGPL; the GPL encoder is present in the bundled native stack but unused. Check `dpkg -l \| grep -E 'libheif\|libde265\|x265'` against a running container. | +| **libheif** (`libheif1`) + HEVC backend | `libheif` LGPLv3; `libde265` (decode) LGPLv3; `x265` (encode) GPLv2; `libaom` (AV1, bundled alongside) BSD-2-Clause plus the Alliance for Open Media patent licence (the wheel's own `LICENSES_bundled.txt` lists it as "BSD 3-Clause" but links libaom's own `LICENSE`, which is the 2-clause text) | Used via the `pillow-heif` wheel's bundled copy for HEIC decode; the runtime image installs `libheif1` (no headers / no dev package — those live in the throwaway builder stage of the multi-stage Dockerfile, see P3-8) | Same as the `pillow-heif` note above — decode path is LGPL; the GPL encoder is present in the bundled native stack but unused. Check `dpkg -l \| grep -E 'libheif\|libde265\|x265'` against a running container. | | **qpdf** | Apache-2.0 | Bundled inside the `pikepdf` wheel (no system package) | Permissive — no obligation beyond notice. | | **cairo / pango** (WeasyPrint rendering) | LGPL-2.1 | Dynamically linked as system shared libraries through WeasyPrint | LGPL via dynamic linking against unmodified system libraries is the standard, unproblematic case for proprietary use (the obligation is to allow relinking, which dynamic linking already does). | +| **Ghostscript** (`ghostscript` apt package) | AGPL-3.0 (Artifex's default public distribution; a commercial licence is also sold by Artifex) | Invoked as a **separate program** (`gswin64c`/`gs` subprocess) for the PDF/A-2b re-render path — never linked into the FileMorph process. Present in **both** the slim and office images (the office stage builds `FROM base`, which already installs it). | Same separate-program reasoning as FFmpeg above: driving an AGPL binary as a subprocess does not make the caller a derivative work, so FileMorph's own licence is unaffected. The *image*, as a bundle, does contain AGPL software — a redistributor carries the AGPL source-availability obligation for that component (Debian's source archive satisfies it). AGPL §13 (the network-use clause) is triggered by *modifying* the program: the image installs Debian's `ghostscript` package as shipped, and FileMorph runs that unmodified binary as a separate program, so §13 does not reach FileMorph. | +| **LibreOffice** (`libreoffice-core`, `libreoffice-writer`) | MPL-2.0 (The Document Foundation; some bundled components are LGPLv3+) | **Office image only** — invoked as a **separate program** (`soffice --headless --convert-to`) for the high-fidelity DOCX → PDF path; not present in the slim image at all. | MPL-2.0 is weak/file-level copyleft, same class as `pikepdf` above — and the subprocess boundary means it doesn't reach into FileMorph's own licensing regardless. A redistributor of the `office` image carries the same source-availability obligation as any bundled MPL-2.0 component (upstream source is public). | +| **Fonts** (`fonts-crosextra-carlito`, `fonts-liberation`, `fonts-dejavu-core`) | Each Debian package under its own font licence — the terms are in `/usr/share/doc//copyright` inside the image | **Office image only** — system fonts installed next to LibreOffice so Word documents render with the metrics they were written for (Carlito for Calibri, Liberation for Arial / Times / Courier, DejaVu as a broad-coverage fallback). | Data files, not linked code. A redistributor of the `office` image passes their licence notices on with the packages. | + +## Vendored frontend assets + +Two non-Python assets ship in `app/static/`, both permissive: + +| Component | Licence | Notes | +|---|---|---| +| **Chart.js** `v4.4.0` (`app/static/vendor/chart.umd.min.js`) | MIT | Vendored unmodified from the upstream `dist/chart.umd.min.js` build; full upstream licence text, source URL and a SHA-256 of the vendored copy live in [`app/static/vendor/LICENSE-chartjs.md`](../app/static/vendor/LICENSE-chartjs.md). Used read-only by the admin cockpit's dashboard charts. | +| **Tailwind CSS** (`app/static/css/tailwind..css`) | MIT (Tailwind Labs) | Self-hosted, not loaded from `cdn.tailwindcss.com`: `scripts/build-tailwind.sh` runs the official standalone Tailwind CLI against `tailwind.config.js` and the app's own templates, producing a purged, minified, content-hashed bundle that is committed under `app/static/css/` and ships from the deployment's own origin; CI only checks that the committed bundle is up to date. See [`docs/tailwind-build-setup.md`](./tailwind-build-setup.md). | ## What this means for the Compliance Edition / commercial licence @@ -95,19 +135,23 @@ converters need: copyleft, already allow embedding in a closed-source product. The licence buyer's only standing obligation toward them is to **preserve their notices** (the per-wheel `LICENSE` files plus the release SBOM are the manifest). -- Two GPL components are present in the default artifact but do not affect - FileMorph's licensing or normal operation: the `pillow-heif` wheel bundles a - GPL-2.0+ HEVC encoder (`x265`) that is **never invoked** (HEIC is decode-only), - and the Docker image bundles Debian's GPL FFmpeg, which FileMorph drives as a - **separate program** (subprocess), not a linked library. **The project's - position:** the default build keeps both — removing them would mean dropping - HEIC input and H.264/H.265 encoding, a real product regression, to chase a - paperwork concern that the separate-program boundary and the never-invoked - status already resolve. The GPL-free builds (LGPL-only FFmpeg; `pillow-heif` - rebuilt `--no-binary` against an `x265`-free `libheif`) are offered **per - Compliance agreement** for deployments with a hard zero-GPL-in-the-artifact - requirement; they are not the default because they degrade the product for - everyone else. +- Four copyleft components are present in the default artifact but do not + affect FileMorph's licensing or normal operation. Two are GPL: the + `pillow-heif` wheel bundles a GPLv2 HEVC encoder (`x265`) that is **never + invoked** (HEIC is decode-only), and the Docker image bundles Debian's GPL + FFmpeg, which FileMorph drives as a **separate program** (subprocess), not a + linked library. The third is Ghostscript (AGPL-3.0, in both images), likewise + a separate program, used unmodified as Debian ships it — which is why AGPL + §13 (network use) does not reach FileMorph. The fourth is LibreOffice + (MPL-2.0, weak copyleft, `office` image only), also a separate program. + **The project's position:** the default build keeps the GPL components — + removing them would mean dropping HEIC input and H.264 encoding, a real + product regression, to chase a paperwork concern that the separate-program + boundary and the never-invoked status already resolve. The GPL-free builds + (LGPL-only FFmpeg; `pillow-heif` rebuilt `--no-binary` against an + `x265`-free `libheif`) are offered **per Compliance agreement** for + deployments with a hard zero-GPL-in-the-artifact requirement; they are not + the default because they degrade the product for everyone else. - No dependency forces FileMorph to drop the dual-license offering, and none did at any point in the project's history; the [`License Map`](./tech-stack-rationale.md#license-map) is updated in the same PR as any new dependency precisely to keep that true. @@ -128,12 +172,20 @@ converters need: the image ships, so the scan matches the artifact). - **Per-dependency rationale:** [`tech-stack-rationale.md` § License Map](./tech-stack-rationale.md#license-map). -**As of 2026-05-12** a scan of the locked dependency set +**As of 2026-09-28** a manual scan of the locked dependency set on `main` (`requirements.lock`) yields the distribution summarised above: the runtime tree is permissive or MPL-2.0 throughout, with `pillow-heif` the single -GPLv2-declared wheel (HEVC encoder bundled, unused). Re-run the SBOM and this -scan on every release; flag any new copyleft entry in the `License Map` and -here. +GPLv2-declared wheel (HEVC encoder bundled, unused), plus `pillow-avif-plugin` +(MIT / BSD-2-Clause-style) documented separately above. + +**The CycloneDX SBOM attached to the `v1.1.0` GitHub Release predates this** +— it was generated before the lockfile-parity (#146) and SBOM-generation +hardening (#163) changes, so it does not reflect `main`'s current dependency +set or the current SBOM pipeline. Until the next tagged release publishes a +fresh one, regenerate the SBOM locally against the current `requirements.lock` +(see "How to verify" above) rather than treating the `v1.1.0` attachment as +current. Re-run the SBOM and this scan on every release; flag any new +copyleft entry in the `License Map` and here. ## See also diff --git a/docs/threat-model.md b/docs/threat-model.md index 8948959..4358041 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -35,12 +35,12 @@ Out of scope: |---|---|---|---|---| | **S — Spoofing** | API caller pretends to hold a key they do not possess. | SHA-256-hashed key stores; the key file is compared in constant time via `hmac.compare_digest`, per-user keys are looked up by hash (revoked keys rejected). | `app/core/security.py::validate_api_key`, `::find_active_api_key` | None known. A lookup-timing leak could reveal at most a stored hash, useless without a preimage of a 256-bit random key. | | **S — Spoofing** | Login as another user via stolen / replayed credential. | bcrypt password hash with adaptive cost; short-lived 15 min JWT access token + 30 day refresh token. | `app/core/auth.py::hash_password`, `create_access_token` | A leaked refresh token grants 30 days of access — operators should rotate `JWT_SECRET` on suspicion (invalidates all sessions). | -| **S — Spoofing** | Rate-limit bypass via spoofed `X-Forwarded-For`. | The application reads `X-Forwarded-For`; the reverse proxy must be configured to overwrite (not append) the header. | [`self-hosting.md`](./self-hosting.md) §trust-proxy, PT-006 anchor in security-overview | Self-hoster misconfiguration. CI-level guard not possible — operational hardening item. | +| **S — Spoofing** | Rate-limit bypass via spoofed `X-Forwarded-For`. | The rate limiter keys on `request.client.host` (slowapi's `get_remote_address`), never the header directly; uvicorn only substitutes that value from `X-Forwarded-For` when the peer is covered by `FORWARDED_ALLOW_IPS` (default `127.0.0.1`). Setting it to `*` reopens the spoof. | [`self-hosting.md` § HSTS behind Docker](./self-hosting.md#hsts-behind-docker), PT-006 anchor in security-overview | Self-hoster misconfiguration. CI-level guard not possible — operational hardening item. | | **T — Tampering** | Malicious upload (PE, ELF, shell, PHP) reaches a converter. | Magic-byte deny-list rejects the request before any decoder runs; HTTP 400 returned. | `app/core/processing.py` (`BLOCKED_MAGIC`), enforced in `app/api/routes/convert.py` + `compress.py` | None known for the listed prefixes. | | **T — Tampering** | Path traversal via filename injection. | Filenames never used as filesystem paths; UUID stems under a `fm_`-prefixed scratch dir. | `app/api/routes/convert.py` (temp-dir handling), PT-001 anchor | None known. | | **T — Tampering** | Header injection via filename in `Content-Disposition`. | `safe_download_name()` strips ASCII-unsafe bytes and RFC 5987-encodes UTF-8. | `app/core/utils.py::safe_download_name`, PT-013 anchor | None known. | | **T — Tampering** | SSRF via WeasyPrint URL fetching. | `url_fetcher=_deny_url_fetcher` denies every external fetch in WeasyPrint. | `app/converters/document.py::_deny_url_fetcher`, PT-008 anchor | Operator must not patch this out — documented as a do-not-disable hardening item. | -| **R — Repudiation** | User denies having performed a billable conversion. | Structured JSON logs record `operation`, `src_format`, `tgt_format`, `file_size_bytes`, `duration_ms`, `success` per request. | `app/core/logging_config.py`, structured-event schema in CLAUDE.md §Business-Metriken | Tamper-evident SHA-256 hash-chain audit log implemented (`app/core/audit.py`): each `record_hash` chains the previous, so altering an old row breaks every following one. | +| **R — Repudiation** | User denies having performed a billable conversion. | Structured JSON logs record `operation`, `src_format`, `tgt_format`, `file_size_bytes`, `duration_ms`, `success` per request. | `app/core/logging_config.py` | Tamper-evident SHA-256 hash-chain audit log implemented (`app/core/audit.py`): each `record_hash` chains the previous, so altering an old row breaks every following one. | | **R — Repudiation** | Admin denies having performed a privileged action. | Cockpit actions (tier changes, role changes, deactivations) are logged with admin user-id, target user-id, and the change. | `app/api/routes/cockpit.py` | Cockpit actions land in the same SHA-256 hash-chained audit log (`app/core/audit.py`). | | **I — Info disclosure** | Internal exception details leak via API error response. | Global error handler returns a generic message; stack traces stay in server logs only. | `app/main.py::server_error_handler`, PT-004 anchor | None known. | | **I — Info disclosure** | File contents written to disk under attacker-controlled name. | Files decoded into `BytesIO`; UUID-stem temp paths if disk is needed; `finally`-block cleanup; startup sweep for stale `fm_*` dirs >10 min. | `app/api/routes/convert.py`, `app/main.py::lifespan` | Temp dir contents during the converter run could be read by a co-tenant on a multi-tenant host — out of scope (one-instance-per-tenant pattern). | @@ -49,10 +49,10 @@ Out of scope: | **D — Denial of service** | Single client floods convert endpoint. | slowapi rate limit (10/min/IP) on `/convert` and `/compress`. | `app/core/rate_limit.py` | Multi-instance deployments split the bucket per worker — Redis backend is on backlog. | | **D — Denial of service** | Decompression-bomb upload. | Pillow `MAX_IMAGE_PIXELS` enforced; per-tier output cap rejects oversized output post-conversion. | `app/core/quotas.py` | Per-batch aggregate cap not yet implemented — listed under *Known Limitations*. | | **D — Denial of service** | Slow / oversized request body exhausts memory. | `Content-Length`-based pre-read rejection at `MAX_UPLOAD_SIZE_MB`. Operator-level cap at the proxy is recommended. | `app/main.py::limit_upload_size` | Slow-loris-style streaming attacks are the proxy's job. | -| **D — Denial of service** | Sync C-binding blocks event loop. | All synchronous binding calls (Pillow saves, WeasyPrint, pikepdf, ffmpeg) wrap in `asyncio.to_thread`. | `app/converters/*.py` | Pinned by lint discipline ("Event-Loop sauber halten" in CLAUDE.md). | +| **D — Denial of service** | Sync C-binding blocks event loop. | All synchronous binding calls (Pillow saves, WeasyPrint, pikepdf, ffmpeg) wrap in `asyncio.to_thread`. | `app/converters/*.py` | Pinned by code-review discipline — a synchronous C-binding call added outside `asyncio.to_thread` blocks every concurrent request on that worker. | | **E — Elevation of privilege** | Stale JWT continues to grant admin after demotion. | Admin role rechecked against the `users` table on every cockpit request — token cannot escalate after a role change. | `app/api/routes/cockpit.py` | None known. | | **E — Elevation of privilege** | Plugin loaded from untrusted source executes during conversion. | Converter plugins are loaded only from `app/converters/` (filesystem-bound), not from request input. | `app/converters/registry.py::_ensure_loaded` | Any third-party plugin pack a self-hoster installs runs with full app privileges — vet plugins before installing. | -| **E — Elevation of privilege** | Container escape via converter sub-process. | Converters run inside the application container; no container-escape primitives are exposed to user input. | Containerised deployment | OS-level isolation is the operator's responsibility — see compose-prod hardening (read-only rootfs, `cap_drop`, no-new-privileges) in the operations runbook. | +| **E — Elevation of privilege** | Container escape via converter sub-process. | Converters run inside the application container; no container-escape primitives are exposed to user input. | Containerised deployment | OS-level isolation is the operator's responsibility — the shipped [`docker-compose.yml`](../docker-compose.yml) already sets `no-new-privileges` and drops every Linux capability (`cap_drop: [ALL]`); `read_only: true` (commented out, needs a `/tmp` tmpfs) is the stronger option for operators who want it. | ## Cross-references @@ -69,6 +69,9 @@ Out of scope: ## Review cadence This model is reviewed on each significant change to the request path, -the auth surface, or the converter plugin set. The review-trigger items -are flagged in CLAUDE.md under "Network-layer changes quadruple-check" -and "Propagation-Guard an Auth-/Quota-Boundaries". +the auth surface, or the converter plugin set. A new cross-origin +endpoint or upload route goes through the "quadruple-check" in the +[pull-request template](../.github/PULL_REQUEST_TEMPLATE.md) (CSP +`connect-src`/`script-src`, `CORS_ORIGINS`, `expose_headers`, +`.env.example` + self-hosting docs); changes to the auth or quota +boundaries get the same level of review. From 06a72761179b8440822d1c2d6f4b1531fc0ee22f Mon Sep 17 00:00:00 2001 From: MrChengLen Date: Tue, 29 Sep 2026 09:52:57 +0200 Subject: [PATCH 2/5] =?UTF-8?q?chore(changelog):=20take=20the=20entry=20ou?= =?UTF-8?q?t=20to=20merge=20main=20in=20=E2=80=94=20back=20in=20two=20comm?= =?UTF-8?q?its?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every PR adds its CHANGELOG entry at the same place, so each merge to main conflicts with every open PR. Taking this PR's entry out lets GitHub merge main in cleanly; the next commit puts it back on top. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fad6285..7998f19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,33 +9,6 @@ Versions follow [Semantic Versioning](https://semver.org/). ## [Unreleased] -### Fixed — self-hosting and security docs: systemd unit, licences and release facts - -The self-hosting, installation, development, security and licensing docs had -drifted from the code they describe. The systemd unit was missing `/usr/bin` -on `PATH` (ffmpeg/Ghostscript/LibreOffice invisible to a non-Docker install), -never loaded `.env` into the process environment (`DATABASE_URL`, -`FORWARDED_ALLOW_IPS` and `FILEMORPH_IMAGE_MAX_MEGAPIXELS` are read directly -from `os.environ`, not through the app's own `.env` parsing), bound -`0.0.0.0` instead of `127.0.0.1`, and ran multiple workers despite the rate -limiter and concurrency caps being per-process. Account deletion, the Stripe -webhook table and the audit-log hash paragraph described a pre-launch state -(Stripe-touched accounts refused with 409) instead of the shipped -tax-retained deletion path. `third-party-licenses.md` was missing -Ghostscript, LibreOffice, `pillow-avif-plugin` and the vendored Chart.js / -Tailwind assets, still listed `httpx` as a runtime dependency (it's dev-only), -and pointed at a scan/SBOM that predates the lockfile-parity and -SBOM-hardening work. `patch-policy.md` claimed every merge is tagged and a -release lands every 1–4 weeks; it now describes the actual two-track model -(continuous `latest`/`sha-*` builds on every merge, a `vX.Y.Z`/`X.Y.Z` tag at -the maintainer's discretion — one so far, `v1.1.0`). Smaller fixes: -`/ready` doesn't check ffmpeg (DB + tempdir only), `APP_PORT` is read only by -`run.py`, `python3.11` is no longer an installable apt package on current -Ubuntu/Debian, a Docker UID/GID mismatch on `./data` needs a `chown` not a -`chmod`, the email-setup guide's "no SMTP" behaviour is `200` + a log line -(not `503`), and the pentest report's Tailwind-CDN / inline-script claims -are corrected in place rather than silently rewritten. - ### Fixed — website texts match what the service does The public pages were checked claim by claim against the code: From 67f2532fce313db2e304920e220aaa1909892d7e Mon Sep 17 00:00:00 2001 From: MrChengLen Date: Tue, 29 Sep 2026 09:54:04 +0200 Subject: [PATCH 3/5] docs(changelog): entry back on top of the merged changelog Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 32dd320..8534824 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,33 @@ Versions follow [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Fixed — self-hosting and security docs: systemd unit, licences and release facts + +The self-hosting, installation, development, security and licensing docs had +drifted from the code they describe. The systemd unit was missing `/usr/bin` +on `PATH` (ffmpeg/Ghostscript/LibreOffice invisible to a non-Docker install), +never loaded `.env` into the process environment (`DATABASE_URL`, +`FORWARDED_ALLOW_IPS` and `FILEMORPH_IMAGE_MAX_MEGAPIXELS` are read directly +from `os.environ`, not through the app's own `.env` parsing), bound +`0.0.0.0` instead of `127.0.0.1`, and ran multiple workers despite the rate +limiter and concurrency caps being per-process. Account deletion, the Stripe +webhook table and the audit-log hash paragraph described a pre-launch state +(Stripe-touched accounts refused with 409) instead of the shipped +tax-retained deletion path. `third-party-licenses.md` was missing +Ghostscript, LibreOffice, `pillow-avif-plugin` and the vendored Chart.js / +Tailwind assets, still listed `httpx` as a runtime dependency (it's dev-only), +and pointed at a scan/SBOM that predates the lockfile-parity and +SBOM-hardening work. `patch-policy.md` claimed every merge is tagged and a +release lands every 1–4 weeks; it now describes the actual two-track model +(continuous `latest`/`sha-*` builds on every merge, a `vX.Y.Z`/`X.Y.Z` tag at +the maintainer's discretion — one so far, `v1.1.0`). Smaller fixes: +`/ready` doesn't check ffmpeg (DB + tempdir only), `APP_PORT` is read only by +`run.py`, `python3.11` is no longer an installable apt package on current +Ubuntu/Debian, a Docker UID/GID mismatch on `./data` needs a `chown` not a +`chmod`, the email-setup guide's "no SMTP" behaviour is `200` + a log line +(not `503`), and the pentest report's Tailwind-CDN / inline-script claims +are corrected in place rather than silently rewritten. + ### Fixed — compliance templates describe what the code does The DPA template and its TOM annex, the records-of-processing template, the From 0645912aa23dd27a30ab55b7bc29cbb2211f3217 Mon Sep 17 00:00:00 2001 From: MrChengLen Date: Tue, 29 Sep 2026 10:00:07 +0200 Subject: [PATCH 4/5] =?UTF-8?q?chore(changelog):=20take=20the=20entry=20ou?= =?UTF-8?q?t=20to=20merge=20main=20in=20=E2=80=94=20back=20in=20two=20comm?= =?UTF-8?q?its?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every PR adds its CHANGELOG entry at the same place, so each merge to main conflicts with every open PR. Taking this PR's entry out lets GitHub merge main in cleanly; the next commit puts it back on top. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8534824..32dd320 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,33 +9,6 @@ Versions follow [Semantic Versioning](https://semver.org/). ## [Unreleased] -### Fixed — self-hosting and security docs: systemd unit, licences and release facts - -The self-hosting, installation, development, security and licensing docs had -drifted from the code they describe. The systemd unit was missing `/usr/bin` -on `PATH` (ffmpeg/Ghostscript/LibreOffice invisible to a non-Docker install), -never loaded `.env` into the process environment (`DATABASE_URL`, -`FORWARDED_ALLOW_IPS` and `FILEMORPH_IMAGE_MAX_MEGAPIXELS` are read directly -from `os.environ`, not through the app's own `.env` parsing), bound -`0.0.0.0` instead of `127.0.0.1`, and ran multiple workers despite the rate -limiter and concurrency caps being per-process. Account deletion, the Stripe -webhook table and the audit-log hash paragraph described a pre-launch state -(Stripe-touched accounts refused with 409) instead of the shipped -tax-retained deletion path. `third-party-licenses.md` was missing -Ghostscript, LibreOffice, `pillow-avif-plugin` and the vendored Chart.js / -Tailwind assets, still listed `httpx` as a runtime dependency (it's dev-only), -and pointed at a scan/SBOM that predates the lockfile-parity and -SBOM-hardening work. `patch-policy.md` claimed every merge is tagged and a -release lands every 1–4 weeks; it now describes the actual two-track model -(continuous `latest`/`sha-*` builds on every merge, a `vX.Y.Z`/`X.Y.Z` tag at -the maintainer's discretion — one so far, `v1.1.0`). Smaller fixes: -`/ready` doesn't check ffmpeg (DB + tempdir only), `APP_PORT` is read only by -`run.py`, `python3.11` is no longer an installable apt package on current -Ubuntu/Debian, a Docker UID/GID mismatch on `./data` needs a `chown` not a -`chmod`, the email-setup guide's "no SMTP" behaviour is `200` + a log line -(not `503`), and the pentest report's Tailwind-CDN / inline-script claims -are corrected in place rather than silently rewritten. - ### Fixed — compliance templates describe what the code does The DPA template and its TOM annex, the records-of-processing template, the From f463593ffc42d0e68d436e487eac437ceb57ecd8 Mon Sep 17 00:00:00 2001 From: MrChengLen Date: Tue, 29 Sep 2026 10:00:17 +0200 Subject: [PATCH 5/5] docs(changelog): entry back on top of the merged changelog Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index d8a53b5..c787024 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,33 @@ Versions follow [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Fixed — self-hosting and security docs: systemd unit, licences and release facts + +The self-hosting, installation, development, security and licensing docs had +drifted from the code they describe. The systemd unit was missing `/usr/bin` +on `PATH` (ffmpeg/Ghostscript/LibreOffice invisible to a non-Docker install), +never loaded `.env` into the process environment (`DATABASE_URL`, +`FORWARDED_ALLOW_IPS` and `FILEMORPH_IMAGE_MAX_MEGAPIXELS` are read directly +from `os.environ`, not through the app's own `.env` parsing), bound +`0.0.0.0` instead of `127.0.0.1`, and ran multiple workers despite the rate +limiter and concurrency caps being per-process. Account deletion, the Stripe +webhook table and the audit-log hash paragraph described a pre-launch state +(Stripe-touched accounts refused with 409) instead of the shipped +tax-retained deletion path. `third-party-licenses.md` was missing +Ghostscript, LibreOffice, `pillow-avif-plugin` and the vendored Chart.js / +Tailwind assets, still listed `httpx` as a runtime dependency (it's dev-only), +and pointed at a scan/SBOM that predates the lockfile-parity and +SBOM-hardening work. `patch-policy.md` claimed every merge is tagged and a +release lands every 1–4 weeks; it now describes the actual two-track model +(continuous `latest`/`sha-*` builds on every merge, a `vX.Y.Z`/`X.Y.Z` tag at +the maintainer's discretion — one so far, `v1.1.0`). Smaller fixes: +`/ready` doesn't check ffmpeg (DB + tempdir only), `APP_PORT` is read only by +`run.py`, `python3.11` is no longer an installable apt package on current +Ubuntu/Debian, a Docker UID/GID mismatch on `./data` needs a `chown` not a +`chmod`, the email-setup guide's "no SMTP" behaviour is `200` + a log line +(not `503`), and the pentest report's Tailwind-CDN / inline-script claims +are corrected in place rather than silently rewritten. + ### Added — the Docker images carry a signed SBOM attestation `docker.yml` now attests the CycloneDX SBOM to each image it pushes — slim and