Skip to content
CI

Merge pull request #183 from MrChengLen/pr-token-check #507

Sign in to view logs

Merge pull request #183 from MrChengLen/pr-token-check

Merge pull request #183 from MrChengLen/pr-token-check #507

Workflow file for this run

.github/workflows/ci.yml at b2deb14
name: CI
on:
push:
branches: ["main", "develop"]
pull_request:
branches: ["main"]
# A newer push on the same ref cancels older queued / running runs.
# Saves Actions quota and reduces queue pressure when iterating on a PR.
# main / develop are excluded by ref-pattern so post-merge gates always
# get to complete.
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/develop' }}
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
# Least-privilege default for GITHUB_TOKEN (OpenSSF Scorecard Token-Permissions).
# Both jobs only read the checkout; no job here writes back to the repo.
permissions:
contents: read
jobs:
lint-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
# Must match the Dockerfile's base image — the app is only ever
# tested on the version it ships on. scripts/check_python_version.py
# gates this; see its docstring for the drift it exists to prevent.
python-version: "3.14"
cache: "pip"
- name: Install system dependencies
run: |
# The runner image ships an apt source for Google Chrome, which this
# project does not use. Its index is intermittently inconsistent and
# then fails `apt-get update` with a hash-sum mismatch (exit 100),
# taking the whole job down with it — it blocked every PR on
# 2026-09-09. Dropping the source first makes the update depend only
# on repositories we actually install from.
#
# Matched by name rather than by a fixed filename: the runner images
# have moved between `google-chrome.list` and the deb822
# `google-chrome.sources`, and a stale `rm -f` of the wrong one fails
# silently — which is exactly how the first attempt at this slipped
# through and left the job failing identically.
sudo find /etc/apt/sources.list.d -iname '*google*' -delete
sudo apt-get update
sudo apt-get install -y ffmpeg ghostscript libheif-dev libcairo2 libpangocairo-1.0-0 libgdk-pixbuf2.0-0
# Tests run against the versions the image ships. requirements-dev.txt
# pulls in requirements.txt, whose `>=` ranges resolve to the newest
# releases: SQLAlchemy 2.1.0 (2026-09-24) reached CI that way and failed a
# test on every branch, while the image stayed on 2.0.52. The lockfile —
# what the Dockerfile installs — therefore constrains every runtime
# package. Dev-only tools (pytest, ruff, uv, aiosqlite, …) are not in it
# and resolve as before; a dependency they share with the app (packaging,
# requests, …) stays at its locked version. The unpinned install runs
# weekly in deps-latest.yml as an early warning.
#
# The hashes are stripped because pip switches the whole install to
# --require-hashes as soon as one constraint carries a hash, and the dev
# tools are unhashed. The step fails if a lockfile entry did not become a
# constraint (or none did), instead of letting it install unpinned.
#
# A resolver conflict here usually means requirements.txt raised a floor
# past the locked version, as Dependabot pip PRs do: lockfile-drift is red
# as well, and recompiling the lockfile fixes both. If lockfile-drift is
# green, a dev tool wants a different version of a locked package. If it
# caps one below the pin, hold that dev-tool bump. If it needs a newer
# one, which a plain recompile does not move, move it explicitly — that
# changes what ships, so review it like a runtime bump (uv leaves the flag
# out of the lockfile header, so lockfile-drift stays green):
# uv pip compile --generate-hashes --python-version 3.14 --python-platform x86_64-unknown-linux-gnu --upgrade-package <name> --output-file requirements.lock requirements.txt
- name: Install Python dependencies
run: |
constraints="$RUNNER_TEMP/constraints.txt"
grep -E '^[A-Za-z0-9._-]+==' requirements.lock | cut -d' ' -f1 > "$constraints"
if [ ! -s "$constraints" ] || grep -E '^[^[:space:]#]' requirements.lock | grep -qvE '^[A-Za-z0-9._-]+=='; then
echo "::error::Not every requirements.lock entry could be turned into a constraint (see the comment above this step in ci.yml)."
exit 1
fi
pip install -r requirements-dev.txt -c "$constraints"
- name: Lint (ruff)
run: ruff check .
- name: Format check (ruff)
run: ruff format --check .
# One Python version across image, CI, and lockfile. A Dependabot base-image
# bump used to move production to a version CI never tested (2b26b49: 3.12 ->
# 3.14, undetected for three months). The Dockerfile is the source of truth;
# this fails if anything disagrees. See the script's docstring.
- name: Python-version consistency gate
run: python scripts/check_python_version.py
# Tailwind dynamic-class gate — forbids `class="prefix-{{ x }}-suffix"`
# patterns that JIT extraction misses, leaving the production bundle
# silently incomplete. See scripts/check_template_classes.py docstring.
- name: Template class gate
run: python scripts/check_template_classes.py
# Tailwind bundle freshness gate — rebuild the purged bundle from the
# committed templates/JS and fail if it differs from the committed one.
# A stale bundle silently drops every utility class added since the last
# manual rebuild: from May to September 2026 about 46 classes the
# templates used (grid columns, gaps, list numbering, bottom-0, …) never
# rendered. The script pins the CLI version and verifies its SHA-256
# before running it. A deleted old bundle shows up as a tracked change and
# a new one as untracked, so `status --porcelain` catches either; the
# status runs on its own line so a git failure aborts instead of passing.
# On failure the CI-built bundle is uploaded — useful when a local build
# differs (Tailwind also scans untracked template files, CI never sees them).
- name: Tailwind bundle freshness gate
id: tailwind
run: |
bash scripts/build-tailwind.sh
changes="$(git status --porcelain -- app/static/css/)"
if [ -n "$changes" ]; then
printf '%s\n' "$changes"
echo "::error::The committed Tailwind bundle is stale. Run 'bash scripts/build-tailwind.sh' and commit app/static/css/ (the CI-built bundle is attached as an artifact)."
exit 1
fi
- name: Upload the CI-built Tailwind bundle
if: failure() && steps.tailwind.outcome == 'failure'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: tailwind-bundle-ci-built
path: |
app/static/css/tailwind.*.css
!app/static/css/tailwind.input.css
retention-days: 7
# i18n drift gate — re-extract the .pot from current sources and fail
# if it differs from the committed file. Catches a developer who added
# `_('...')` calls but forgot `python scripts/i18n.py extract` before
# pushing. Translation work then stays in sync with code.
- name: i18n message-catalog drift
run: python scripts/i18n.py drift-check
# pip-audit scans dependencies for known CVEs.
# Blocking — a failure means a published advisory affects a pinned dep.
# If a finding is genuinely unfixable upstream, add the ID to the --ignore-vuln list
# below with a comment naming the package + reason; never silence the whole step.
#
# Ignored advisories:
# - PYSEC-2026-1325 (= CVE-2024-23342, GHSA-wj6h-64fc-37mp): Minerva timing
# side-channel in `ecdsa`, transitive via python-jose. No fixed version
# exists; upstream declares side-channel attacks out of scope, so none is
# planned. Not reachable in FileMorph: JWTs are HS256-only
# (app/core/tokens.py) — no ECDSA sign/keygen/ECDH path ever runs.
# Added 2026-07-15; re-evaluate at each release, drop once we migrate
# off python-jose (e.g. PyJWT) or upstream ships a fix.
# Audits requirements.lock, not requirements.txt: the lockfile is what the
# image installs (Dockerfile: pip install --require-hashes), so it is the
# only file whose contents actually reach production. The lockfile-drift
# job below guarantees it still corresponds to requirements.txt.
# - CVE-2026-55073 (= GHSA-jf6q-chmf-3h3v), MODERATE / CVSS 6.2 (AV:L):
# SSRF-protection bypass in WeasyPrint < 70.0. Two write_pdf() channels
# build a fresh default URLFetcher instead of the document's one, so a
# restrictive `url_fetcher` is silently ignored for `xmp_metadata=[url]`
# and `stylesheets=[url]`. Not reachable in FileMorph: all four
# write_pdf() call sites pass the output path only and never those
# parameters (app/converters/document.py:228, 392, 426, 491), so the
# advisory's third precondition -- forwarding an attacker-influenced
# URL into either -- cannot occur. The fix is 70.0, which we cannot take
# yet: it reworks the url_fetcher contract from a callable to an object
# and would break `_deny_url_fetcher` (see the cap in requirements.txt).
# Re-evaluate when the fetcher is ported; drop this ignore then.
# Added 2026-09-09.
- name: Dependency vulnerability scan (pip-audit)
run: pip-audit -r requirements.lock --ignore-vuln PYSEC-2026-1325 --ignore-vuln CVE-2026-55073
- name: Run tests
run: pytest tests/ -v --tb=short
# requirements.lock is what the image installs, so it has to keep
# corresponding to requirements.txt. Before this gate existed it did not:
# Dependabot pushed 32 commits to requirements.txt in six months while the
# lockfile got 4 (one of them only line endings), leaving it short of four
# packages entirely and behind on five more. Recompiling here and diffing is
# the only way to notice.
#
# On drift this fails and uploads the correct lockfile as an artifact — grab
# it and commit it, or run the deps-lock workflow to have it committed for
# you. The lockfile targets the shipped Python version (markers resolve per
# version, so one resolved elsewhere can be uninstallable in the image);
# `uv pip compile --python-version` resolves *for* that version without
# needing it installed, which is why anyone can regenerate the lockfile from
# any machine.
#
# requirements-sbom.lock, the SBOM generator's lockfile, is checked the same
# way against requirements-sbom.txt. Dependabot only edits that .txt, so
# without this a generator bump would merge green and change nothing.
lockfile-drift:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
# Any interpreter would do here — uv resolves for the target version
# via --python-version. Kept at the shipped version so every workflow
# agrees, which the gate in lint-and-test enforces.
python-version: "3.14"
cache: "pip"
# The exact uv requirements-dev.txt pins, read from that file: a resolver
# update can legitimately produce a different lockfile, which would turn
# this gate red with no dependency change behind it, so this job,
# deps-lock.yml and a local recompile must run the same release.
# Dependabot bumps only requirements-dev.txt; a version typed in here was
# left behind twice. The assignment is deliberate: under `bash -e` it
# stops the step when the pin is missing, whereas pip takes an empty
# argument as nothing to install and exits 0. The pattern admits only a
# version, and --only-binary keeps build code out of deps-lock's job,
# which can push.
- name: Install uv
run: |
uv_pin=$(grep -oE '^uv==[0-9][0-9A-Za-z.!+-]*' requirements-dev.txt)
pip install --only-binary :all: "$uv_pin"
# The command must match the one recorded in the committed lockfile's
# header verbatim, since uv writes that command into the file. uv also
# reads the existing output file as a preference, so unchanged
# dependencies keep their pins and a new upstream release does not turn
# this gate red on its own.
#
# --python-platform matters as much as --python-version: uvicorn[standard]
# pulls uvloop on Linux and not on Windows, so a lockfile resolved on a
# developer's Windows box is missing a package the image needs. Pinning
# the platform makes the result identical wherever it is generated. This
# gate caught exactly that on its first run.
- name: Recompile the lockfiles
run: |
cp requirements.lock requirements.lock.committed
uv pip compile --generate-hashes --python-version 3.14 --python-platform x86_64-unknown-linux-gnu --output-file requirements.lock requirements.txt
cp requirements-sbom.lock requirements-sbom.lock.committed
uv pip compile --generate-hashes --python-version 3.14 --python-platform x86_64-unknown-linux-gnu --output-file requirements-sbom.lock requirements-sbom.txt
- name: Compare
id: compare
run: |
drifted=""
diff -u requirements.lock.committed requirements.lock || drifted="$drifted requirements.lock"
diff -u requirements-sbom.lock.committed requirements-sbom.lock || drifted="$drifted requirements-sbom.lock"
if [ -z "$drifted" ]; then
echo "requirements.lock and requirements-sbom.lock are in sync with their .txt files"
else
echo "::error::Out of sync with its .txt file:$drifted. Download the 'requirements-lock-recompiled' artifact and commit the file(s) named here, or run the deps-lock workflow."
exit 1
fi
- name: Upload the recompiled lockfiles
if: failure() && steps.compare.outcome == 'failure'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: requirements-lock-recompiled
path: |
requirements.lock
requirements-sbom.lock
retention-days: 7
secret-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- name: gitleaks (server-side secret scanner)
uses: gitleaks/gitleaks-action@e0c47f4f8be36e29cdc102c57e68cb5cbf0e8d1e # v3.0.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}