Skip to content

Put the API in a container, with the models in it and the documents out - #34

Merged
Badri-Rajendran merged 3 commits into
feature_bm25_in_postgresfrom
feature_container_image
Sep 23, 2026
Merged

Badri-Rajendran merged 3 commits into
feature_bm25_in_postgresfrom
feature_container_image

Conversation

@Badri-Rajendran

Copy link
Copy Markdown
Owner

Problem

CLAUDE.md requires a Docker build and image vulnerability scan in CI. docs/decisions/0002-ci-scope.md:31-36 records it as deliberately deferred: "There is no Dockerfile for the app." This adds one, and closes the first of that ADR's two deferred items.

Two properties of this app shape the image more than anything else:

  • It carries a model runtime. torch (553 MB of the 1.1 GB venv), sentence-transformers and two models are in the request path.
  • It must never carry the documents. data/sbc/ is 983 MB of issuers' PDFs, kept forever and strictly local (ADR 0016, ADR 0019).

Implementation

  • Multi-stage, non-root, /app holding only src, migrations and alembic.ini.
  • CPU torch on Linux via [[tool.uv.index]] + a sys_platform == 'linux' marker, so macOS development is untouched. Re-locking removed every nvidia-* package and triton.
  • Both models baked in at the revisions config.py pins, with HF_HUB_OFFLINE=1 at runtime. No HuggingFace call on a cold start, and an HF outage cannot take the API down.
  • CI builds the image, runs it, and fails if /app/data is not empty. A .dockerignore regression is otherwise invisible until an image is gigabytes too large — this is the rule that must not break quietly. Trivy then fails on fixable HIGH/CRITICAL.
  • One gunicorn worker, eight threads. Flask-Limiter has no shared storage, so a second worker keeps its own counters and doubles the 5/min register and 10/min login limits. A deliberate ceiling, recorded as one in ADR 0022.
  • Production logs JSON to stdout as well as to the file — a container's filesystem is ephemeral and unread. Development is unchanged so an ingestion run's log lines don't interleave with its progress output.

Also here because the image made them matter:

  • .env.example, which did not exist although three settings have no default.
  • make test / lint / check, so "it passed locally" means what CI means.
  • tiktoken and anyio dropped — verified zero imports repo-wide.

Validation — measured, not assumed

Two real defects were found by actually building it:

  1. .dockerignore excluded *.md, but pyproject.toml declares readme = "README.md", so hatchling failed: OSError: Readme file does not exist: README.md.
  2. uv run re-syncs the default dependency groups, silently pulling ruff, debugpy, jedi and pip back in after --no-dev had removed them. Now the venv's python is invoked directly.

The built image:

size 2.48GB   user policypal   data/: 0 entries
torch 2.14.0+cpu   nvidia packages: 0   ruff: no   pytest: no
models: 217M   gunicorn 26.2.0

The decisive test — a container with no data/ directory, against the live database:

data/ on disk: False
index from Postgres: ['bm25', 'chunk_ids'] 1568 chunks
reranked: 5
  0.999  hcg_glossary_Deductible.md

Served over HTTP it returns the right counties for a ZIP ({"counties":[{"county_fips":"12086","county_name":"Miami-Dade",...), 401 for an unauthenticated /api/auth/me, and access logs on stdout.

  • uv run pytest -q — 415 passing. ruff check clean.
  • Two new tests pin the logging split: production installs a stdout handler, development does not.

Security / DB / API impact

  • Security, positive: non-root user, no dev tooling in the image, no secrets baked in, image scanned on every PR, and CI enforces that no issuer PDF can reach it.
  • DB: none.
  • API: none. Behaviour is unchanged; only how it is packaged and where it logs.

Note

Stacked on #33 → #32 → #31 → #30 → #29 → #28. CD to Azure is ADR 0002's second deferred item and the next PR.

Copilot AI lite review requested due to automatic review settings September 22, 2026 20:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Closes the first of ADR 0002's two deferred items. Multi-stage, non-root, and
verified by running rather than by reading.

Two properties of this app shaped it. It carries a model runtime — torch and
two transformers are in the request path — and it must never carry the
documents, which are 983 MB of issuers' PDFs kept strictly local (ADR 0016).
So torch comes from PyTorch's CPU index on linux, which dropped every nvidia
package and triton from the lock, and both models are baked in at the
revisions config.py pins, with HF_HUB_OFFLINE set at runtime: a cold start
loads them from the image, and a HuggingFace outage cannot take the API down.

CI builds the image, runs it, and fails if /app/data is not empty. A
.dockerignore regression is otherwise invisible until an image is gigabytes
too large, and this is the rule that must not break quietly. Trivy fails the
build on fixable HIGH or CRITICAL findings.

One gunicorn worker, eight threads. Flask-Limiter has no shared storage, so a
second worker would keep its own counters and double the 5/min on register and
10/min on login. A deliberate ceiling, recorded as one.

Production also logs JSON to stdout, because a container's filesystem is
ephemeral and nobody reads it; development is unchanged so an ingestion run's
lines do not interleave with its progress output.

Also here because the image made them matter: .env.example, which did not
exist though three settings have no default; `make test`, `lint` and `check`
so passing locally means what CI means; and tiktoken and anyio dropped, both
imported nowhere, tiktoken in particular sitting next to core/text.py's
deliberate approximation looking like an unfinished intention.

Measured: 2.48 GB, torch 2.14.0+cpu, zero nvidia packages, no dev tools, and a
container with no data/ directory answering "What is a deductible?" from the
Postgres-stored index at 0.999.
The action's setup step runs an install script that exits 1 on the runner
with nothing on stderr, so the scan never started and the job failed without
saying why. Its published image is pinned instead, the same way gitleaks is
installed by checksum rather than by an action.

The image is scanned from a `docker save` tar with `--input`, so the scanner
is not handed the docker socket.
A tag can be moved to different bytes, so pinning by tag is not pinning. The
repository already SHA-pins its GitHub Actions and installs gitleaks by
checksum; images were the gap.

python:3.12-slim-bookworm, ghcr.io/astral-sh/uv and aquasec/trivy now carry
name:tag@sha256, the tag kept beside the digest so the version is still
readable.
@Badri-Rajendran
Badri-Rajendran merged commit 935b5a7 into feature_bm25_in_postgres Sep 23, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants