[dimslaev]

Step 3 of the backend architecture plan. Stacked on #83.

What moved

• app/features/ holds seven feature folders: catalog, stories, curation, collect, newsletter, audience, analytics. Each one has model.py (shapes and errors) and service.py (the verbs the outside calls).
• catalog, audience, newsletter and analytics came from app/<name>. stories/ was split out of catalog/stories.py, and its shapes out of catalog/models.py.
• curation came from pipeline/curation.py, steps/persist.py and steps/enrich.py (now publish.py), tags.py, and fetching/page_facts.py (now links.py). The feed and ledger row loading behind similar and stories is in coverage.py. Story (coverage) is now Coverage.
• collect came from the rest of pipeline/: run.py became service.py (run, run_nightly, send_report), steps/* became fetch.py, filter.py and queue.py, sources/substack.py became feeds.py, and sources/email.py became newsletters.py. RunStats and SourceStats moved to model.py.
• pipeline/run.py and pipeline/report.py are now two thin modules that call run_nightly and send_report. The systemd units are unchanged and need no reinstall.
• The email templates now sit beside the feature that sends them: audience/sign_in.html, audience/welcome.html and audience/email.py (the sign-in and welcome builders), and newsletter/weekly_issue.html. core/email.py keeps rendering and sending.
• subscribe moved into newsletter/service.py. DEDUP_DIST_THRESHOLD and RELATED_DIST moved to core/embedding.py, because curation and the newsletter both read them.
• The routes moved to app/entry/api/ now (login, users, likes, articles, stories, analytics, deps), not in step 4. The login and likes routes compose audience with newsletter and catalog, so leaving them in a feature folder would have meant a feature importing another feature. Features now import only core.
• The analytics ingest route body moved into analytics.service.record_event.
• The tests for core modules moved from tests/pipeline/ to tests/core/, together with the pure embedding maths tests (tests/core/test_embedding.py). The best_per_url tests moved to curation/tests/test_publish.py.
• Config: the T201 per-file ignore now covers app/features/collect/**. pipeline is gone from testpaths and the coverage source. The end-of-file-fixer exclude now points at the moved templates.

Behaviour changes

• None in the API or MCP. The OpenAPI document is byte-identical, scripts/generate-client.sh leaves frontend/src/client unchanged, and a dump of the MCP tool list is identical before and after.
• In the newsletter source, a Jev answer that is missing or not a number now counts as 0, so the link is dropped as off-topic. Before, it raised TypeError outside the per-link try, and that ended the source.

Bug fixes

• A link on a shared platform's subdomain (old.reddit.com, m.youtube.com, mobile.twitter.com, gist.github.com) or on youtu.be used to get a new publisher named after that host. It now stays with the aggregator, like links on the bare host. Subdomains of substack.com and medium.com are still one writer's site.
• In the newsletter source, a SelectProductLink index past the list of search results raised IndexError and ended the whole Newsletter source for the night. That product is now dropped.

Tests added

• approve and reject against the real schema: one article per approval with sanitized title, kind, tags, found_via and publisher naming; the pending row deleted; approving twice refused; a refused approval writes nothing; a failed embedding still publishes; a rejected candidate cannot then be approved; the same URL twice in one reject_many.
• Publisher resolution: shared-host subdomains, quarantining an unknown source once per run, slug collisions for a new author, and credit by URL over the feed that carried it.
• The collect run: a failed source rolls its session back, the errors a fetch reports reach the bookkeeping, one source failing leaves the next one running, run_nightly exit codes (crash, still recorded, recording failure), and send_report failing.
• The newsletter product pick, including indexes past the list.

Types

cd backend && uv run mypy app/features app/core (strict) reports 0 errors, tests included, with no per-module ignores. Root mypy and ty pass too.

How I verified

• ruff check and format, mypy (root and strict on app/features plus app/core), ty, typos, the banned-names hook, and all of prek over this branch's range.
• Backend tests on a fresh Postgres 16 + pgvector db: 638 passed, 1 failed. The one failure is /search, which needs the Hugging Face model download that this sandbox blocks. It passes in CI. Coverage is 87%.
• python -m pipeline.run and python -m pipeline.report against the local db: both exit 0 and record or report as before.
• OpenAPI diff and MCP tools/list diff are both empty.

Follow-ups (not in this PR)

• GET /articles?min_score= is capped at 10 (le=10), but scores run 1-100. Fixing it changes the generated client.
• article.url and publisher.slug have no unique index. Approve and _free_slug guard against duplicates in code only.
• docs/backend.md, the glossary and the deploy/README.md env table still name the old paths. That is step 8.

---

Generated by Claude Code