Skip to content

ci: enforce backend import layers - #87

Merged
dimslaev merged 4 commits into
test/backend-flat-testsfrom
ci/backend-import-layers
Oct 19, 2026
Merged

dimslaev merged 4 commits into
test/backend-flat-testsfrom
ci/backend-import-layers

Conversation

@dimslaev

@dimslaev dimslaev commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Step 6 of the backend migration. Stacked on #86.

What changed

  • import-linter 2.15 is a backend dev dependency (>=2.15,<3.0.0), and uv.lock is updated.
  • The contracts are in backend/pyproject.toml under [tool.importlinter].
  • pre-commit has a local import-linter hook (uv run lint-imports --config backend/pyproject.toml). It runs on every commit, like mypy, so a deleted file or a lock bump can't skip it.
  • The test-backend workflow has a Check import layers step before the db comes up.
  • backend/conftest.py sits outside the app package, so it isn't linted.

Contracts

  1. main > entry > features > core: layers, with containers = ["app"] and exhaustive = true. A new top-level module under app/ must join a layer. app/test_main.py is the one exception.
  2. Features never import each other: independence over app.features.*.
  3. api, mcp and cli never import each other: independence over app.entry.api, app.entry.mcp and app.entry.cli.
  4. Flat core modules never import core.db: forbidden, from app.core.* to app.core.db. Indirect chains count too.
  5. Entry reaches a feature only through its service and model: forbidden, from app.entry to app.features.*.*, with allow_indirect_imports = true. The direct app.entry.** -> app.features.*.service and -> app.features.*.model edges are ignored, so no list has to be kept by hand. The one exception is app.entry.api.test_articles -> app.features.catalog.semantic_search, because the search tests stub embed where it is called.

Proof each contract fails

For each one I added a violating import, ran lint-imports, and then removed it.

  1. app/core/dates.py: from app.features.catalog import model
main > entry > features > core
app.core is not allowed to import app.features:
- app.core.dates -> app.features.catalog.model (l.31)

A new empty app/probe.py also breaks it:

- app.probe
(Since this contract is marked as 'exhaustive', every child of every container must be declared as a layer.)
  1. app/features/stories/service.py: from app.features.catalog import model
Features never import each other BROKEN
app.features.stories is not allowed to import app.features.catalog:
- app.features.stories.service -> app.features.catalog.model (l.377)
  1. app/entry/cli/report.py: from app.entry.api import deps
api, mcp and cli never import each other BROKEN
app.entry.cli is not allowed to import app.entry.api:
- app.entry.cli.report -> app.entry.api.deps (l.11)
  1. app/core/urls.py: from app.core.db import article
Flat core modules never import core.db BROKEN
app.core.urls is not allowed to import app.core.db:
-   app.core.urls -> app.core.db.article (l.84)

It also reports every flat module that reaches it, for example app.core.extract -> app.core.urls -> app.core.db.article.

  1. app/entry/api/articles.py: from app.features.catalog import facets
Entry reaches a feature only through its service and model BROKEN
app.entry is not allowed to import app.features.catalog.facets:
-   app.entry.api.articles -> app.features.catalog.facets (l.139)

With the test_articles ignore line removed:

-   app.entry.api.test_articles -> app.features.catalog.semantic_search (l.27)

mypy and ty in pre-commit

  • The hooks ran uv run mypy backend/app and uv run ty check backend/app from the repo root. Neither one read backend/pyproject.toml.
  • mypy ran non-strict. A probe def _probe(x): return x got no error from the hook, but cd backend && uv run mypy app gave no-untyped-def.
  • ty ignored [tool.ty.terminal]. With error-on-warning = false, the root run still exited 1 and the backend run exited 0.
  • The hooks are now uv run mypy --config-file backend/pyproject.toml backend/app and uv run ty check --project backend backend/app. Both pass, with the strict config applied.

Bug fixes

  • fix: run mypy and ty in pre-commit with the backend config, as described above.
  • fix: name the exact claude-code-action tag in its pin comments. claude.yml and claude-code-review.yml pin f27be6b with the comment # v1. The v1 tag moved to 5eb090f (v1.0.253), so zizmor's online ref-version-mismatch audit started failing on every PR. The comment now says # v1.0.252, which is the tag on f27be6b. The pinned commit is the same.

Checks

  • ruff check and format, mypy (strict), ty, typos on changed files, banned names, lint-imports, zizmor --offline: clean. Online zizmor can't reach the GitHub API from the container.
  • Backend tests on a fresh Postgres 16 + pgvector db: 689 passed. The one failure is /search?q=agents, because the HF model download is blocked here. It passes in CI.
  • bash scripts/generate-client.sh: no diff.
  • prek run --all-files: typos fails on .claude/skills/curate/regression.md, which is already like that on the base and isn't touched here.

Review findings (code-review, high)

Fixed:

  • The hook had a files filter. A commit that only deleted a .py file or bumped uv.lock could skip it. The filter is gone.
  • The layers contract wasn't exhaustive, so a new top-level module under app/ was in no layer. It is now exhaustive.

Not changed:

  • A name re-exported from a feature's __init__.py would get past contract 5. Every feature __init__.py is empty today.
  • app.main isn't a source in contract 5. It imports only entry and core.
  • lint-imports runs in both the pre-commit and test-backend workflows on a PR. The test-backend step also covers pushes to master, where pre-commit doesn't run.
  • Contract 4's app.core.* also matches app.core.db, and import-linter skips that overlap. A comment in the config says so.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Lv3GURL2oLiS9Rff2o5vQU

* docs: point the deploy README's env table at app/features

* ci: deploy only what the box runs

* test: check the deploy filter ships every path the units and the job use

* docs: record the layered backend (#89)

* docs: record the layered backend

* fix: address the backend migration review (#91)

* fix: configure logging for the api process

* fix: let a publisher's link claim a platform subdomain

* fix: fail a link loudly on a non-numeric jev answer

* docs: point stale docstrings at where the code moved

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
@dimslaev
dimslaev merged commit 0d88d95 into test/backend-flat-tests Oct 19, 2026
5 checks passed
@dimslaev
dimslaev deleted the ci/backend-import-layers branch October 19, 2026 22:37
dimslaev added a commit that referenced this pull request Oct 19, 2026
* test: colocate backend tests beside their modules

* test: cover the curation tools, sign-in refusals, publisher lookups and the known-URL filter

* fix: fail the daily report when its mail does not send

* test: pin that the report reaches the send, and clean up the filter's publisher

* ci: enforce backend import layers (#87)

* fix: run mypy and ty in pre-commit with the backend config

* ci: enforce backend import layers

* fix: name the exact claude-code-action tag in its pin comments

* ci: deploy only what the box runs (#88)

* docs: point the deploy README's env table at app/features

* ci: deploy only what the box runs

* test: check the deploy filter ships every path the units and the job use

* docs: record the layered backend (#89)

* docs: record the layered backend

* fix: address the backend migration review (#91)

* fix: configure logging for the api process

* fix: let a publisher's link claim a platform subdomain

* fix: fail a link loudly on a non-numeric jev answer

* docs: point stale docstrings at where the code moved

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
dimslaev added a commit that referenced this pull request Oct 19, 2026
* refactor: expose what entry needs through each feature's service

* refactor: split the mcp tools into app/entry/mcp and fold the router into main

* refactor: move scripts and the timer entrypoints into app/entry/cli

* test: cover the mcp token gate end to end over http

* test: cover the cli entrypoints' targets and exit codes

* fix: fail the backup when pg_dump writes nothing

* refactor: make app/entry pass strict mypy

* fix: keep sentry off in development

* chore: name the seed module in restore-db's hint without backticks

* docs: say what tests-start.sh does with its argument

* test: return the tool result's error flag and text from the gate tests' helper

* fix: stop importing the email module from configuring the root logger

* fix: require the alembic that reads its config from pyproject

* test: check the collect run leaves the root logger alone

* docs: say where command-line entrypoints live

* refactor: name the newsletter service alike in every mcp tool

* test: colocate backend tests beside their modules (#86)

* test: colocate backend tests beside their modules

* test: cover the curation tools, sign-in refusals, publisher lookups and the known-URL filter

* fix: fail the daily report when its mail does not send

* test: pin that the report reaches the send, and clean up the filter's publisher

* ci: enforce backend import layers (#87)

* fix: run mypy and ty in pre-commit with the backend config

* ci: enforce backend import layers

* fix: name the exact claude-code-action tag in its pin comments

* ci: deploy only what the box runs (#88)

* docs: point the deploy README's env table at app/features

* ci: deploy only what the box runs

* test: check the deploy filter ships every path the units and the job use

* docs: record the layered backend (#89)

* docs: record the layered backend

* fix: address the backend migration review (#91)

* fix: configure logging for the api process

* fix: let a publisher's link claim a platform subdomain

* fix: fail a link loudly on a non-numeric jev answer

* docs: point stale docstrings at where the code moved

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
dimslaev added a commit that referenced this pull request Oct 19, 2026
* refactor: split backend domains into app/features

* refactor: type app/features for mypy strict

* fix: keep links on a shared platform's subdomains with the aggregator

* fix: drop a newsletter product whose picked link is past the search results

* test: cover approve and reject against the schema, and publisher resolution edges

* chore: point the end-of-file exclude at the moved email templates

* refactor: tidy comments and test helpers from the features split

* refactor: move mcp tools and scripts into app/entry (#85)

* refactor: expose what entry needs through each feature's service

* refactor: split the mcp tools into app/entry/mcp and fold the router into main

* refactor: move scripts and the timer entrypoints into app/entry/cli

* test: cover the mcp token gate end to end over http

* test: cover the cli entrypoints' targets and exit codes

* fix: fail the backup when pg_dump writes nothing

* refactor: make app/entry pass strict mypy

* fix: keep sentry off in development

* chore: name the seed module in restore-db's hint without backticks

* docs: say what tests-start.sh does with its argument

* test: return the tool result's error flag and text from the gate tests' helper

* fix: stop importing the email module from configuring the root logger

* fix: require the alembic that reads its config from pyproject

* test: check the collect run leaves the root logger alone

* docs: say where command-line entrypoints live

* refactor: name the newsletter service alike in every mcp tool

* test: colocate backend tests beside their modules (#86)

* test: colocate backend tests beside their modules

* test: cover the curation tools, sign-in refusals, publisher lookups and the known-URL filter

* fix: fail the daily report when its mail does not send

* test: pin that the report reaches the send, and clean up the filter's publisher

* ci: enforce backend import layers (#87)

* fix: run mypy and ty in pre-commit with the backend config

* ci: enforce backend import layers

* fix: name the exact claude-code-action tag in its pin comments

* ci: deploy only what the box runs (#88)

* docs: point the deploy README's env table at app/features

* ci: deploy only what the box runs

* test: check the deploy filter ships every path the units and the job use

* docs: record the layered backend (#89)

* docs: record the layered backend

* fix: address the backend migration review (#91)

* fix: configure logging for the api process

* fix: let a publisher's link claim a platform subdomain

* fix: fail a link loudly on a non-numeric jev answer

* docs: point stale docstrings at where the code moved

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
dimslaev added a commit that referenced this pull request Oct 19, 2026
* fix: percent-encode database credentials in the dsn

* fix: declare the indexes and column types the migrations built

* refactor: move tables into app/core/db

* test: cover the schema against migrations, the engine and ledger writes

* refactor: split backend domains into app/features (#84)

* refactor: split backend domains into app/features

* refactor: type app/features for mypy strict

* fix: keep links on a shared platform's subdomains with the aggregator

* fix: drop a newsletter product whose picked link is past the search results

* test: cover approve and reject against the schema, and publisher resolution edges

* chore: point the end-of-file exclude at the moved email templates

* refactor: tidy comments and test helpers from the features split

* refactor: move mcp tools and scripts into app/entry (#85)

* refactor: expose what entry needs through each feature's service

* refactor: split the mcp tools into app/entry/mcp and fold the router into main

* refactor: move scripts and the timer entrypoints into app/entry/cli

* test: cover the mcp token gate end to end over http

* test: cover the cli entrypoints' targets and exit codes

* fix: fail the backup when pg_dump writes nothing

* refactor: make app/entry pass strict mypy

* fix: keep sentry off in development

* chore: name the seed module in restore-db's hint without backticks

* docs: say what tests-start.sh does with its argument

* test: return the tool result's error flag and text from the gate tests' helper

* fix: stop importing the email module from configuring the root logger

* fix: require the alembic that reads its config from pyproject

* test: check the collect run leaves the root logger alone

* docs: say where command-line entrypoints live

* refactor: name the newsletter service alike in every mcp tool

* test: colocate backend tests beside their modules (#86)

* test: colocate backend tests beside their modules

* test: cover the curation tools, sign-in refusals, publisher lookups and the known-URL filter

* fix: fail the daily report when its mail does not send

* test: pin that the report reaches the send, and clean up the filter's publisher

* ci: enforce backend import layers (#87)

* fix: run mypy and ty in pre-commit with the backend config

* ci: enforce backend import layers

* fix: name the exact claude-code-action tag in its pin comments

* ci: deploy only what the box runs (#88)

* docs: point the deploy README's env table at app/features

* ci: deploy only what the box runs

* test: check the deploy filter ships every path the units and the job use

* docs: record the layered backend (#89)

* docs: record the layered backend

* fix: address the backend migration review (#91)

* fix: configure logging for the api process

* fix: let a publisher's link claim a platform subdomain

* fix: fail a link loudly on a non-numeric jev answer

* docs: point stale docstrings at where the code moved

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
dimslaev added a commit that referenced this pull request Oct 19, 2026
* refactor: move shared backend mechanism into app/core

* refactor: reword the check_link module docstring

* fix: read a feed date with no timezone as utc

* test: cover token rejection, the daily mail send and github lookups

* test: spell unparsable the way typos expects

* refactor: move tables into app/core/db (#83)

* fix: percent-encode database credentials in the dsn

* fix: declare the indexes and column types the migrations built

* refactor: move tables into app/core/db

* test: cover the schema against migrations, the engine and ledger writes

* refactor: split backend domains into app/features (#84)

* refactor: split backend domains into app/features

* refactor: type app/features for mypy strict

* fix: keep links on a shared platform's subdomains with the aggregator

* fix: drop a newsletter product whose picked link is past the search results

* test: cover approve and reject against the schema, and publisher resolution edges

* chore: point the end-of-file exclude at the moved email templates

* refactor: tidy comments and test helpers from the features split

* refactor: move mcp tools and scripts into app/entry (#85)

* refactor: expose what entry needs through each feature's service

* refactor: split the mcp tools into app/entry/mcp and fold the router into main

* refactor: move scripts and the timer entrypoints into app/entry/cli

* test: cover the mcp token gate end to end over http

* test: cover the cli entrypoints' targets and exit codes

* fix: fail the backup when pg_dump writes nothing

* refactor: make app/entry pass strict mypy

* fix: keep sentry off in development

* chore: name the seed module in restore-db's hint without backticks

* docs: say what tests-start.sh does with its argument

* test: return the tool result's error flag and text from the gate tests' helper

* fix: stop importing the email module from configuring the root logger

* fix: require the alembic that reads its config from pyproject

* test: check the collect run leaves the root logger alone

* docs: say where command-line entrypoints live

* refactor: name the newsletter service alike in every mcp tool

* test: colocate backend tests beside their modules (#86)

* test: colocate backend tests beside their modules

* test: cover the curation tools, sign-in refusals, publisher lookups and the known-URL filter

* fix: fail the daily report when its mail does not send

* test: pin that the report reaches the send, and clean up the filter's publisher

* ci: enforce backend import layers (#87)

* fix: run mypy and ty in pre-commit with the backend config

* ci: enforce backend import layers

* fix: name the exact claude-code-action tag in its pin comments

* ci: deploy only what the box runs (#88)

* docs: point the deploy README's env table at app/features

* ci: deploy only what the box runs

* test: check the deploy filter ships every path the units and the job use

* docs: record the layered backend (#89)

* docs: record the layered backend

* fix: address the backend migration review (#91)

* fix: configure logging for the api process

* fix: let a publisher's link claim a platform subdomain

* fix: fail a link loudly on a non-numeric jev answer

* docs: point stale docstrings at where the code moved

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants