Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
2421323
feat(core,db): AuditLink.for_model and independent_session (#523)
antosubash Oct 21, 2026
ea604dc
chore(deps): bound FastAPI below 0.157 with a private-API contract te…
antosubash Oct 21, 2026
83d6c18
feat(pagination): PageParams + paginate helper, PaginationBar, migrat…
antosubash Oct 21, 2026
e4a12b2
Merge branch 'slice/t3-helpers-523' into chore/tier3-batch-502-523
antosubash Oct 21, 2026
549553e
Merge branch 'slice/t3-pagination-513' into chore/tier3-batch-502-523
antosubash Oct 21, 2026
147edf8
refactor(scaffold): one module scaffolder; make new-module uses the C…
antosubash Oct 21, 2026
139713d
Merge branch 'slice/t3-scaffold-509' into chore/tier3-batch-502-523
antosubash Oct 21, 2026
14ccf4f
refactor(optimize): drop one-use aliases and a duplicate page clamp, …
antosubash Oct 21, 2026
f2c0cd8
fix(review): bump optional-dependencies, fixed tenants page size, bou…
antosubash Oct 21, 2026
1b146e7
fix: address code review findings (round 1, pass 1)
antosubash Oct 21, 2026
46c532f
fix(pagination): consistent page/per_page bounds on JSON APIs and the…
antosubash Oct 21, 2026
5e01ea1
fix: pagination view leniency for outbox per_page and users admin (re…
antosubash Oct 21, 2026
3e5747a
fix: address code review findings (round 2, pass 1)
antosubash Oct 21, 2026
94fc0de
fix: address code review findings (round 2, pass 2)
antosubash Oct 21, 2026
c735eb3
Merge remote-tracking branch 'origin/main' into chore/tier3-batch-502…
antosubash Oct 21, 2026
0f93657
test(db): create and drop the independent_session scratch table per t…
antosubash Oct 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,18 @@ All notable changes to this project are documented in this file. The format is b
## [Unreleased]

### Added
- `AuditLink.for_model(Model, url_template=..., ...)` fills `entity_type` (class
name) and `table_name` from the model; every module's `register_audit_links`
now uses it. `simple_module_db.independent_session(app)` is an async context
manager for out-of-request sessions that commits on success and rolls back on
error (`permissions` startup replay and `api_tokens` last-used write use it)
(#523).
- Framework pagination helper (#513): `simple_module_db.PageParams`, `Page` and
`async paginate(session, stmt, params)` (clamps to the last page, never a
negative offset, counts through soft-delete/tenant filters), `clamp_page` /
`total_pages`; `simple_module_hosting.pagination` adds the strict
`page_params(n)` and clamping `lenient_page_params(n)` dependencies; and a
`PaginationBar` component in `@simple-module-py/ui` (`ui.pagination.*` keys).
- New `mail` module (`simple_module_mail`) owns outgoing mail: the SMTP/console
transport, its settings (`/admin/settings/`, package `mail`) and the on-demand
`mail.mailer` health check behind "Test mailer connection". Modules send through
Expand Down Expand Up @@ -39,6 +51,35 @@ All notable changes to this project are documented in this file. The format is b
it, which removes the `users` ↔ `audit_log` import cycle.

### Changed
- Bound FastAPI to `>=0.140,<0.157` in framework and module dependencies; a
contract test guards the route guard's private FastAPI/Starlette APIs (#516).
- Declare and document the optional `file_storage[tasks]` extra for the lockstep
background-tasks package; document `DurableTransport` as the durable-event
extension point and verify uploads without background tasks (#502).
- Users, file_storage, background_tasks (executions and outbox), audit_log,
settings and tenants lists now share the pagination helpers instead of
per-module copies (#513). Browser view routes (`background_tasks`
including the outbox page, `file_storage`, `tenants` admin, `users` admin) clamp
`?page=0`/negative/garbage to page 1 rather than reject; every JSON list API
(`background_tasks` executions and outbox, `audit_log`, `file_storage`,
`users` admin) answers 422 for `page` outside 1..1,000,000 and `per_page`
outside 1..200, where some used to accept `page=0` or an unbounded `per_page`
or forward a huge page to SQL as an out-of-range offset. The outbox API now
reports the clamped page when asked past the end. The audit log, executions
and outbox tables use `PaginationBar`; their private `showing/previous/next`
locale keys are removed in favour of `ui.pagination.*`.
- One module scaffolder (#509). `make new-module` now calls the CLI's
`create_module` (`framework_repo=True` overlay) instead of ~980 lines of
f-string templates in `scripts/_templates_*.py`, which are deleted, so
`smpy create-module` / `smpy new` and `make new-module` share one set of
`templates/module/` files. Both now emit `constants.py`, `models.py`,
`contracts/`, `service.py`, `deps.py`, permission-guarded REST + Inertia
endpoints, Browse/Create/Edit pages, a menu item with `label_key`,
`register_permissions`, `locale_dirs` and `locales/en.json` (the CLI sample
page `Index.tsx` is replaced by `Browse.tsx`). `make new-module` also adds
the module to the `publish-pypi` matrix and runs `npm install` and
`make gen-i18n`; a fresh module passes `make lint`. The scaffold tests live in
`framework/cli/tests/` only.
- `audit_log` and `dashboard` no longer depend on `Users` (meta `depends_on`
and package dependency); `background_tasks` drops its unused `Users`
dependency (#501). `api_tokens` moves `Users` to `optional_depends_on` and
Expand Down
53 changes: 33 additions & 20 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,13 +131,13 @@ The Python package name (`feature_flags`) keys the i18n namespace,
the namespace is the meta name.

**Module layout** (`make new-module` scaffolds the files below except
`constants.py`, `tasks.py` and the CSS, plus `pyproject.toml`, `README.md`,
`LICENSE`, `package.json`, `tsconfig.json`, `tests/`):
`tasks.py` and the CSS, plus `pyproject.toml`, `README.md`, `LICENSE`,
`package.json`, `tsconfig.json`, `tests/`):

```
modules/<name>/<name>/
├── module.py # ModuleBase subclass, meta = ModuleMeta(...)
├── constants.py # add it: PERM_*, ROLE_*, _PAGE_*, _MODULE_* — see § Conventions
├── constants.py # PERM_*, PAGE_*, MENU_*, ROLE_*, _MODULE_* — see § Conventions (scaffolded)
├── models.py # SQLModel tables (table=True)
├── contracts/ # SQLModel DTOs — the public surface; add a Protocol only for real extension points
├── settings.py # DbBackedSettings subclass
Expand Down Expand Up @@ -371,8 +371,9 @@ development and testing (`session_cookie_secure`).
durable handlers, which have no app). The `mail` module owns SMTP and its
settings; without it the sender only logs (`delivers_email` is false). Tests
use the `captured_mail` fixture.
- **Audit links / design packs**: `register_audit_links` takes
`AuditLink(entity_type=Model.__name__, …)` (class name, not table name, or it
- **Audit links / design packs**: `register_audit_links` builds
links with `AuditLink.for_model(Model, url_template=…, …)` (it fills
`entity_type=Model.__name__` and `table_name`; a table-name `entity_type`
silently never matches); `register_design_packs` slugs must match
`^[a-z0-9][a-z0-9-]*$` and the host must still import the CSS.
- **i18n.** Ship `<package>/locales/<lang>.json`, declare in `locale_dirs()` with
Expand Down Expand Up @@ -407,23 +408,35 @@ development and testing (`session_cookie_secure`).

## Adding a module or package

`make new-module name=<name>` scaffolds the package, wires it into
`host/pyproject.toml` and the root `testpaths`/`extra-paths`, and runs
`uv sync`. Expect `make lint` to fail right after: the scaffold still emits a
literal `inertia.render("<Class>/Create")` and permission and menu strings
without constants or `label_key`. Fix those first. It also does **not**:

1. Run `npm install`, so regenerate and commit `package-lock.json` (CI uses
`npm ci`). Commit the `uv.lock` change the scaffold's
`uv sync` makes too (CI uses `uv sync --locked`).
2. Add `simple_module_<name>` to the `publish-pypi` matrix in
`.github/workflows/release.yml`; a test and `check_metadata.py` fail until
you do. `pyproject.toml` needs description, `readme`, `license = "MIT"`,
keyword `simple-module`, `urls.Repository`; `README.md` needs ≥ 500 bytes, an
H1 and Install/Usage sections.
There is **one scaffolder**: `make new-module name=<name>` runs
`scripts/new_module.py`, a thin wrapper over the CLI's
`simple_module_cli.scaffolding.create_module(..., framework_repo=True)`, the same
templates `smpy create-module` / `smpy new` use
(`framework/cli/simple_module_cli/templates/module/`). Change a convention there,
once. The `framework_repo` variant is a template overlay
(`_optional/framework_repo/`: workspace-sourced `pyproject.toml` with the
`check_metadata.py` fields, `LICENSE`, `README.md`, i18n-keyed pages).

It generates a CRUD sample (model, DTOs, service, REST + Inertia endpoints,
Browse/Create/Edit pages, `locales/en.json`, tests) whose permissions, page keys
and menu item come from `constants.py` (`label_key` set, routes guarded), so a
fresh module passes `make lint`. Then
`simple_module_cli.framework_repo.wire_module` adds it to `host/pyproject.toml`,
the root `testpaths`/`extra-paths` and the `publish-pypi` matrix in
`release.yml`, and the target runs `uv sync --all-packages`, `npm install` and
`make gen-i18n`. Commit the resulting `uv.lock`, `package-lock.json` and
`packages/i18n/src/{keys.generated,generated-resources}.ts` changes (CI uses
`--locked` / `npm ci` and diffs the i18n files). It does **not**:

1. Create a migration for the sample table: edit `models.py`, then follow
§ Database (`--head <label>@head`, set `branch_labels` on the first
revision). `make doctor` reports `SM011` until you do.
2. Register a PyPI pending publisher for the new `simple_module_<name>`
(manual; the release leg 403s without it).
3. Write `docs/modules/<name>.md`, add it to `docs/modules/index.md` and the
VitePress sidebar in `docs/.vitepress/config.ts`.
4. Set `branch_labels` on the first migration, or add a `CHANGELOG.md` entry.
4. Add a `CHANGELOG.md` entry, or add the module to the e2e/perf
`SM_MODULES_ENABLED` lists if those jobs should exercise it.

A new `framework/*` package is wired by hand: root `pyproject.toml`
`testpaths` + `[tool.ty.environment].extra-paths` and the `publish-pypi`
Expand Down
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,8 @@ new-module: ## Scaffold a new module (usage: make new-module nam
@test -n "$(name)" || (echo "Error: Please provide a module name, e.g. make new-module name=orders" && exit 1)
uv run python scripts/new_module.py $(name)
uv sync --all-packages
npm install
$(MAKE) gen-i18n

# Kill dev servers
kill:
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ packages/
ui/ # shared shadcn/ui components, layouts, and design-system primitives
i18n/ # generated i18n keys + translation runtime
scripts/
new_module.py # module scaffolder (called by `make new-module`)
new_module.py # thin wrapper over the CLI scaffolder (`make new-module`)
docs/
plans/ # design + implementation plans
framework-conventions.md # invariants for module authors
Expand Down
1 change: 1 addition & 0 deletions docs/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ export default defineConfig({
{ text: "Per-module Base", link: "/database/per-module-base" },
{ text: "Mixins", link: "/database/mixins" },
{ text: "Session lifecycle", link: "/database/sessions" },
{ text: "Pagination", link: "/database/pagination" },
{ text: "Migrations", link: "/database/migrations" },
{ text: "Extensions and column types", link: "/database/extensions" },
],
Expand Down
36 changes: 36 additions & 0 deletions docs/database/pagination.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Pagination

One helper clamps, counts and slices, so every list agrees on what `?page=0`
or a page past the end means.

```python
from simple_module_db import PageParams, paginate
from simple_module_hosting.pagination import page_params


# JSON API: page < 1 or per_page outside 1..200 is a 422.
@router.get("/", response_model=None)
async def list_orders(
params: PageParams = Depends(page_params(default_per_page=50)),
db: AsyncSession = Depends(get_db),
):
page = await paginate(db, select(Order).order_by(Order.id), params)
return page # items, total, page, per_page, pages
```

- `PageParams(page, per_page)` lives in `simple_module_db` (no FastAPI import).
It normalises on construction (`page >= 1`, `1 <= per_page <= 200`), so
`.offset` is never negative. `simple_module_hosting.pagination` builds it from
the query string: `page_params(n)` is strict (422), `lenient_page_params(n)` /
`LenientPageParamsDep` clamp garbage instead, for Inertia views reached from
bookmarked URLs.
- `paginate(session, stmt, params, scalars=True)` counts `stmt` as a subquery,
so soft-delete and tenant filters apply to the count exactly as to the rows.
`stmt` needs its own `ORDER BY`. A page past the end is clamped to the last
page and the returned `page` is the one served. Pass `scalars=False` for
multi-column selects (items are rows).
- Queries that cannot use `paginate` (a window-function total, a service that
counts separately) still share `clamp_page(page, total, per_page)`,
`total_pages(total, per_page)` and `PageParams.offset`.
- Frontend: `PaginationBar` from `@simple-module-py/ui/components/PaginationBar`
takes `page`, `perPage`, `total`, `onPageChange` (the `Page` shape).
15 changes: 15 additions & 0 deletions docs/database/sessions.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,21 @@ async def worker(db: DatabaseState):

The `async with session.begin()` pattern opens a transaction you control. Use it in code that runs **outside** a request scope.

### `independent_session(app)`

For the common case, a short unit of work outside the request that should commit when it succeeds and roll back when it fails, use `independent_session`:

```python
from simple_module_db import independent_session


async def on_startup(self, app):
async with independent_session(app) as db:
await replay_rows(db) # committed on clean exit, rolled back on error
```

It opens a session from `app.state.sm.db.session_factory`, commits when the block exits cleanly, and rolls back and re-raises otherwise. Do not call `commit()` inside the block. Keep using `session_factory()` directly when the block is read-only, must commit mid-way (for example to do follow-up work after the commit, as the branding image reaper does), or runs against another engine.

## Sessions and `MultiTenantMixin`

The active tenant is carried in the `current_tenant_id` contextvar (set by `TenantMiddleware`, not on the session). The tenant listeners read that contextvar to filter SELECTs and stamp INSERTs. Sessions stay **request-scoped** so they always run under the request's tenant context — sharing one across tenants would silently leak data.
Expand Down
4 changes: 2 additions & 2 deletions docs/framework-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ modules/<name>/
└── pages/ # *.tsx — auto-discovered by Vite
```

Scaffold a fresh module with `smpy create-module <name> --dest modules/<name>` — it generates a working subset (`module.py`, `services.py`, `settings.py`, `endpoints/api.py`, an empty `pages/`, plus packaging files); add `models.py`, `deps.py`, `contracts/`, `endpoints/views.py`, and `locales/` as your module needs them. Then run `uv add ./modules/<name>` to register it on your app.
Scaffold a fresh module with `smpy create-module <name> --dest modules/<name>` — it generates a working CRUD sample (`module.py`, `constants.py`, `models.py`, `contracts/`, `service.py`, `deps.py`, `settings.py`, `services.py`, `endpoints/{api,views}.py`, `pages/{Browse,Create,Edit}.tsx`, `locales/en.json`, plus packaging files and tests). `make new-module` uses the same templates for a module inside this repo. Then run `uv add ./modules/<name>` to register it on your app.

## ModuleMeta

Expand Down Expand Up @@ -621,7 +621,7 @@ class OrdersModule(ModuleBase):
return {"orders": Path(str(importlib.resources.files(__package__) / "locales"))}
```

Add this method (and a matching `locales/en.json`) yourself — the `smpy create-module` scaffold does not emit a `locales/` directory.
The `smpy create-module` / `make new-module` scaffold emits this method and a starter `locales/en.json`; add it yourself only for a hand-rolled module.

### Key naming

Expand Down
4 changes: 2 additions & 2 deletions docs/framework/events.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Events

`EventBus` is an **in-process** pub/sub mechanism. `publish` is not a message broker — no persistence, no retries, no cross-process delivery. For events that must be handled even across a crash, use [durable delivery](#durable-delivery) (`subscribe_durable` / `publish_durable`), which the `background_tasks` module backs with an outbox. To tell *other worker processes* that something they cached is stale, use the [invalidation bus](/framework/invalidation) instead — same shape, deliberately narrower contract.
`EventBus` is an **in-process** pub/sub mechanism. `publish` is not a message broker — no persistence, no retries, no cross-process delivery. For events that must be handled even across a crash, use [durable delivery](#durable-delivery) (`subscribe_durable` / `publish_durable`), through the `DurableTransport` protocol extension point. The framework ships an outbox implementation in `background_tasks`. To tell *other worker processes* that something they cached is stale, use the [invalidation bus](/framework/invalidation) instead — same shape, deliberately narrower contract.

Events are the preferred way for modules to react to **other modules'** actions without creating a direct code dependency.

Expand Down Expand Up @@ -186,7 +186,7 @@ await bus.publish_durable(OrderPlaced(order_id=order.id, ...), db)
`register_event_handlers(bus, None)` — subscribe durably *before* any early
`return` taken when `app is None`.

`DurableTransport` is a protocol (`async enqueue(event, handler_names, db)`), so
`DurableTransport` (`simple_module_core.durable_events`) is the extension point: a protocol (`async enqueue(event, handler_names, db)`), so
another module could provide one; `background_tasks` is the shipped
implementation — see [its docs](/modules/background_tasks#durable-events-outbox)
for the outbox, retries, the admin page and the worker side.
Expand Down
6 changes: 3 additions & 3 deletions docs/framework/lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,16 +272,16 @@ def register_audit_links(self, registry: AuditLinkRegistry) -> None:
from users.models import User

registry.register(
AuditLink(
entity_type=User.__name__, # class name — NOT __tablename__
AuditLink.for_model(
User, # fills entity_type=User.__name__ (NOT __tablename__) and table_name
url_template="/admin/users/{id}",
label="User",
label_key="users.audit.user",
)
)
```

`entity_type` matches `AuditEntry.entity_type`, which `snapshot_changes` writes as `type(obj).__name__`. Keying it off `__tablename__` (`"users_user"`) produces a link that never matches — and nothing errors: an unmatched lookup falls back to rendering `entity_type` as a plain label, so the row still shows text while silently never becoming a link. Using `Model.__name__` rather than a string literal makes a rename impossible to get wrong.
`entity_type` matches `AuditEntry.entity_type`, which `snapshot_changes` writes as `type(obj).__name__`. Keying it off `__tablename__` (`"users_user"`) produces a link that never matches — and nothing errors: an unmatched lookup falls back to rendering `entity_type` as a plain label, so the row still shows text while silently never becoming a link. `AuditLink.for_model(Model, ...)` fills `entity_type` from `Model.__name__` and `table_name` from `Model.__tablename__`, so a rename or a table-name mix-up is impossible; the plain `AuditLink(...)` constructor stays available for links without a model class.

`url_template` must contain the literal `{id}` placeholder, substituted with the entity id. A template without it raises at boot rather than pointing every row at the same page.

Expand Down
12 changes: 10 additions & 2 deletions docs/modules/file_storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

Pluggable file storage with two shipped backends — local filesystem and S3-compatible — plus a browse / upload / download admin UI. Backend selection is per-deployment (DB-backed setting), so you can switch from `filesystem` (dev / single-node) to `s3` (prod) without code changes.

Install `simple_module_file_storage[tasks]` to include the matching
`simple_module_background_tasks` package, then enable `background_tasks` and run
its worker for asynchronous multipart-upload checksums. Run its beat scheduler
as well for upload cleanup jobs.
The extras combine as `simple_module_file_storage[s3,tasks]`. Without
`background_tasks`, uploads still work; completed multipart uploads keep
`checksum_sha256` null. Streamed uploads compute their checksum inline.

## ModuleMeta

| Field | Value |
Expand Down Expand Up @@ -250,8 +258,8 @@ The audit-log label resolver names files across tenants on purpose: the audit
log is a platform screen over every tenant's entries, each of which already
records the filename.

The module has no background jobs, sweeps or CLI commands. Anything added
later that touches `StoredFile` outside a request must run under
The optional checksum task and upload reaper use the background-tasks worker.
Any task that touches `StoredFile` outside a request must run under
`tenant_context(row.tenant_id)` for per-tenant work, or `all_tenants()` for a
deliberate platform-wide sweep.

Expand Down
Loading
Loading