feat: user directory protocol; modules stop importing users.models; strict dependency discovery (#501)

CHANGELOG.md

@@ -23,8 +23,58 @@ All notable changes to this project are documented in this file. The format is b
   emailed when `public_base_url` is set. The invite API response gains
   `emailed`, and the invite form says whether the link was emailed.
 - `captured_mail` pytest fixture in `simple_module_test` records sent mail.
+- User directory protocol (#501): `simple_module_core.user_directory`
+  (`UserDirectory`, `DirectoryUser`, `DirectoryStats`, `NullUserDirectory`,
+  `get_user_directory` / `set_user_directory`). `users` installs
+  `UsersDirectory`, `keycloak` installs `KeycloakDirectory` (users who have
+  signed in; roles unknown). `audit_log`, `dashboard`, `file_storage`,
+  `api_tokens` and `permissions` read users through it instead of importing
+  `users.models`, so actor names and search, dashboard tiles and uploader
+  names now work under Keycloak instead of querying the empty `users_user`
+  table. `get_many` returns an entry for every caller spelling of a known id
+  (`parse_user_ids` maps a uuid to all of them). See
+  `docs/framework/user-directory.md`.
+- Activity feed protocol (#501): `simple_module_core.activity_feed`.
+  `audit_log` installs it; the users edit page's *Recent activity* card reads
+  it, which removes the `users` ↔ `audit_log` import cycle.
 
 ### Changed
+- `audit_log` and `dashboard` no longer depend on `Users` (meta `depends_on`
+  and package dependency); `background_tasks` drops its unused `Users`
+  dependency (#501). `api_tokens` moves `Users` to `optional_depends_on` and
+  its package depends on `fastapi-users-db-sqlalchemy` instead of
+  `simple_module_users`, so a Keycloak install keeps service keys (personal
+  tokens are refused there). `permissions` (role pages) still depends on
+  `Users`.
+- **Behaviour change (#501):** dashboard user figures the auth provider cannot
+  give are `null` in `/api/dashboard/stats` and their tile is hidden (with no
+  provider, all three; under Keycloak, `users_created_this_month`). Personal
+  API tokens are refused for an owner whose roles the directory cannot read
+  (Keycloak), as before.
+- `audit_log.resolve`: `resolve_actors`, `actor_filter` and `actor_link` take
+  the `UserDirectory`; `USER_ENTITY_TYPE` is gone (use the directory's
+  `entity_type`). `audit_log.export.stream_csv` takes the directory too.
+  `PermissionService` and `ApiTokenService` accept a directory (defaults to
+  the null directory). With no directory, the audit Actor search matches the
+  term as an exact actor id (#501).
+- `file_storage.resolve.resolve_uploader_labels` is now core's
+  `resolve_user_labels(directory, db, ids)` (the directory is the first
+  argument), and `users.audit.resolve_user_labels` delegates to it (#501).
+- `smpy` module catalog: `background_tasks` requires `settings` instead of
+  `users` and `dashboard` requires nothing; a test pins that every entry's
+  `requires` covers its module's `depends_on` (#501).
+- **Behaviour change (#501):** outside development, a module whose
+  `depends_on` names a module that is not loaded now refuses the boot with one
+  `MissingDependencyError` (an `InvalidModuleError`) listing every
+  `(module, dependency)` pair, instead of a warning and a module running
+  against tables nobody owns. Development still logs it; `make doctor`
+  reports it as the new `SM030` error. Selecting `keycloak` drops `Users`, so
+  a Keycloak install must leave out `permissions` (`SM_MODULES_ENABLED`). The outbox worker still sorts leniently.
+- New `ModuleMeta.optional_depends_on`: modules to sort after when loaded,
+  never required (#501). `users`, `tenants`, `file_storage`,
+  `background_tasks`, `site_lock`, `api_tokens` and `keycloak` move their
+  ordering-only `Settings` entry there, so the boot order is unchanged and
+  the `settings` plugin can still be disabled (#500).
 - The module-settings mechanism moved from the `settings` plugin into the
   framework (#500): `register_module_settings` / `ModuleSettingsRegistry` to
   `simple_module_core.module_settings`, hydration (`hydrate_settings`,

CLAUDE.md

@@ -118,8 +118,11 @@ until added.
 **Discovery.** Modules declare `[project.entry-points.simple_module]` pointing
 at a `ModuleBase` subclass with `meta = ModuleMeta(...)`.
 `simple_module_core.discovery.discover_modules()` loads them, topologically sorts
-by `ModuleMeta.depends_on`, and the host calls each `register_*` hook in that
-order (`docs/framework/discovery.md`).
+by `ModuleMeta.depends_on` (plus `optional_depends_on`, ordering only, may be
+absent), and the host calls each `register_*` hook in that order
+(`docs/framework/discovery.md`). Outside development a missing `depends_on`
+target fails the boot (`MissingDependencyError`, `SM030`); selecting
+`keycloak` drops `Users`, so modules needing it must be left out.
 
 **Two names per module.** `ModuleMeta.name` (scaffold default: PascalCase,
 `FeatureFlags`) keys Inertia render paths, `depends_on` and `SM_MODULES_ENABLED`.
@@ -195,6 +198,12 @@ every Tailwind utility, so keep tokens in `theme.css` and rules in `styles.css`
   delivery (outbox via `background_tasks`): `bus.subscribe_durable` +
   `await bus.publish_durable(event, db)` inside the request transaction; a
   durable handler must not commit `ctx.db`. `docs/framework/events.md`.
+- Users: read accounts through `get_user_directory(app)`
+  (`simple_module_core.user_directory`; `get_many`, `search`, `match_ids`,
+  `stats`), never `users.models` outside `users`; the auth provider installs
+  it in `register_settings`. `DirectoryUser.roles is None` means unknown
+  (Keycloak), not none. Recent activity: `get_activity_feed(app)` (audit_log
+  installs it). `docs/framework/user-directory.md`.
 - Menus: `MenuRegistry.add_provider(fn)` for per-request items; permissions:
   `PermissionRegistry.add_source(name, provider)` + `invalidate_source(name)`.
   `docs/framework/permissions.md`.
@@ -426,7 +435,8 @@ Unlisted test directories are silently never collected.
 `make doctor` and a development boot run the `SM0xx` checks; the table with
 triggers and fixes is `docs/reference/diagnostic-codes.md`. **Production does
 not run the suite** (only `SM025` is reported there). What fails a production
-boot is strict discovery (a bad entry point or missing `meta`), the
+boot is strict discovery (a bad entry point, missing `meta`, or a missing
+`depends_on` target), the
 auth-provider check, placeholder secrets (`SM_SECRET_KEY`, users token
 secrets), a localhost Celery broker, and the `SM010` migration check (except
 on a never-migrated first run, which goes to `/setup`). Catch everything else

docs/.vitepress/config.ts

@@ -123,6 +123,7 @@ export default defineConfig({
             { text: "Settings & app.state", link: "/framework/settings" },
             { text: "Permissions", link: "/framework/permissions" },
             { text: "Principal resolvers", link: "/framework/principal-resolvers" },
+            { text: "User directory", link: "/framework/user-directory" },
             { text: "Public routes", link: "/framework/public-routes" },
             { text: "Events", link: "/framework/events" },
             { text: "Cache invalidation", link: "/framework/invalidation" },

docs/framework/discovery.md

@@ -32,6 +32,7 @@ Failures:
 - **Missing `meta`** — emits `SM001`. Strict mode: raises.
 - **Duplicate `meta.name`** — emits `SM008` (error). Always fails boot, even in dev, because names are used as table-name prefixes.
 - **Cycle in `depends_on`** — raises `CircularDependencyError`. Always fails boot.
+- **Missing `depends_on` target** (a module that is not installed, not enabled, or dropped because another auth provider was selected) — outside development, `topological_sort(strict=True)` raises one `MissingDependencyError` (an `InvalidModuleError`) naming every `(module, dependency)` pair, and the boot fails. In development it is logged per pair and reported as `SM030` (a warning at boot, an error in `make doctor`). The outbox worker sorts leniently. GH #501.
 
 ## ModuleMeta
 
@@ -42,7 +43,8 @@ meta = ModuleMeta(
     name="Orders",  # PascalCase, globally unique
     route_prefix="/api/orders",  # where the API router mounts
     view_prefix="/orders",  # where the view router mounts
-    depends_on=["Products"],  # hard ordering requirements
+    depends_on=["Products"],  # required, and sorted after
+    optional_depends_on=["Settings"],  # sorted after when loaded; may be absent
     version="1.0.0",  # semver for the module
     i18n_audience="public",  # who gets this module's locale catalog
 )
@@ -67,7 +69,11 @@ A list of other modules' `meta.name` values that **must be loaded first**. The f
 - Your module's `register_routes` reads another module's service to compose endpoints.
 - Your module subscribes to events published by another module.
 
-`depends_on` is **not** a dependency-injection mechanism. It only controls order — you still `import` the services you need.
+`depends_on` is **not** a dependency-injection mechanism. It controls order and presence — you still `import` the services you need. Outside development a missing entry refuses the boot (see Failures above), so list only modules yours cannot work without.
+
+### `optional_depends_on`
+
+Modules to sort after **when they are loaded**, for an ordering-only relationship: your middleware wraps outside theirs, or you subscribe to an event they publish, but your module works without them. Present modules order exactly as `depends_on` would, so moving an entry here never changes the boot order; an absent one is fine in every environment. The modules that sorted after `Settings` only for ordering (`users`, `tenants`, `file_storage`, `background_tasks`, `site_lock`, `api_tokens`, `keycloak`) declare it here, because the settings mechanism lives in the framework (#500) and the `settings` plugin may be disabled.
 
 ### `version`
 

docs/framework/user-directory.md

@@ -0,0 +1,93 @@
+# User directory and activity feed
+
+Which module holds the user accounts depends on the auth provider: `users`
+keeps them in `users_user`, `keycloak` keeps a cache of everyone who has signed
+in. A module that only needs to *name* a user, look a few up, search them or
+count them reads them through the **user directory**, a protocol in
+`simple_module_core.user_directory`, and never imports `users.models`. That
+keeps it working under either provider instead of silently querying an empty
+table on a Keycloak install (GH #501).
+
+## Reading users
+
+```python
+from simple_module_core.user_directory import get_user_directory
+
+directory = get_user_directory(request.app)
+found = await directory.get_many(db, ["3f0c…", "celery-worker"])  # {id: DirectoryUser}
+```
+
+| Member | Returns |
+|---|---|
+| `get_many(db, ids, *, with_roles=False)` | `{id: DirectoryUser}` keyed by **your** id strings. Ids that are not uuids or name nobody are absent, never an error. Implementations chunk (`GET_MANY_CHUNK`, 500 ids per query). |
+| `search(db, term, *, limit)` | Users whose name or email contains `term`, ordered by email; everyone up to `limit` when `term` is empty. |
+| `match_ids(term)` | An **unexecuted** `SELECT` of the matching text ids, for use as an unbounded subquery (the audit log's Actor filter). `None` when the provider cannot express it. |
+| `stats(db, *, active_window)` | `DirectoryStats(total, active_since, created_this_month)`; a `None` field means "cannot say", not zero. |
+| `entity_type` | The audit-link key user rows are recorded under (`"User"`), or `None` when they have no audited model to link to. |
+
+`DirectoryUser` carries `id`, `email`, `full_name`, `is_active` (false for a
+disabled account too) and `roles`: a tuple of role names when `with_roles=True`
+and the provider knows them, otherwise `None`. **`None` means unknown, not "no
+roles"**: Keycloak keeps roles in the IdP, so code that would act on them
+(personal API tokens) must refuse rather than treat the user as role-less.
+
+Name a user with `DirectoryUser.label` (the full name, else the email). It is
+the one naming rule, so a row is never called two things on one screen.
+`resolve_user_labels(directory, db, ids)` maps a page of ids to labels in one
+call; `parse_user_ids` and `chunked` help an implementation key and bound its
+queries. `get_many` is keyed by the caller's strings: `parse_user_ids` maps each
+uuid to every spelling the caller passed (`ABC…`, `abc…`, unhyphenated), and an
+implementation returns an entry under each one.
+
+## Providers
+
+The active auth provider installs its directory in `register_settings` with
+`set_user_directory(app, directory)`:
+
+| Provider | Directory | Knows |
+|---|---|---|
+| `users` | `users.directory.UsersDirectory` | every local account, roles via `users_user_role`, all three stats |
+| `keycloak` | `keycloak.directory.KeycloakDirectory` | users who have signed in at least once (`keycloak_user_cache`); roles `None`; `created_this_month` `None`; `entity_type` `None` |
+| none | `NullUserDirectory` | nobody; every stat `None`; `match_ids` `None` |
+
+A new provider implements the four methods and `entity_type`, and registers
+it. Ids must come back in the canonical hyphenated text form the rest of the
+app stores (`str(uuid)`); `KeycloakDirectory` shows how to render that on
+SQLite, where SQLAlchemy's `Uuid` is 32 bare hex characters.
+
+## Who uses it
+
+| Module | Reads |
+|---|---|
+| `audit_log` | actor names (`resolve_actors`), the Actor search (`actor_filter`, a subquery via `match_ids`, an exact id match without one) and actor links (keyed by `entity_type`) |
+| `dashboard` | the user tiles from `stats()`; a `None` tile is hidden |
+| `file_storage` | uploader names |
+| `api_tokens` | owner checks and emails; personal-token resolution via `get_many(with_roles=True)` |
+| `permissions` | the user side of the grants screens (`get_many(with_roles=True)`, `search`) |
+
+`permissions` role pages still depend on `Users`: roles have no directory
+protocol yet (the `Role` table is read from `users.contracts.roles`).
+`api_tokens` lists `Users` in `optional_depends_on` only: a personal token
+needs the owner's roles, so under Keycloak it is refused while service keys
+keep working.
+
+## Activity feed
+
+The users edit page shows what an account did lately. That comes from the
+audit log, but `users` and `audit_log` must not import each other (each is
+optional to the other, and importing both ways was a cycle). So
+`simple_module_core.activity_feed` defines an `ActivityFeed` protocol:
+
+```python
+from simple_module_core.activity_feed import get_activity_feed
+
+feed = get_activity_feed(request.app)  # None: nothing records activity
+items = await feed.recent_for_actor(db, user_id, limit=6, permissions=granted)
+```
+
+Each `ActivityItem` has `at`, `action`, `entity_type`, `entity_id`,
+`entity_label` (or `None` when the owner cannot name the row, or the reader
+lacks its `AuditLink.label_permission`) and the raw `changes`. `audit_log`
+installs the feed in `register_settings`; `set_activity_feed(app, None)`
+removes it. A missing feed renders no card, which is a different claim from
+an empty one.

docs/modules/api_tokens.md

@@ -13,7 +13,7 @@ rest, shown once, and revocable with effect on the next request.
 | `route_prefix` | `/api/api_tokens` |
 | `view_prefix` | `/api-tokens` |
 | `admin_view_prefix` | `/admin/api-tokens` |
-| `depends_on` | `Auth`, `Users`, `Settings` |
+| `depends_on` | `Auth` (optional: `Users`, `Settings`) |
 
 Installed in this repository's host. It is **not** part of the `smpy new`
 scaffold; add `simple_module_api_tokens` to a host's dependencies to enable it,
@@ -104,7 +104,7 @@ everywhere) authenticate through the session cookie only and never see a token.
 | Expiry is required and at most `max_lifetime_days` (setting, default 365) | service |
 | Service key roles: platform roles only — never `admin`, never `tenant:*` | admin endpoint + service + resolver |
 | Only a **cookie session** may create, list or revoke tokens: `auth_kind == "session"` and no `Authorization` header. A token, a `users` access token or a Keycloak JWT gets 403 | every endpoint and page |
-| The creator must have a local `users_user` row the resolver can load (under Keycloak, a cached user); otherwise 400 `api_tokens.errors.account_required` | personal create endpoint |
+| The creator must be an active account in the app's [user directory](/framework/user-directory) whose roles are known; otherwise 400 `api_tokens.errors.account_required`. Under Keycloak roles live in the IdP, so personal tokens are refused there | personal create endpoint |
 | Cookie-authenticated mutations require `X-CSRF-Token` (`RequiresCsrf`) | every mutation |
 
 ## Tenancy

docs/modules/audit_log.md

@@ -9,7 +9,7 @@ Automatic field-level audit trail for every SQLModel entity in the app. Each cre
 | `name` | `AuditLog` |
 | `route_prefix` | `/api/audit_log` |
 | `view_prefix` | `/admin/audit-log` |
-| `depends_on` | `["Users"]` |
+| `depends_on` | `[]` |
 
 ## How capture works
 
@@ -155,6 +155,8 @@ An entry stores a model class name and a primary key, which proves what happened
 
 **The entity column's names are permission-gated.** An `AuditLink` may declare a `label_permission`, and a reader who does not hold it gets the id instead of the name. `users` declares `users.manage`, because a user row is named by `full_name or email` — so without the gate, `audit_log.view` alone was a way to read the display name of every account that appears in the log, and the email address of every account still holding an unaccepted invite (those have no `full_name` yet). The entry itself is never withheld: the action, the type, the timestamp and the id are the audit trail. Links whose owner declares nothing — setting keys, filenames, task names, flag names — are shown to anyone who may read the trail.
 
+Actor names come from the [user directory](/framework/user-directory) (`DirectoryUser.label`), so they follow whichever auth provider holds the accounts. The Actor search box matches a uuid exactly and anything else against names and emails as a database subquery (`match_ids`), with no ceiling on how many accounts match; with no directory it matches the term as an exact actor id.
+
 **The actor column's names are not.** `audit_log.view` does imply "may see who acted", and that is deliberate: an audit trail that will not say who performed a change is not one. The two columns disclose the same field and answer different questions — the entity column names people who were merely *edited*, which reviewing the trail does not require. Grant `audit_log.view` knowing it discloses the display name (or email) of everyone who has ever made a change.
 
 The `changes` column is a third surface, and a wider one: it records the values that were written, so a `User` create entry contains the email that was set. Use `__audit_exclude_fields__` on a model to keep a column out of the diff (see § Opting out).
@@ -192,4 +194,4 @@ The module is discovered automatically once `simple_module_audit_log` is install
 pip install simple_module_audit_log
 ```
 
-It depends on the [`users`](/modules/users) module (for the menu/permission wiring and the request-scoped `user_id`).
+It depends on no other module. Actor names, the Actor search and actor links come from the app's [user directory](/framework/user-directory), so they work under `users`, `keycloak` (users who have signed in) or no provider at all (raw ids). The module also installs the core [activity feed](/framework/user-directory#activity-feed) the users edit page reads.

docs/modules/background_tasks.md

@@ -9,7 +9,7 @@ Celery + Redis task queue with persistent history, an admin UI for monitoring +
 | `name` | `BackgroundTasks` |
 | `route_prefix` | `/api/background_tasks` |
 | `view_prefix` | `/admin/background-tasks` |
-| `depends_on` | `["Users"]` |
+| `depends_on` | `[]` (optional: `["Settings"]`) |
 
 ## Writing a task