diff --git a/CHANGELOG.md b/CHANGELOG.md index db0fc4ac3..9cc64a0ea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,15 @@ All notable changes to this project are documented in this file. The format is b installed, otherwise run the durable handlers inline in `db`'s transaction (#583). ### Changed +- Users: lockout, registration and bulk-invite writes that must land before the request ends + now commit through `simple_module_db.independent_session` (its own short-lived session) + instead of committing the request session. Failed or unverified password logins now roll + back the request session before the lockout bookkeeping is checkpointed. Lockout, + invite and demo code moved into internal subpackages (#592). +- Users login lockout raises `LoginLockedError`, mapped to the same 429 body and `Retry-After` + by a registered exception handler; settings errors now share a `SettingsError` base (their + HTTP mapping is unchanged). The convention is in + `docs/module-authoring.md` and the module scaffold (#595). - Packaging: **Behaviour change.** Published packages now cap their third-party runtime dependencies (next major for 1.x+ libraries, next minor for 0.x ones such as `sqlmodel`), so an install can no longer resolve an untested major release (#600). diff --git a/docs/database/sessions.md b/docs/database/sessions.md index 5b1b65bf9..13013d839 100644 --- a/docs/database/sessions.md +++ b/docs/database/sessions.md @@ -112,8 +112,9 @@ At callback time the original transaction is already durable, so callback errors cannot roll it back. The framework logs a `db.session.on_commit_failed` error, continues with remaining callbacks, and preserves the successful response. -`on_commit()` belongs to the framework-managed request lifecycle. For a manually -managed session, perform post-commit work explicitly after the transaction block. +`on_commit()` is supported by framework-managed requests and independent +checkpoints. For other manually managed sessions, perform post-commit work +explicitly after the transaction block. ## Manual transactions @@ -133,20 +134,41 @@ async def worker(db: DatabaseState): The `async with session.begin()` pattern opens a transaction you control. Use it in code that runs **outside** a request scope. -### `independent_session(app)` +### `independent_session(source)` -For the common case, a short unit of work outside the request that should commit when it succeeds and roll back when it fails, use `independent_session`: +Use an independent checkpoint when a write must be durable before continuing, +including before a request can fail. Examples are reserving a login attempt +before checking its password and saving invitation roles before sending mail. +Ordinary request writes still belong to `CommitBeforeResponseMiddleware`. ```python from simple_module_db import independent_session -async def on_startup(self, app): +async def reserve_attempt(app): async with independent_session(app) as db: - await replay_rows(db) # committed on clean exit, rolled back on error + await reserve_slot(db) + # The slot is durable here, even if subsequent authentication fails. ``` -It opens a session from `app.state.sm.db.session_factory`, commits when the block exits cleanly, and rolls back and re-raises otherwise. Do not call `commit()` inside the block. Keep using `session_factory()` directly when the block is read-only, must commit mid-way (for example to do follow-up work after the commit, as the branding image reaper does), or runs against another engine. +The source can be an app, its `app.state.sm.db.session_factory`, or a +framework-created session (which supplies that same factory). The helper opens +and closes its own short-lived session, commits on clean exit, and rolls back +and re-raises on error. It never commits or transfers the source session's +pending work. Do not call `commit()` inside the block. + +On SQLite, an independent session cannot write while the request session holds uncommitted writes (there is only one writer), so call it before the request writes, as users lockout and bulk invite do. + +`on_commit` callbacks run in registration order after the checkpoint commits; +rollback or commit failure discards them. Callback failures are logged without +undoing the durable checkpoint, as for request sessions. + +A checkpoint cannot see uncommitted request rows. Pass identifiers and reload +objects in the independent session rather than sharing attached ORM objects. +Keep checkpoints short and open them before the request holds a write +transaction: SQLite permits only one writer, and overlapping writes can block. +Use `session_factory()` directly for read-only work or transactions needing +several explicit commit boundaries. ## Sessions and `MultiTenantMixin` diff --git a/docs/framework/request-guards.md b/docs/framework/request-guards.md index fca535799..9faad6f70 100644 --- a/docs/framework/request-guards.md +++ b/docs/framework/request-guards.md @@ -166,7 +166,7 @@ which counts, a store has `peek(key, spec)`, which reports the current window without counting (`allowed` answers "would one more hit be within budget"), `reset(key)`, which forgets a window, and `hit(..., sliding=True)`, which restarts the window on every hit. The users login lockout -(`users/auth_local/lockout.py`) uses all three. Keys are +(`users/auth_local/lockout/`) uses all three. Keys are namespaced by the caller (`login:…`); Redis keys get the `sm:rl:` prefix. ### Response shape diff --git a/docs/module-authoring.md b/docs/module-authoring.md index 3f7567372..5172a581b 100644 --- a/docs/module-authoring.md +++ b/docs/module-authoring.md @@ -85,6 +85,20 @@ canonical example is `file_storage.StorageBackend`: it has a registry, two shipped implementations (`FilesystemBackend`, `S3Backend`), and tests that mock against the Protocol. If none of those apply to your module, skip it. +### Service errors: domain exceptions + +Services raise domain exceptions derived from a per-module base error in +`/exceptions.py` or `errors.py`; they must not raise `HTTPException`. +Keep the failure and its context independent of HTTP so CLI commands, tasks +and durable handlers can reuse the same business logic. + +Follow `file_storage`: implement the HTTP mapping in the endpoint layer or +install handlers from `register_exception_handlers(app)`. Preserve each +endpoint's status, JSON body, headers and translation keys when migrating an +existing service. Translate messages at the request boundary. Endpoint-level +`HTTPException` remains appropriate for pure request validation. Inertia forms +may catch domain errors to report field errors or redirect instead. + ### Minimal `pyproject.toml` ```toml diff --git a/framework/cli/simple_module_cli/templates/module/__PACKAGE__/service.py.tpl b/framework/cli/simple_module_cli/templates/module/__PACKAGE__/service.py.tpl index 3acf774d9..dc9a6e8b3 100644 --- a/framework/cli/simple_module_cli/templates/module/__PACKAGE__/service.py.tpl +++ b/framework/cli/simple_module_cli/templates/module/__PACKAGE__/service.py.tpl @@ -1,4 +1,10 @@ -"""{{SINGULAR_CLASS}} service implementation.""" +"""{{SINGULAR_CLASS}} service implementation. + +Business failures raise domain exceptions derived from a per-module base in +exceptions.py or errors.py, never HTTPException. Map them in endpoints or via +ModuleBase.register_exception_handlers; keep HTTP status, payload, headers and +translation at that boundary so this service also works in CLI/tasks. +""" from __future__ import annotations diff --git a/framework/db/simple_module_db/independent.py b/framework/db/simple_module_db/independent.py index d0b619351..685478cb6 100644 --- a/framework/db/simple_module_db/independent.py +++ b/framework/db/simple_module_db/independent.py @@ -1,4 +1,4 @@ -"""A session opened outside the request, with the commit convention in one place.""" +"""Short-lived durable transactions, independent of the request unit of work.""" from __future__ import annotations @@ -6,29 +6,47 @@ from contextlib import asynccontextmanager from typing import Any -from sqlalchemy.ext.asyncio import AsyncSession +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker + +from simple_module_db.callbacks import discard_on_commit_callbacks, run_on_commit_callbacks +from simple_module_db.session import SESSION_FACTORY_KEY, RequestSession @asynccontextmanager -async def independent_session(app: Any) -> AsyncGenerator[AsyncSession]: - """Open a session from ``app``'s factory, outside any request. +async def independent_session(source: Any) -> AsyncGenerator[RequestSession]: + """Commit an independent session on success; roll back and re-raise on error. - Commits when the block exits cleanly and rolls back (then re-raises) when it - raises, so startup hooks, background cleanups and out-of-band writes do not - each hand-roll the pair. The session is closed either way. + Accept an app or session factory. An existing session can also supply its + framework session factory; its pending work is never transferred. + Request code may use this for a deliberate durable checkpoint, whose writes + must survive a later request failure. The new session cannot see pending + request writes. Keep checkpoints short and avoid holding a write transaction + on the same database while opening one (especially on SQLite). - Request code must not use this: the request session (``get_db``) commits in - ``CommitBeforeResponseMiddleware``, and a second session cannot see its - uncommitted rows. Code that must commit mid-block, never commit, or run on a - different engine keeps using ``session_factory()`` directly. + Post-commit callbacks run in registration order after the commit succeeds. + The session closes on every exit, including cancellation. """ - async with app.state.sm.db.session_factory() as session: + if isinstance(source, async_sessionmaker): + factory = source + elif isinstance(source, AsyncSession): + try: + factory = source.info[SESSION_FACTORY_KEY] + except KeyError: + raise TypeError( + "independent_session(session) needs a session opened by the framework's " + "session factory; pass the app or the async_sessionmaker instead" + ) from None + else: + factory = source.state.sm.db.session_factory + async with factory() as session: try: yield session await session.commit() - except Exception: - await session.rollback() + except BaseException as exc: + discard_on_commit_callbacks(session) + # Closing the session handles cancellation without masking it with + # another cancellable await; ordinary failures roll back explicitly. + if isinstance(exc, Exception): + await session.rollback() raise - # Cancellation and other BaseExceptions skip the explicit rollback: awaiting - # it there could be cancelled too and mask the original error, and closing - # the session discards the open transaction anyway. + await run_on_commit_callbacks(session) diff --git a/framework/db/simple_module_db/session.py b/framework/db/simple_module_db/session.py index 0aa9e2d1f..735765b7b 100644 --- a/framework/db/simple_module_db/session.py +++ b/framework/db/simple_module_db/session.py @@ -19,6 +19,8 @@ from simple_module_db.provider import DatabaseProvider, detect_provider from simple_module_db.tenancy import DEFAULT_TENANT_ID +SESSION_FACTORY_KEY = "sm_session_factory" + class RequestSession(AsyncSession): """Async session with hooks for the request-owned transaction.""" @@ -135,6 +137,8 @@ def init_db( sync_session_class=scoped_session_class, ) + session_factory.configure(info={SESSION_FACTORY_KEY: session_factory}) + return DatabaseState( engine=engine, session_factory=session_factory, diff --git a/framework/db/simple_module_db/transaction.py b/framework/db/simple_module_db/transaction.py index 92a2c41c1..971872a25 100644 --- a/framework/db/simple_module_db/transaction.py +++ b/framework/db/simple_module_db/transaction.py @@ -130,11 +130,13 @@ async def finalize_session(session: AsyncSession) -> None: async def rollback_session(session: AsyncSession) -> None: """Roll ``session`` back and drop its pending work. Idempotent.""" discard_on_commit_callbacks(session) - if session.info.get(_FINALIZED_KEY) and not _has_pending(session): + has_pending = _has_pending(session) + if session.info.get(_FINALIZED_KEY) and not has_pending: return _settle(session) await session.rollback() - logger.warning( + logger.log( + logging.WARNING if has_pending else logging.DEBUG, "db.session.rollback", extra={"operation": "rollback", "db_duration_ms": _elapsed_ms(session)}, ) diff --git a/framework/db/tests/test_db_independent_checkpoint.py b/framework/db/tests/test_db_independent_checkpoint.py new file mode 100644 index 000000000..de70eb026 --- /dev/null +++ b/framework/db/tests/test_db_independent_checkpoint.py @@ -0,0 +1,72 @@ +"""Independent checkpoints own their transaction and post-commit callbacks.""" + +from types import SimpleNamespace + +import pytest +from simple_module_db import independent_session +from sqlalchemy import text + + +@pytest.fixture +async def checkpoint_db(db_state): + async with db_state.engine.begin() as connection: + await connection.execute(text("CREATE TABLE checkpoint_row (id integer primary key)")) + yield db_state + async with db_state.engine.begin() as connection: + await connection.execute(text("DROP TABLE checkpoint_row")) + + +async def test_factory_checkpoint_runs_callbacks_after_durable_commit(checkpoint_db): + observed = [] + + async def observe(): + async with checkpoint_db.session_factory() as reader: + observed.append((await reader.execute(text("SELECT id FROM checkpoint_row"))).scalar()) + + async with independent_session(checkpoint_db.session_factory) as session: + await session.execute(text("INSERT INTO checkpoint_row VALUES (1)")) + session.on_commit(observe) + session.on_commit(lambda: observed.append("second")) + assert observed == [1, "second"] + + +async def test_app_checkpoint_discards_callbacks_on_failure(checkpoint_db): + app = SimpleNamespace(state=SimpleNamespace(sm=SimpleNamespace(db=checkpoint_db))) + observed = [] + with pytest.raises(RuntimeError, match="abort"): + async with independent_session(app) as session: + await session.execute(text("INSERT INTO checkpoint_row VALUES (1)")) + session.on_commit(lambda: observed.append("committed")) + raise RuntimeError("abort") + assert observed == [] + async with checkpoint_db.session_factory() as reader: + assert (await reader.execute(text("SELECT count(*) FROM checkpoint_row"))).scalar() == 0 + + +async def test_session_checkpoint_leaves_request_work_and_callbacks_uncommitted(tmp_path): + from simple_module_db import init_db + from sqlmodel import Field, SQLModel + + class CheckpointRow(SQLModel, table=True): + __tablename__ = "db_checkpoint_isolation" + id: int = Field(primary_key=True) + + state = init_db(f"sqlite+aiosqlite:///{tmp_path / 'checkpoint.db'}") + observed = [] + try: + async with state.engine.begin() as connection: + await connection.run_sync(CheckpointRow.__table__.create) + async with state.session_factory() as request: + request.add(CheckpointRow(id=1)) + request.on_commit(lambda: observed.append("request")) + async with independent_session(request) as checkpoint: + assert checkpoint is not request + checkpoint.add(CheckpointRow(id=2)) + checkpoint.on_commit(lambda: observed.append("checkpoint")) + assert observed == ["checkpoint"] + await request.rollback() + async with state.session_factory() as reader: + rows = (await reader.execute(text("SELECT id FROM db_checkpoint_isolation"))).scalars() + assert list(rows) == [2] + finally: + await state.engine.dispose() diff --git a/framework/db/tests/test_rollback_logging.py b/framework/db/tests/test_rollback_logging.py new file mode 100644 index 000000000..89dff180b --- /dev/null +++ b/framework/db/tests/test_rollback_logging.py @@ -0,0 +1,23 @@ +"""Only rollbacks that discard writes warrant a warning.""" + +import logging + +import pytest +from _models import _TxnThing +from simple_module_db import mark_written +from simple_module_db.transaction import rollback_session +from sqlalchemy.ext.asyncio import AsyncSession + + +@pytest.mark.parametrize("pending", ["none", "new", "written"]) +async def test_rollback_log_level_matches_pending_work(caplog, pending): + async with AsyncSession() as session: + if pending == "new": + session.add(_TxnThing(name="discarded")) + elif pending == "written": + mark_written(session) + with caplog.at_level(logging.DEBUG, logger="simple_module.db"): + await rollback_session(session) + records = [record for record in caplog.records if record.message == "db.session.rollback"] + assert len(records) == 1 + assert records[0].levelno == (logging.DEBUG if pending == "none" else logging.WARNING) diff --git a/modules/settings/settings/_managed_keys.py b/modules/settings/settings/_managed_keys.py index 233d9fe18..391ab1a41 100644 --- a/modules/settings/settings/_managed_keys.py +++ b/modules/settings/settings/_managed_keys.py @@ -25,11 +25,12 @@ from settings.constants import ERR_MANAGED_KEY_DELETE from settings.contracts.registry import SettingsRegistry, clear_route from settings.contracts.schemas import SettingScope +from settings.exceptions import SettingsError TenantIsLive = Callable[[str], Awaitable[bool]] -class ManagedKeyError(Exception): +class ManagedKeyError(SettingsError): """The key is cleared through ``clear_via``, not by deleting its row.""" def __init__(self, key: str, clear_via: str) -> None: diff --git a/modules/settings/settings/_unique_write.py b/modules/settings/settings/_unique_write.py index 65e6cb3a6..de8fcbcfd 100644 --- a/modules/settings/settings/_unique_write.py +++ b/modules/settings/settings/_unique_write.py @@ -12,8 +12,10 @@ from sqlalchemy.exc import IntegrityError from sqlalchemy.ext.asyncio import AsyncSession +from settings.exceptions import SettingsError -class DuplicateSettingError(Exception): + +class DuplicateSettingError(SettingsError): """A row with this (scope, scope_id, key) already exists.""" diff --git a/modules/settings/settings/exceptions.py b/modules/settings/settings/exceptions.py new file mode 100644 index 000000000..514a3fd84 --- /dev/null +++ b/modules/settings/settings/exceptions.py @@ -0,0 +1,5 @@ +"""Transport-independent settings business failures.""" + + +class SettingsError(Exception): + """Base for errors raised by settings services.""" diff --git a/modules/settings/tests/test_settings_domain_errors.py b/modules/settings/tests/test_settings_domain_errors.py new file mode 100644 index 000000000..84e023820 --- /dev/null +++ b/modules/settings/tests/test_settings_domain_errors.py @@ -0,0 +1,10 @@ +"""Settings business failures share a transport-independent base.""" + + +def test_settings_errors_share_a_module_base(): + from settings._managed_keys import ManagedKeyError + from settings._unique_write import DuplicateSettingError + from settings.exceptions import SettingsError + + assert isinstance(ManagedKeyError("key", "/clear"), SettingsError) + assert isinstance(DuplicateSettingError("key"), SettingsError) diff --git a/modules/users/tests/test_demo_account.py b/modules/users/tests/demo/test_users_demo_account.py similarity index 100% rename from modules/users/tests/test_demo_account.py rename to modules/users/tests/demo/test_users_demo_account.py diff --git a/modules/users/tests/test_demo_guard.py b/modules/users/tests/demo/test_users_demo_guard.py similarity index 99% rename from modules/users/tests/test_demo_guard.py rename to modules/users/tests/demo/test_users_demo_guard.py index 734c07c80..6a1304a32 100644 --- a/modules/users/tests/test_demo_guard.py +++ b/modules/users/tests/demo/test_users_demo_guard.py @@ -9,7 +9,7 @@ import httpx from users.constants import ADMIN_ROLE_NAME, SESSION_USER_ID_KEY, USER_ROLE_NAME from users.demo import SESSION_DEMO_KEY -from users.demo_guard import DEMO_READ_ONLY_DETAIL +from users.demo.guard import DEMO_READ_ONLY_DETAIL def _demo_route(role: str) -> str: diff --git a/modules/users/tests/test_demo_login.py b/modules/users/tests/demo/test_users_demo_login.py similarity index 100% rename from modules/users/tests/test_demo_login.py rename to modules/users/tests/demo/test_users_demo_login.py diff --git a/modules/users/tests/test_admin_resend_invite.py b/modules/users/tests/invites/test_users_admin_resend_invite.py similarity index 98% rename from modules/users/tests/test_admin_resend_invite.py rename to modules/users/tests/invites/test_users_admin_resend_invite.py index b008c2e7a..73a1fc9cb 100644 --- a/modules/users/tests/test_admin_resend_invite.py +++ b/modules/users/tests/invites/test_users_admin_resend_invite.py @@ -202,7 +202,7 @@ class TestMailerFailureIsNotEchoed: async def test_the_admin_gets_the_link_and_a_generic_reason( self, admin_client, users_app, caplog ): - from users.admin.bulk_invite import MAILER_FAILURE_DETAIL + from users.invites.bulk import MAILER_FAILURE_DETAIL class _BrokenMailer(_SpyMailer): async def send_invite(self, email, token, invited_by_name, message=None): @@ -212,7 +212,7 @@ async def send_invite(self, email, token, invited_by_name, message=None): original = users_app.state.users.mailer users_app.state.users.mailer = _BrokenMailer() try: - with caplog.at_level(logging.ERROR, logger="users.admin.resend_invite"): + with caplog.at_level(logging.ERROR, logger="users.invites.resend"): resp = await admin_client.post(f"/api/users/admin/{user_id}/resend-invite") finally: users_app.state.users.mailer = original diff --git a/modules/users/tests/test_users_bulk_invite.py b/modules/users/tests/invites/test_users_bulk_invite.py similarity index 98% rename from modules/users/tests/test_users_bulk_invite.py rename to modules/users/tests/invites/test_users_bulk_invite.py index 7bc3b9e57..6141ef4d5 100644 --- a/modules/users/tests/test_users_bulk_invite.py +++ b/modules/users/tests/invites/test_users_bulk_invite.py @@ -139,7 +139,7 @@ async def test_empty_list_is_accepted_and_does_nothing( async def test_address_count_is_capped(self, authenticated_client: httpx.AsyncClient): """One submit must not be able to mint unbounded live invite tokens.""" - from users.admin.bulk_invite import MAX_ADDRESSES + from users.invites.bulk import MAX_ADDRESSES emails = [f"bulk{i}@example.com" for i in range(MAX_ADDRESSES + 5)] resp = await authenticated_client.post(_URL, json={"emails": emails, "role_names": []}) @@ -151,7 +151,7 @@ async def test_addresses_over_the_cap_are_reported_not_dropped( ): """Silent truncation reports "100 invites sent" while 5 people are never contacted, with nothing on screen saying so.""" - from users.admin.bulk_invite import MAX_ADDRESSES + from users.invites.bulk import MAX_ADDRESSES emails = [f"over{i}@example.com" for i in range(MAX_ADDRESSES + 5)] resp = await authenticated_client.post(_URL, json={"emails": emails, "role_names": []}) diff --git a/modules/users/tests/test_bulk_invite_message.py b/modules/users/tests/invites/test_users_bulk_invite_message.py similarity index 100% rename from modules/users/tests/test_bulk_invite_message.py rename to modules/users/tests/invites/test_users_bulk_invite_message.py diff --git a/modules/users/tests/test_invite_flow.py b/modules/users/tests/invites/test_users_invite_flow.py similarity index 100% rename from modules/users/tests/test_invite_flow.py rename to modules/users/tests/invites/test_users_invite_flow.py diff --git a/modules/users/tests/test_invite_reuse.py b/modules/users/tests/invites/test_users_invite_reuse.py similarity index 100% rename from modules/users/tests/test_invite_reuse.py rename to modules/users/tests/invites/test_users_invite_reuse.py diff --git a/modules/users/tests/test_users_list_invites.py b/modules/users/tests/invites/test_users_list_invites.py similarity index 100% rename from modules/users/tests/test_users_list_invites.py rename to modules/users/tests/invites/test_users_list_invites.py diff --git a/modules/users/tests/test_users_admin_unlock.py b/modules/users/tests/lockout/test_users_admin_unlock.py similarity index 100% rename from modules/users/tests/test_users_admin_unlock.py rename to modules/users/tests/lockout/test_users_admin_unlock.py diff --git a/modules/users/tests/lockout/test_users_lockout_checkpoint.py b/modules/users/tests/lockout/test_users_lockout_checkpoint.py new file mode 100644 index 000000000..6838f9afd --- /dev/null +++ b/modules/users/tests/lockout/test_users_lockout_checkpoint.py @@ -0,0 +1,114 @@ +"""Checkpoint isolation on file SQLite, where overlapping writers cannot hide.""" + +from datetime import UTC, datetime + +import pytest +from _users_app_builders import ALICE, _build_users_app, _client, _make_standard_user, user_row +from pwdlib.hashers.bcrypt import BcryptHasher +from simple_module_core.rate_limit import InProcessWindowStore +from sqlalchemy import select +from users.auth_local.lockout import LoginAttempt, admit +from users.models import Role, User + + +@pytest.fixture +async def checkpoint_app(monkeypatch, tmp_path): + app, context = await _build_users_app( + monkeypatch, + allow_signup=False, + database_url=f"sqlite+aiosqlite:///{tmp_path / 'auth-checkpoint.db'}", + ) + await _make_standard_user(app, ALICE) + try: + yield app + finally: + await context.__aexit__(None, None, None) + + +async def test_admission_never_flushes_or_commits_unrelated_request_work(checkpoint_app): + observed = [] + factory = checkpoint_app.state.sm.db.session_factory + attempt = LoginAttempt.build( + ALICE, + "1.2.3.4", + InProcessWindowStore(), + checkpoint_app.state.users.settings, + datetime.now(UTC), + ) + async with factory() as request: + pending = Role(name="pending-request-role") + request.add(pending) + request.on_commit(lambda: observed.append("request")) + admission = await admit(request, attempt) + assert admission.count == 1 + assert pending in request.new + assert observed == [] + await request.rollback() + assert (await user_row(checkpoint_app, ALICE)).failed_login_count == 1 + async with factory() as reader: + assert ( + await reader.execute(select(Role).where(Role.name == "pending-request-role")) + ).scalar_one_or_none() is None + + +async def test_unverified_legacy_password_releases_slot_without_overlapping_writers(checkpoint_app): + factory = checkpoint_app.state.sm.db.session_factory + async with factory() as session: + user = (await session.execute(select(User).where(User.email == ALICE))).scalar_one() + user.is_verified = False + user.hashed_password = BcryptHasher(rounds=4).hash("UserPass1!") + await session.commit() + checkpoint_app.state.users.settings = checkpoint_app.state.users.settings.model_copy( + update={"require_verification": True} + ) + async with _client(checkpoint_app, "1.2.3.4") as client: + response = await client.post( + "/api/users/auth/token", json={"email": ALICE, "password": "UserPass1!"} + ) + assert response.status_code == 400, response.text + assert response.json()["detail"] == "LOGIN_USER_NOT_VERIFIED" + assert (await user_row(checkpoint_app, ALICE)).failed_login_count == 0 + + +@pytest.mark.parametrize("kind", ["cookie", "token"]) +async def test_unverified_logins_release_request_connections_before_checkpoint( + checkpoint_app, monkeypatch, kind +): + import asyncio + + from users.auth_local import api, token_api + + # SQLite's default pool has five connections plus ten overflow connections. + emails = [f"unverified-{i}@example.com" for i in range(15)] + for email in emails: + await _make_standard_user(checkpoint_app, email) + async with checkpoint_app.state.sm.db.session_factory() as session: + for user in (await session.execute(select(User).where(User.email.in_(emails)))).scalars(): + user.is_verified = False + await session.commit() + checkpoint_app.state.users.settings = checkpoint_app.state.users.settings.model_copy( + update={"require_verification": True} + ) + endpoint = api if kind == "cookie" else token_api + real_release = endpoint.release + ready = asyncio.Barrier(len(emails)) + + async def together(session, attempt, admission): + await ready.wait() + return await real_release(session, attempt, admission) + + monkeypatch.setattr(endpoint, "release", together) + + async def login(email): + async with _client(checkpoint_app, "1.2.3.4") as client: + if kind == "cookie": + return await client.post( + "/api/users/auth/login", data={"username": email, "password": "UserPass1!"} + ) + return await client.post( + "/api/users/auth/token", json={"email": email, "password": "UserPass1!"} + ) + + responses = await asyncio.wait_for(asyncio.gather(*(login(email) for email in emails)), 5) + assert [response.status_code for response in responses] == [400] * 15 + assert all(response.json()["detail"] == "LOGIN_USER_NOT_VERIFIED" for response in responses) diff --git a/modules/users/tests/test_users_lockout_columns.py b/modules/users/tests/lockout/test_users_lockout_columns.py similarity index 100% rename from modules/users/tests/test_users_lockout_columns.py rename to modules/users/tests/lockout/test_users_lockout_columns.py diff --git a/modules/users/tests/test_users_lockout_enumeration.py b/modules/users/tests/lockout/test_users_lockout_enumeration.py similarity index 100% rename from modules/users/tests/test_users_lockout_enumeration.py rename to modules/users/tests/lockout/test_users_lockout_enumeration.py diff --git a/modules/users/tests/test_users_lockout_service.py b/modules/users/tests/lockout/test_users_lockout_service.py similarity index 95% rename from modules/users/tests/test_users_lockout_service.py rename to modules/users/tests/lockout/test_users_lockout_service.py index 426c87103..5d87c3aed 100644 --- a/modules/users/tests/test_users_lockout_service.py +++ b/modules/users/tests/lockout/test_users_lockout_service.py @@ -5,7 +5,6 @@ from datetime import UTC, datetime, timedelta from _users_app_builders import ALICE -from fastapi import HTTPException from simple_module_core.rate_limit import InProcessWindowStore from sqlalchemy import select from users.auth_local import lockout @@ -16,6 +15,7 @@ record_failure, record_success, ) +from users.exceptions import LoginLockedError from users.models import User from users.settings import UsersSettings @@ -38,17 +38,17 @@ async def _fail(session, attempt: LoginAttempt) -> int: """One wrong password: admit, then settle. Returns the status it would answer.""" try: admission = await admit(session, attempt) - except HTTPException as exc: - return exc.status_code + except LoginLockedError: + return 429 locked = await record_failure(session, attempt, admission) - return locked.status_code if locked else 400 + return 429 if locked else 400 async def _admit_or_status(session, attempt: LoginAttempt) -> Admission | int: try: return await admit(session, attempt) - except HTTPException as exc: - return exc.status_code + except LoginLockedError: + return 429 async def _row(users_db, user) -> User: @@ -160,7 +160,7 @@ async def test_success_mid_burst_does_not_lift_a_lock(users_db, alice) -> None: owner_slot = await admit(users_db, owner) # admitted before the lock attacker = _attempt(ip="10.0.0.3", store=store, account=3) locked = await record_failure(users_db, attacker, await admit(users_db, attacker)) - assert locked is not None and locked.status_code == 429 + assert isinstance(locked, LoginLockedError) await record_success(users_db, owner, owner_slot) # the owner's password lands last await users_db.commit() @@ -188,8 +188,8 @@ async def test_lock_landing_after_the_read_reports_the_time_left(users_db, alice mid = _attempt(now=T0 + timedelta(seconds=300), ip="9.9.9.9", account=3) try: await lockout._admit_account(users_db, mid, [None]) # read before the lock landed - except HTTPException as exc: - assert (exc.status_code, exc.headers) == (429, {"Retry-After": "600"}) + except LoginLockedError as exc: + assert exc.retry_after == 600 else: raise AssertionError("a locked row was admitted") diff --git a/modules/users/tests/test_users_login_lockout.py b/modules/users/tests/lockout/test_users_login_lockout.py similarity index 100% rename from modules/users/tests/test_users_login_lockout.py rename to modules/users/tests/lockout/test_users_login_lockout.py diff --git a/modules/users/tests/test_users_default_role.py b/modules/users/tests/test_users_default_role.py index fce5013b1..ad00a5808 100644 --- a/modules/users/tests/test_users_default_role.py +++ b/modules/users/tests/test_users_default_role.py @@ -1,39 +1,24 @@ -"""Which registrations get the standard ``user`` role (GH #488 review). - -OAuth provisioning is not gated by ``allow_signup``, so the role it attaches -must be: with signups closed, an account any provider user can create holds no -role until an admin grants one. -""" +"""Which registrations receive the standard role, using real checkpoint sessions.""" from __future__ import annotations -import uuid from types import SimpleNamespace from unittest.mock import AsyncMock, MagicMock -import pytest -from users.constants import OAUTH_REGISTRATION_REQUEST_FLAG +from sqlalchemy import select +from users.constants import OAUTH_REGISTRATION_REQUEST_FLAG, USER_ROLE_ID +from users.db_adapter import UserDatabaseWithRoles from users.manager import UserManager -from users.models import UserRole -from users.settings import UsersSettings - -_ROLE_ID = uuid.uuid4() +from users.models import OAuthAccount, Role, User, UserRole -def _manager(*, allow_signup: bool) -> UserManager: - settings = UsersSettings( - verification_token_secret="test-verify-secret-at-least-32-bytes-long", - reset_password_token_secret="test-reset-secret-at-least-32-bytes-long", - allow_signup=allow_signup, +def _manager(app, session, *, allow_signup: bool) -> UserManager: + settings = app.state.users.settings.model_copy(update={"allow_signup": allow_signup}) + return UserManager( + UserDatabaseWithRoles(session, User, OAuthAccount), + MagicMock(send_verification=AsyncMock()), + settings, ) - user_db = MagicMock() - user_db.update = AsyncMock() - user_db.session.flush = AsyncMock() - user_db.session.commit = AsyncMock() - user_db.session.execute = AsyncMock( - return_value=MagicMock(scalar_one_or_none=MagicMock(return_value=MagicMock(id=_ROLE_ID))) - ) - return UserManager(user_db, MagicMock(send_verification=AsyncMock()), settings) def _request(*, oauth: bool) -> MagicMock: @@ -43,40 +28,48 @@ def _request(*, oauth: bool) -> MagicMock: return request -def _user() -> MagicMock: - return MagicMock(id=uuid.uuid4(), email="new@example.com", is_verified=True) +async def _user(session) -> User: + user = User(email="new@example.com", is_verified=True) + session.add(user) + await session.commit() # fastapi-users creates the account before invoking its hook + return user -def _added_roles(manager: UserManager) -> list[UserRole]: - return [ - c.args[0] - for c in manager.user_db.session.add.call_args_list - if isinstance(c.args[0], UserRole) - ] +async def _roles(session, user: User) -> list: + return list( + ( + await session.execute(select(UserRole.role_id).where(UserRole.user_id == user.id)) + ).scalars() + ) -@pytest.mark.anyio -async def test_oauth_provisioning_with_signup_closed_gets_no_role(): - manager = _manager(allow_signup=False) - await manager.on_after_register(_user(), _request(oauth=True)) - assert _added_roles(manager) == [] +async def test_oauth_provisioning_with_signup_closed_gets_no_role(users_app, users_db): + manager = _manager(users_app, users_db, allow_signup=False) + user = await _user(users_db) + await manager.on_after_register(user, _request(oauth=True)) + assert await _roles(users_db, user) == [] -@pytest.mark.anyio -async def test_oauth_provisioning_with_signup_open_gets_the_user_role(): - manager = _manager(allow_signup=True) - user = _user() +async def test_oauth_provisioning_with_signup_open_gets_the_user_role(users_app, users_db): + manager = _manager(users_app, users_db, allow_signup=True) + user = await _user(users_db) await manager.on_after_register(user, _request(oauth=True)) - (link,) = _added_roles(manager) - assert (link.user_id, link.role_id) == (user.id, _ROLE_ID) + assert await _roles(users_db, user) == [USER_ROLE_ID] + + +async def test_request_bound_registration_gets_the_user_role(users_app, users_db): + manager = _manager(users_app, users_db, allow_signup=True) + user = await _user(users_db) + await manager.on_after_register(user, _request(oauth=False)) + assert await _roles(users_db, user) == [USER_ROLE_ID] -@pytest.mark.anyio -async def test_local_registration_gets_the_user_role(): - # /register itself 404s while signup is closed, so reaching the hook - # means the operator opened it. - manager = _manager(allow_signup=True) - user = _user() +async def test_a_deleted_user_role_is_not_recreated(users_app, users_db): + role = await users_db.get(Role, USER_ROLE_ID) + await users_db.delete(role) + await users_db.commit() + manager = _manager(users_app, users_db, allow_signup=True) + user = await _user(users_db) await manager.on_after_register(user, _request(oauth=False)) - (link,) = _added_roles(manager) - assert (link.user_id, link.role_id) == (user.id, _ROLE_ID) + assert await _roles(users_db, user) == [] + assert await users_db.get(Role, USER_ROLE_ID) is None diff --git a/modules/users/tests/test_users_domain_errors.py b/modules/users/tests/test_users_domain_errors.py new file mode 100644 index 000000000..c8f741e79 --- /dev/null +++ b/modules/users/tests/test_users_domain_errors.py @@ -0,0 +1,40 @@ +"""Domain failures remain usable outside HTTP request handlers.""" + +import uuid + +from fastapi import HTTPException +from users.auth_local.lockout import too_many_attempts +from users.exceptions import UserNotFoundError + + +def test_lockout_is_a_domain_error(): + error = too_many_attempts(0) + assert not isinstance(error, HTTPException) + assert error.retry_after == 1 + + +def test_user_errors_share_a_module_base(): + from users.exceptions import UsersError + + assert isinstance(UserNotFoundError(uuid.uuid4()), UsersError) + + +async def test_lockout_http_mapping_preserves_body_and_retry_after(): + import httpx + from fastapi import FastAPI + from users.module import UsersModule + + app = FastAPI() + UsersModule().register_exception_handlers(app) + + @app.get("/locked") + async def locked(): + raise too_many_attempts(0) + + async with httpx.AsyncClient( + transport=httpx.ASGITransport(app=app), base_url="http://test" + ) as client: + response = await client.get("/locked") + assert response.status_code == 429 + assert response.content == '{"detail":"Too many attempts — try again later"}'.encode() + assert response.headers["Retry-After"] == "1" diff --git a/modules/users/tests/test_user_manager.py b/modules/users/tests/test_users_manager.py similarity index 86% rename from modules/users/tests/test_user_manager.py rename to modules/users/tests/test_users_manager.py index c95552f12..69ebcaf78 100644 --- a/modules/users/tests/test_user_manager.py +++ b/modules/users/tests/test_users_manager.py @@ -134,39 +134,6 @@ async def test_publishes_user_registered(self, manager, fake_user): assert event.user_id == fake_user.id assert event.email == fake_user.email - @pytest.mark.anyio - async def test_request_bound_registration_gets_the_user_role( - self, manager, fake_user, fake_user_db - ): - """GH #488: a signup with no role holds no permissions at all.""" - from users.models import UserRole - - request = MagicMock() - request.state = MagicMock(spec=[]) # a local signup, not OAuth - request.app.state.sm.event_bus = MagicMock(publish=AsyncMock()) - fake_user.is_verified = True - role = MagicMock(id=uuid.uuid4()) - fake_user_db.session.execute = AsyncMock( - return_value=MagicMock(scalar_one_or_none=MagicMock(return_value=role)) - ) - await manager.on_after_register(fake_user, request) - - (added,) = fake_user_db.session.add.call_args.args - assert isinstance(added, UserRole) - assert (added.user_id, added.role_id) == (fake_user.id, role.id) - - @pytest.mark.anyio - async def test_a_deleted_user_role_is_not_recreated(self, manager, fake_user, fake_user_db): - """An admin who removed the role meant it; a public signup must not undo that.""" - request = MagicMock() - request.app.state.sm.event_bus = MagicMock(publish=AsyncMock()) - fake_user.is_verified = True - fake_user_db.session.execute = AsyncMock( - return_value=MagicMock(scalar_one_or_none=MagicMock(return_value=None)) - ) - await manager.on_after_register(fake_user, request) - fake_user_db.session.add.assert_not_called() - @pytest.mark.anyio async def test_without_request_skips_publish(self, manager, fake_user, fake_user_db): """CLI/bootstrap and admin flows pass request=None — no crash, no role: diff --git a/modules/users/tests/test_users_register_single_connection.py b/modules/users/tests/test_users_register_single_connection.py new file mode 100644 index 000000000..6cbe51ee8 --- /dev/null +++ b/modules/users/tests/test_users_register_single_connection.py @@ -0,0 +1,55 @@ +"""Registration must release its connection before the durable role checkpoint.""" + +import asyncio + +import pytest +from simple_module_hosting.settings import Settings +from sqlalchemy import select +from sqlalchemy.ext.asyncio import create_async_engine +from users.constants import USER_ROLE_NAME +from users.models import Role, User, UserRole + + +@pytest.fixture +def settings(settings: Settings, tmp_path) -> Settings: + database_url = settings.database_url + if database_url.startswith("sqlite"): + database_url = f"sqlite+aiosqlite:///{tmp_path / 'signup.db'}" + return settings.model_copy( + update={"database_url": database_url, "multi_tenant": False, "tenant_header": ""} + ) + + +async def test_signup_attaches_default_role_with_one_connection(app, client): + # Constrain registration after startup, which has its own DB operations. + db = app.state.sm.db + engine = create_async_engine(db.engine.url, pool_size=1, max_overflow=0) + await db.engine.dispose() + db.engine = engine + db.session_factory.configure(bind=engine) + app.state.users.settings = app.state.users.settings.model_copy( + update={"allow_signup": True, "require_verification": False} + ) + email = "single-connection@example.com" + + async with asyncio.timeout(3): + response = await client.post( + "/api/users/auth/register", json={"email": email, "password": "SecurePass1!"} + ) + + assert response.status_code == 201, response.text + assert response.json()["email"] == email + async with app.state.sm.db.session_factory() as session: + roles = ( + ( + await session.execute( + select(Role.name) + .join(UserRole, UserRole.role_id == Role.id) + .join(User, User.id == UserRole.user_id) + .where(User.email == email) + ) + ) + .scalars() + .all() + ) + assert roles == [USER_ROLE_NAME] diff --git a/modules/users/users/admin/api.py b/modules/users/users/admin/api.py index 46ba85e94..590069a09 100644 --- a/modules/users/users/admin/api.py +++ b/modules/users/users/admin/api.py @@ -12,9 +12,7 @@ from simple_module_hosting.pagination import page_params from simple_module_hosting.permissions import RequiresPermission -from users.admin.bulk_invite import bulk_router from users.admin.links import admin_link_base -from users.admin.resend_invite import resend_router from users.admin.service import UserService from users.constants import PERM_USERS_MANAGE, sanitize_list_filters from users.contracts.events import ( @@ -38,6 +36,8 @@ ExternalUserNoPasswordError, UserNotFoundError, ) +from users.invites.bulk import bulk_router +from users.invites.resend import resend_router admin_router = APIRouter( prefix="/admin", diff --git a/modules/users/users/auth_local/api.py b/modules/users/users/auth_local/api.py index 9cd09e716..0f77c672b 100644 --- a/modules/users/users/auth_local/api.py +++ b/modules/users/users/auth_local/api.py @@ -18,6 +18,7 @@ from fastapi_users import exceptions as fu_exceptions from simple_module_core.redirect_safety import SESSION_NEXT_KEY from simple_module_db.deps import get_db +from simple_module_db.transaction import rollback_session from simple_module_hosting.session import SESSION_REMEMBER_KEY, stamp_session_expiry from simple_module_hosting.token_guard import Authenticated from sqlalchemy.ext.asyncio import AsyncSession @@ -114,6 +115,7 @@ async def login( user = None if user is None or not user.is_active: + await rollback_session(db) locked = await record_failure(db, attempt, admission) raise locked or HTTPException(status_code=400, detail="LOGIN_BAD_CREDENTIALS") @@ -121,6 +123,7 @@ async def login( # Match fastapi-users' own behavior when requires_verification=True. # With the setting off nobody is asked to verify (#547) — including # accounts that signed up while it was on. + await rollback_session(db) await release(db, attempt, admission) raise HTTPException(status_code=400, detail="LOGIN_USER_NOT_VERIFIED") diff --git a/modules/users/users/auth_local/lockout.py b/modules/users/users/auth_local/lockout/__init__.py similarity index 63% rename from modules/users/users/auth_local/lockout.py rename to modules/users/users/auth_local/lockout/__init__.py index 6af3feacb..32620169d 100644 --- a/modules/users/users/auth_local/lockout.py +++ b/modules/users/users/auth_local/lockout/__init__.py @@ -28,106 +28,32 @@ from __future__ import annotations -import hashlib import math -import uuid -from dataclasses import dataclass from datetime import UTC, datetime, timedelta from typing import TYPE_CHECKING, Any -from fastapi import HTTPException, Request, status -from simple_module_core.rate_limit import RateSpec, WindowStore +from fastapi import Request +from simple_module_core.rate_limit import RateSpec +from simple_module_db import independent_session from sqlalchemy import case, func, select, update from sqlmodel import col +from users.auth_local.lockout.attempt import Admission, LoginAttempt from users.auth_local.rate_limit import client_ip +from users.exceptions import LOGIN_LOCKED_DETAIL, LoginLockedError from users.models import User if TYPE_CHECKING: from sqlalchemy.ext.asyncio import AsyncSession - from users.settings import UsersSettings -LOCKED_DETAIL = "Too many attempts — try again later" +LOCKED_DETAIL = LOGIN_LOCKED_DETAIL def _utcnow() -> datetime: return datetime.now(UTC) -@dataclass(frozen=True, slots=True) -class LoginAttempt: - """One password login: who, from where, against which store, and when.""" - - username: str - ip: str - store: WindowStore - settings: UsersSettings - now: datetime - - @classmethod - def build( - cls, - username: str, - ip: str, - store: WindowStore, - settings: UsersSettings, - now: datetime, - ) -> LoginAttempt: - # Normalised once, here: ``Alice@x `` and ``alice@x`` are one account - # to the user lookup, so they must be one counter too. - return cls(username.strip().lower(), ip, store, settings, now) - - @property - def threshold(self) -> int: - return self.settings.login_lockout_account_failures - - @property - def cooldown(self) -> int: - return self.settings.login_rate_limit_cooldown_seconds - - @property - def key_name(self) -> str: - """The username as it appears in store keys. - - The body limit is 10 MiB, so an unauthenticated caller could otherwise - park a megabyte-long key in Redis per request. Anything longer than an - address can be (254) is hashed; it matches no account either way. - """ - if len(self.username) <= 254: - return self.username - return hashlib.sha256(self.username.encode()).hexdigest() - - @property - def ip_key(self) -> str: - return f"login:ip:{self.ip}:{self.key_name}" - - @property - def ip_spec(self) -> RateSpec: - s = self.settings - return RateSpec(s.login_rate_limit_failures, s.login_rate_limit_window_seconds) - - @property - def account_key(self) -> str: - return f"login:acct:{self.key_name}" - - @property - def lock_key(self) -> str: - return f"login:acct-lock:{self.key_name}" - - @property - def lock_spec(self) -> RateSpec: - return RateSpec(1, self.cooldown) - - -@dataclass(frozen=True, slots=True) -class Admission: - """A slot taken by :func:`admit`: the account rows it counted against and the count.""" - - user_ids: tuple[uuid.UUID, ...] - count: int - - def login_attempt(request: Request, username: str) -> LoginAttempt: """The :class:`LoginAttempt` for *username* arriving on *request*.""" return LoginAttempt.build( @@ -139,13 +65,9 @@ def login_attempt(request: Request, username: str) -> LoginAttempt: ) -def too_many_attempts(retry_after: int) -> HTTPException: - """The 429 for a locked account or a throttled source — deliberately identical.""" - return HTTPException( - status_code=status.HTTP_429_TOO_MANY_REQUESTS, - detail=LOCKED_DETAIL, - headers={"Retry-After": str(max(1, retry_after))}, - ) +def too_many_attempts(retry_after: int) -> LoginLockedError: + """The same domain failure for a locked account or a throttled source.""" + return LoginLockedError(retry_after) def _by_username(username: str) -> Any: @@ -156,15 +78,15 @@ def _by_username(username: str) -> Any: async def admit(session: AsyncSession, attempt: LoginAttempt) -> Admission: """Take a slot for this attempt in both tiers, or raise the 429. - Commits *session* when it counts against an account row, so other workers - see the slot taken before this one checks the password. The request has - only read by this point, so nothing else is committed with it. + Counts account rows in an independent checkpoint, so other workers see + the slot taken before this one checks the password. """ source = await attempt.store.hit(attempt.ip_key, attempt.ip_spec) if not source.allowed: raise too_many_attempts(source.retry_after) stmt = select(User.locked_until).where(_by_username(attempt.username)) - rows = list((await session.execute(stmt)).scalars().all()) + async with independent_session(session) as checkpoint: + rows = list((await checkpoint.execute(stmt)).scalars().all()) if rows: return await _admit_account(session, attempt, rows) return await _admit_unknown(attempt) @@ -201,15 +123,16 @@ async def _admit_account( .returning(User.id, User.failed_login_count) .execution_options(synchronize_session=False) ) - counted = (await session.execute(stmt)).all() - await session.commit() + async with independent_session(session) as checkpoint: + counted = (await checkpoint.execute(stmt)).all() if not counted: # Locked by another worker since the read above: report the time left, # as the unknown-username lock does, or ``Retry-After`` would tell the # two apart. Otherwise the last slot is taken and its lock is being # written — the full cooldown, as on the unknown path. stmt = select(User.locked_until).where(_by_username(attempt.username)) - locks = list((await session.execute(stmt)).scalars().all()) + async with independent_session(session) as checkpoint: + locks = list((await checkpoint.execute(stmt)).scalars().all()) raise too_many_attempts(_lock_remaining(locks, attempt.now) or attempt.cooldown) return Admission(tuple(row[0] for row in counted), max(row[1] for row in counted)) @@ -231,26 +154,22 @@ async def _admit_unknown(attempt: LoginAttempt) -> Admission: async def record_failure( session: AsyncSession, attempt: LoginAttempt, admission: Admission -) -> HTTPException | None: +) -> LoginLockedError | None: """Settle a wrong password: the 429 to raise if it was the last slot, else ``None``. - The failure itself was counted by :func:`admit`. Locking commits *session*: - the caller answers by raising, and the request's unit of work rolls back on - that path. + The failure itself was counted by :func:`admit`. Locking uses an independent checkpoint: + the caller raises next, rolling back the request unit of work. """ if admission.count < attempt.threshold: return None if admission.user_ids: - for user_id in admission.user_ids: - # Through the ORM, not an UPDATE, so the audit log records the lock. - # Only ``locked_until`` and the count are written, so a concurrent - # counting UPDATE cannot undo it — and none can run: this row has - # no free slot until the lock lifts. - user = await session.get(User, user_id, populate_existing=True) - if user is not None: - user.locked_until = attempt.now + timedelta(seconds=attempt.cooldown) - user.failed_login_count = 0 - await session.commit() + async with independent_session(session) as checkpoint: + for user_id in admission.user_ids: + # ORM writes preserve the audit record for the lock. + user = await checkpoint.get(User, user_id) + if user is not None: + user.locked_until = attempt.now + timedelta(seconds=attempt.cooldown) + user.failed_login_count = 0 else: await attempt.store.hit(attempt.lock_key, attempt.lock_spec) await attempt.store.reset(attempt.account_key) @@ -277,14 +196,14 @@ async def release(session: AsyncSession, attempt: LoginAttempt, admission: Admis """Give back the account slot of an attempt that must not count. For a correct password refused for another reason (an unverified email): - that answer is not a failure. Commits, since the caller raises next. + that answer is not a failure. Checkpoints, since the caller raises next. """ await attempt.store.reset(attempt.ip_key) if admission.user_ids: - await session.execute( - update(User) - .where(col(User.id).in_(admission.user_ids), col(User.failed_login_count) > 0) - .values(failed_login_count=User.failed_login_count - 1, updated_at=User.updated_at) - .execution_options(synchronize_session=False) - ) - await session.commit() + async with independent_session(session) as checkpoint: + await checkpoint.execute( + update(User) + .where(col(User.id).in_(admission.user_ids), col(User.failed_login_count) > 0) + .values(failed_login_count=User.failed_login_count - 1, updated_at=User.updated_at) + .execution_options(synchronize_session=False) + ) diff --git a/modules/users/users/auth_local/lockout/attempt.py b/modules/users/users/auth_local/lockout/attempt.py new file mode 100644 index 000000000..6f0c38e4d --- /dev/null +++ b/modules/users/users/auth_local/lockout/attempt.py @@ -0,0 +1,87 @@ +"""Normalised login attempt keys and admission results.""" + +from __future__ import annotations + +import hashlib +import uuid +from dataclasses import dataclass +from datetime import datetime +from typing import TYPE_CHECKING + +from simple_module_core.rate_limit import RateSpec, WindowStore + +if TYPE_CHECKING: + from users.settings import UsersSettings + + +@dataclass(frozen=True, slots=True) +class LoginAttempt: + """One password login: who, from where, against which store, and when.""" + + username: str + ip: str + store: WindowStore + settings: UsersSettings + now: datetime + + @classmethod + def build( + cls, + username: str, + ip: str, + store: WindowStore, + settings: UsersSettings, + now: datetime, + ) -> LoginAttempt: + # Normalised once, here: ``Alice@x `` and ``alice@x`` are one account + # to the user lookup, so they must be one counter too. + return cls(username.strip().lower(), ip, store, settings, now) + + @property + def threshold(self) -> int: + return self.settings.login_lockout_account_failures + + @property + def cooldown(self) -> int: + return self.settings.login_rate_limit_cooldown_seconds + + @property + def key_name(self) -> str: + """The username as it appears in store keys. + + The body limit is 10 MiB, so an unauthenticated caller could otherwise + park a megabyte-long key in Redis per request. Anything longer than an + address can be (254) is hashed; it matches no account either way. + """ + if len(self.username) <= 254: + return self.username + return hashlib.sha256(self.username.encode()).hexdigest() + + @property + def ip_key(self) -> str: + return f"login:ip:{self.ip}:{self.key_name}" + + @property + def ip_spec(self) -> RateSpec: + s = self.settings + return RateSpec(s.login_rate_limit_failures, s.login_rate_limit_window_seconds) + + @property + def account_key(self) -> str: + return f"login:acct:{self.key_name}" + + @property + def lock_key(self) -> str: + return f"login:acct-lock:{self.key_name}" + + @property + def lock_spec(self) -> RateSpec: + return RateSpec(1, self.cooldown) + + +@dataclass(frozen=True, slots=True) +class Admission: + """A slot taken by :func:`admit`: the account rows it counted against and the count.""" + + user_ids: tuple[uuid.UUID, ...] + count: int diff --git a/modules/users/users/auth_local/token_api.py b/modules/users/users/auth_local/token_api.py index 9a718ddd3..f1d268a88 100644 --- a/modules/users/users/auth_local/token_api.py +++ b/modules/users/users/auth_local/token_api.py @@ -12,6 +12,7 @@ from fastapi import APIRouter, Depends, HTTPException, Request from simple_module_db.deps import get_db +from simple_module_db.transaction import rollback_session from sqlalchemy import func, select from sqlalchemy.ext.asyncio import AsyncSession from sqlmodel import SQLModel, col @@ -90,13 +91,22 @@ async def token_login( # email enumeration (existing user + wrong password takes ~50ms for # bcrypt; missing user would be instant without this). helper.verify_and_update(body.password, _DUMMY_HASH) + await rollback_session(db) locked = await record_failure(db, attempt, admission) raise locked or HTTPException(status_code=401, detail="Invalid credentials") verified, updated_hash = helper.verify_and_update(body.password, user.hashed_password) if not verified: + await rollback_session(db) locked = await record_failure(db, attempt, admission) raise locked or HTTPException(status_code=401, detail="Invalid credentials") + settings = request.app.state.users.settings + if not user.is_verified and settings.require_verification: + # Same rule as the session login: a self-registered account that never + # proved its address must not mint bearer tokens either (#547). + await rollback_session(db) + await release(db, attempt, admission) + raise HTTPException(status_code=400, detail="LOGIN_USER_NOT_VERIFIED") if updated_hash is not None: # A legacy-scheme hash (e.g. bcrypt) was accepted and re-hashed with the # current scheme; persist it, as ``UserManager.authenticate`` does for @@ -104,12 +114,6 @@ async def token_login( user.hashed_password = updated_hash await db.flush() - settings = request.app.state.users.settings - if not user.is_verified and settings.require_verification: - # Same rule as the session login: a self-registered account that never - # proved its address must not mint bearer tokens either (#547). - await release(db, attempt, admission) - raise HTTPException(status_code=400, detail="LOGIN_USER_NOT_VERIFIED") await record_success(db, attempt, admission) return await _create_token_pair(db, user.id, settings, user.session_version) diff --git a/modules/users/users/auth_local/token_preview.py b/modules/users/users/auth_local/token_preview.py index 18f6706fc..bf6941054 100644 --- a/modules/users/users/auth_local/token_preview.py +++ b/modules/users/users/auth_local/token_preview.py @@ -6,7 +6,7 @@ carry both answers, so the view decodes them read-only and hands the page a decided state instead of a form that is guaranteed to fail. -Sibling of :mod:`users.auth_local.invite_preview`, which does the same for the +Sibling of :mod:`users.invites.preview`, which does the same for the invite card; kept apart because that one also reads the account's roles. """ diff --git a/modules/users/users/auth_local/views.py b/modules/users/users/auth_local/views.py index 5ee11c1f3..56ba1fb2b 100644 --- a/modules/users/users/auth_local/views.py +++ b/modules/users/users/auth_local/views.py @@ -12,11 +12,11 @@ from sqlalchemy.ext.asyncio import AsyncSession from starlette.responses import RedirectResponse -from users.auth_local.invite_preview import preview_invite from users.auth_local.token_preview import decode_verify_token, preview_reset from users.bootstrap import resolve_bootstrap_credentials from users.contracts.schemas import UserRead from users.demo import resolve_demo_accounts +from users.invites.preview import preview_invite from users.mailer import mailer_delivers from users.manager import UserManager, get_user_manager from users.models import User diff --git a/modules/users/users/demo.py b/modules/users/users/demo/__init__.py similarity index 99% rename from modules/users/users/demo.py rename to modules/users/users/demo/__init__.py index b63d452be..3fc6e9e77 100644 --- a/modules/users/users/demo.py +++ b/modules/users/users/demo/__init__.py @@ -17,7 +17,7 @@ Deliberately *not* the dev quick-login buttons (``users.auth_local.views``): those are development-only and paste real credentials into the form, which is exactly what a published demo must not do. Here the passwords never leave the -server — :mod:`users.auth_local.demo_api` mints the session directly. +server — :mod:`users.demo.api` mints the session directly. """ from __future__ import annotations diff --git a/modules/users/users/auth_local/demo_api.py b/modules/users/users/demo/api.py similarity index 98% rename from modules/users/users/auth_local/demo_api.py rename to modules/users/users/demo/api.py index ca2b9bf97..3a0fd4a81 100644 --- a/modules/users/users/auth_local/demo_api.py +++ b/modules/users/users/demo/api.py @@ -10,7 +10,7 @@ between the two is a plain link's worth of work. No password reaches the browser either way: a visitor clicks a button, the server looks the account up itself, and the session it mints is stamped :data:`users.demo.SESSION_DEMO_KEY` -so :class:`users.demo_guard.DemoReadOnlyMiddleware` can recognise it. +so :class:`users.demo.guard.DemoReadOnlyMiddleware` can recognise it. """ from __future__ import annotations diff --git a/modules/users/users/demo_guard.py b/modules/users/users/demo/guard.py similarity index 100% rename from modules/users/users/demo_guard.py rename to modules/users/users/demo/guard.py diff --git a/modules/users/users/errors.py b/modules/users/users/errors.py new file mode 100644 index 000000000..8a00d595a --- /dev/null +++ b/modules/users/users/errors.py @@ -0,0 +1,19 @@ +"""HTTP mapping of users domain errors.""" + +from fastapi import FastAPI, Request, status +from fastapi.responses import JSONResponse + +from users.exceptions import LoginLockedError + + +async def _login_locked(request: Request, exc: Exception) -> JSONResponse: + assert isinstance(exc, LoginLockedError) + return JSONResponse( + {"detail": str(exc)}, + status_code=status.HTTP_429_TOO_MANY_REQUESTS, + headers={"Retry-After": str(exc.retry_after)}, + ) + + +def install_exception_handlers(app: FastAPI) -> None: + app.add_exception_handler(LoginLockedError, _login_locked) diff --git a/modules/users/users/exceptions.py b/modules/users/users/exceptions.py index 5a6ad6f59..ecdfa1d03 100644 --- a/modules/users/users/exceptions.py +++ b/modules/users/users/exceptions.py @@ -10,7 +10,11 @@ import uuid -class UserNotFoundError(Exception): +class UsersError(Exception): + """Base for users business failures, independent of HTTP.""" + + +class UserNotFoundError(UsersError): """Raised when a user lookup by id/email returns nothing.""" def __init__(self, user_id: uuid.UUID) -> None: @@ -18,7 +22,7 @@ def __init__(self, user_id: uuid.UUID) -> None: self.user_id = user_id -class EmailAlreadyExistsError(Exception): +class EmailAlreadyExistsError(UsersError): """Raised when updating a user to an email already owned by another user.""" def __init__(self, email: str) -> None: @@ -26,7 +30,7 @@ def __init__(self, email: str) -> None: self.email = email -class ExternalUserNoPasswordError(Exception): +class ExternalUserNoPasswordError(UsersError): """Raised when a password-credential action targets an external (SSO) user. External users have no local password, so password reset / set has no @@ -38,7 +42,7 @@ def __init__(self, user_id: uuid.UUID) -> None: self.user_id = user_id -class NotPendingInviteError(Exception): +class NotPendingInviteError(UsersError): """Raised when resending an invite to a row that is not awaiting one. Resend is only offered on rows in the ``invited`` state, so reaching this @@ -55,3 +59,14 @@ def __init__(self, user_id: uuid.UUID, reason: str) -> None: super().__init__(f"User {user_id} has no pending invite: {reason}") self.user_id = user_id self.reason = reason + + +LOGIN_LOCKED_DETAIL = "Too many attempts — try again later" + + +class LoginLockedError(UsersError): + """A login attempt is locked until the retry delay elapses.""" + + def __init__(self, retry_after: int) -> None: + super().__init__(LOGIN_LOCKED_DETAIL) + self.retry_after = max(1, retry_after) diff --git a/modules/users/users/invites/__init__.py b/modules/users/users/invites/__init__.py new file mode 100644 index 000000000..30f453c92 --- /dev/null +++ b/modules/users/users/invites/__init__.py @@ -0,0 +1 @@ +"""Invitation creation, delivery, resend and preview.""" diff --git a/modules/users/users/admin/bulk_invite.py b/modules/users/users/invites/bulk.py similarity index 74% rename from modules/users/users/admin/bulk_invite.py rename to modules/users/users/invites/bulk.py index ac8bc913c..b3be12443 100644 --- a/modules/users/users/admin/bulk_invite.py +++ b/modules/users/users/invites/bulk.py @@ -8,22 +8,23 @@ from __future__ import annotations -import contextlib import logging from fastapi import APIRouter, Depends, Request from fastapi_users.exceptions import UserAlreadyExists from pydantic import EmailStr, TypeAdapter, ValidationError from simple_module_core.events import EventBus -from simple_module_db.deps import get_db -from sqlalchemy.ext.asyncio import AsyncSession +from simple_module_db import independent_session from users.admin.links import admin_link_base from users.admin.service import UserService from users.contracts.events import UserInvited from users.contracts.schemas import BulkInviteResponse, BulkInviteResult, UserBulkInvite -from users.deps import get_event_bus, get_mailer, get_user_service +from users.db_adapter import UserDatabaseWithRoles +from users.deps import get_event_bus, get_mailer from users.mailer import mailer_delivers +from users.manager import UserManager +from users.models import OAuthAccount, User logger = logging.getLogger(__name__) @@ -82,10 +83,6 @@ async def admin_bulk_invite( data: UserBulkInvite, request: Request, bus: EventBus = Depends(get_event_bus), - service: UserService = Depends(get_user_service), - # The same request-scoped session the service was built from, taken through - # the dependency rather than off the service's private attribute. - db: AsyncSession = Depends(get_db), mailer=Depends(get_mailer), ) -> BulkInviteResponse: """Invite every address in *data*, all sharing the same roles.""" @@ -122,12 +119,17 @@ async def admin_bulk_invite( results: list[BulkInviteResult] = [] for email in accepted: try: - user, token = await service.invite(email, None, data.role_names, invited_by=invited_by) - # Make this invite durable before touching the session again. The - # failure path below rolls back, and the role rows the service - # flushed but did not commit would go with it — leaving the person - # invited with none of the roles the admin picked. - await db.commit() + async with independent_session(request.app) as checkpoint: + manager = UserManager( + UserDatabaseWithRoles(checkpoint, User, OAuthAccount), + mailer, + request.app.state.users.settings, + ) + service = UserService(checkpoint, manager, request.app) + user, token = await service.invite( + email, None, data.role_names, invited_by=invited_by + ) + # The invite and roles are durable before delivery or event handlers. except Exception as exc: # Already-registered is the common case and reads fine as-is; # anything else is logged so the admin's summary stays short. @@ -135,15 +137,6 @@ async def admin_bulk_invite( results.append( BulkInviteResult(email=email, status=STATUS_FAILED, detail=_failure_detail(exc)) ) - # Clear any failed transaction state before touching the session - # again. A real DB error (an IntegrityError from a concurrent - # signup, say) otherwise leaves every remaining address dying with - # PendingRollbackError — one bad row turning into total failure, - # the opposite of the partial success this endpoint promises. - # Safe to discard: every successful invite above is committed - # before the next one starts, so nothing durable is pending here. - with contextlib.suppress(Exception): - await db.rollback() continue if delivers: @@ -180,22 +173,9 @@ async def admin_bulk_invite( invited_by=(str(invited_by.id) if invited_by else None), ) ) - # Handlers are awaited inline and may write through this same - # request-scoped session. Commit again so a *later* address failing — - # and rolling back to clear the transaction — cannot take their work - # with it, which is what makes the "nothing durable is pending" - # assumption in the failure branch above actually hold. - # - # Guarded: the invite itself is already committed, so a handler writing - # something the DB rejects must not 500 the request and discard every - # per-address result collected so far — that is exactly the total - # failure this endpoint exists to avoid. - try: - await db.commit() - except Exception as exc: - logger.warning("post-invite handler commit failed for %s: %s", email, exc) - with contextlib.suppress(Exception): - await db.rollback() + # Handlers keep the request unit of work. Later invite failures roll + # back only their independent session, so handler writes remain pending + # for CommitBeforeResponseMiddleware and keep their on_commit ordering. results.extend( BulkInviteResult( diff --git a/modules/users/users/auth_local/invite_preview.py b/modules/users/users/invites/preview.py similarity index 100% rename from modules/users/users/auth_local/invite_preview.py rename to modules/users/users/invites/preview.py diff --git a/modules/users/users/admin/resend_invite.py b/modules/users/users/invites/resend.py similarity index 98% rename from modules/users/users/admin/resend_invite.py rename to modules/users/users/invites/resend.py index 4ec883ff8..834332aa9 100644 --- a/modules/users/users/admin/resend_invite.py +++ b/modules/users/users/invites/resend.py @@ -19,16 +19,16 @@ from fastapi import APIRouter, Depends, HTTPException, Request from fastapi import status as http_status -from users.admin.bulk_invite import ( +from users.admin.service import UserService +from users.contracts.schemas import BulkInviteResult +from users.deps import get_mailer, get_user_service +from users.exceptions import NotPendingInviteError, UserNotFoundError +from users.invites.bulk import ( MAILER_FAILURE_DETAIL, STATUS_LINK, STATUS_SENT, invite_link, ) -from users.admin.service import UserService -from users.contracts.schemas import BulkInviteResult -from users.deps import get_mailer, get_user_service -from users.exceptions import NotPendingInviteError, UserNotFoundError from users.mailer import mailer_delivers logger = logging.getLogger(__name__) diff --git a/modules/users/users/module.py b/modules/users/users/module.py index 7ee2fc51b..50aa7d55b 100644 --- a/modules/users/users/module.py +++ b/modules/users/users/module.py @@ -128,6 +128,11 @@ def register_invalidations(self, bus: InvalidationBus, app: FastAPI) -> None: subscribe(bus) + def register_exception_handlers(self, app: FastAPI) -> None: + from users.errors import install_exception_handlers + + install_exception_handlers(app) + def register_middleware(self, app: FastAPI) -> None: """Refuse writes from either shared demo session. @@ -135,7 +140,7 @@ def register_middleware(self, app: FastAPI) -> None: re-reads them per request — so an install that never hosts a demo pays one frozenset lookup on unsafe methods and nothing on reads. """ - from users.demo_guard import DemoReadOnlyMiddleware + from users.demo.guard import DemoReadOnlyMiddleware app.add_middleware(DemoReadOnlyMiddleware) @@ -210,10 +215,10 @@ def locale_dirs(self) -> dict[str, Path]: def register_routes(self, api_router: APIRouter, view_router: APIRouter) -> None: from users.admin.api import admin_router from users.auth_local import api as auth_local_api - from users.auth_local.demo_api import router as demo_router from users.auth_local.token_api import router as token_router from users.auth_local.views import router as auth_views from users.contracts.schemas import UserCreate, UserRead + from users.demo.api import router as demo_router from users.deps import fastapi_users from users.oauth.api import register_oauth_routes diff --git a/modules/users/users/registration.py b/modules/users/users/registration.py index 6f61d1e86..62299027f 100644 --- a/modules/users/users/registration.py +++ b/modules/users/users/registration.py @@ -4,6 +4,7 @@ import logging +from simple_module_db import independent_session from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession from sqlmodel import col @@ -28,15 +29,19 @@ async def attach_default_role(session: AsyncSession, user: User) -> None: public signup must not undo that. It also keeps concurrent signups from racing to insert it. """ - role = ( - await session.execute(select(Role).where(col(Role.name) == USER_ROLE_NAME)) - ).scalar_one_or_none() - if role is None: - logger.warning("No '%s' role exists; %s registered with no role", USER_ROLE_NAME, user.id) - return - session.add(UserRole(user_id=user.id, role_id=role.id)) - # Committed, not just flushed: fastapi-users already committed the - # user row in ``create()``, so a failure after this point (event - # publish, the verification email) would otherwise roll the role back - # and leave a committed account with no role that cannot re-register. + user_id = user.id + # End fastapi-users' post-commit refresh transaction before borrowing again. + # Framework sessions keep objects loaded on commit for the remaining hooks. await session.commit() + async with independent_session(session) as checkpoint: + role = ( + await checkpoint.execute(select(Role).where(col(Role.name) == USER_ROLE_NAME)) + ).scalar_one_or_none() + if role is None: + logger.warning( + "No '%s' role exists; %s registered with no role", USER_ROLE_NAME, user_id + ) + return + # fastapi-users already committed the account. Keep its default role + # even if a later registration hook or verification email fails. + checkpoint.add(UserRole(user_id=user_id, role_id=role.id)) diff --git a/modules/users/users/shared_props.py b/modules/users/users/shared_props.py index 4ffa0dfaa..d48660780 100644 --- a/modules/users/users/shared_props.py +++ b/modules/users/users/shared_props.py @@ -13,7 +13,7 @@ from typing import TYPE_CHECKING -from users.demo_guard import is_demo_session +from users.demo.guard import is_demo_session if TYPE_CHECKING: from starlette.requests import Request diff --git a/skills/simple-module-conventions/SKILL.md b/skills/simple-module-conventions/SKILL.md index 1c4323ac2..86970fe76 100644 --- a/skills/simple-module-conventions/SKILL.md +++ b/skills/simple-module-conventions/SKILL.md @@ -121,6 +121,17 @@ Build a `MailMessage(to, subject, text)` and send it with `current_mail_sender() Which module holds the accounts depends on the auth provider (`users` or `keycloak`). To name, look up, search or count users, use `get_user_directory(app)` from `simple_module_core.user_directory`: `get_many(db, ids, with_roles=...)`, `search(db, term, limit=...)`, `match_ids(term)` (a subquery) and `stats(db, active_window=...)`. Name a user with `DirectoryUser.label`. `roles is None` means the provider does not know them (Keycloak) — refuse anything that needs roles rather than treating the user as role-less. Importing `users.models` from another module silently queries an empty table under Keycloak. +### 13. Raise domain exceptions from services + +Define a per-module base error in `/exceptions.py` or `errors.py` and +subclass it for business failures. Services must not raise `HTTPException`: +CLI commands, tasks and durable handlers need transport-independent failures. +Follow `file_storage`: map errors in endpoints or install HTTP handlers through +`register_exception_handlers(app)`. Preserve status codes, exact JSON bodies, +headers and translation keys when migrating; translate at the request boundary. +Endpoint-level `HTTPException` for pure request validation is fine. Inertia +forms may catch domain errors to report field errors or redirect. + ## Related skills - **simple-module-database** — per-module Base, mixins, session lifecycle diff --git a/skills/simple-module-creating/SKILL.md b/skills/simple-module-creating/SKILL.md index 44899b38e..e664b1cce 100644 --- a/skills/simple-module-creating/SKILL.md +++ b/skills/simple-module-creating/SKILL.md @@ -129,4 +129,10 @@ The new module should appear in the registered-modules log line. If it has a vie - **`name=` collides.** Two modules with the same `ModuleMeta.name` raise `SM008` at dev boot / `make doctor` (their lowercased names would also collide as table prefixes in the shared schema). - **Registered the module by hand in host code.** Don't — discovery is entry-point-only; host code never imports module code. +Services raise domain exceptions derived from a per-module base in +`exceptions.py` or `errors.py`, never `HTTPException`. Map them in endpoints or +`register_exception_handlers(app)` (as in `file_storage`); translate at the +request boundary. Preserve existing status codes, JSON bodies and headers. +Endpoint-level `HTTPException` for pure request validation remains fine. + For framework-wide rules that apply once the module exists (SQLModel-everywhere, file-size cap, settings layout, no `session.commit()` in services), see the **simple-module-conventions** skill. For migration mechanics see **simple-module-migrations**.