feat(users): host-overridable auth email templates with an HTML part (#712)

CHANGELOG.md

@@ -15,6 +15,9 @@ All notable changes to this project are documented in this file. The format is b
 ## [Unreleased]
 
 ### Added
+- Users auth emails support ordered host template directories through the
+  restart-required DB setting `mail_template_dirs`, with per-file bundled
+  fallback and optional autoescaped HTML alternatives (#712).
 - Public durable delivery context (`in_delivery()` and `IN_DELIVERY`), inherited
   by handler child tasks and reset after delivery (#682).
 - Transactional, tenant-scoped `sharing.contracts.forget_resource` for resource

docs/modules/users.md

@@ -202,6 +202,7 @@ Everything else is DB-backed (initial values are pydantic defaults; edit under U
 | `cookie_secure` | `True` (flipped to `False` in dev at startup) |
 | `cookie_samesite` | `"lax"` |
 | `base_url` | `"http://localhost:8000"` |
+| `mail_template_dirs` | `[]` — ordered absolute template directories, restart required; see [Overriding auth email templates](#overriding-auth-email-templates) |
 | `login_rate_limit_failures` | `5` — failures from one IP for one username before that source is throttled for the window |
 | `login_rate_limit_window_seconds` | `300` — the window for both lockout tiers |
 | `login_rate_limit_cooldown_seconds` | `900` — how long an account lock lasts |
@@ -418,14 +419,49 @@ Trade-offs to know about:
 
 ## Mailer
 
-`mailer/` ships one implementation of the `Mailer` protocol, `TemplatedMailer`. It renders the bundled verify / reset / invite templates and sends each as a `MailMessage` through the framework mail sender (`simple_module_core.mail.current_mail_sender()`), looked up at send time. The transport (SMTP or console), its credentials and the `mail.mailer` health check belong to the [`mail` module](./mail.md); the former `mailer` and `smtp_*` users settings no longer exist.
+`mailer/` ships one implementation of the `Mailer` protocol, `TemplatedMailer`. It renders the verify / reset / invite templates (bundled, or [overridden by the host](#overriding-auth-email-templates)) and sends each as a `MailMessage` through the framework mail sender (`simple_module_core.mail.current_mail_sender()`), looked up at send time. The transport (SMTP or console), its credentials and the `mail.mailer` health check belong to the [`mail` module](./mail.md); the former `mailer` and `smtp_*` users settings no longer exist.
 
 Bulk invite failures return a known, safe reason for an existing account and a
 generic retry message for unexpected errors (#698). SQL statements, parameters
 and backend exception messages stay in the application log.
 
 `delivers_email` reflects the installed sender, so screens that need to hand the one-time link back (reset-sent card, add people, resend and bulk invite) do so whenever the sender only logs. `build_mailer(settings)` returns the `TemplatedMailer`; a third-party `Mailer` can still be assigned to `app.state.users.mailer`.
 
+### Overriding auth email templates
+
+Set the DB-backed Users setting `mail_template_dirs` to an ordered list of
+absolute directory paths, for example `["/app/email-templates/build"]` (#712).
+It defaults to an empty list and has no environment variable. Restart every
+application worker after changing it. Startup skips, with a warning, relative paths,
+missing directories and paths that refer to files, so a typo cannot stop the app
+from booting. These directories contain trusted
+deployment code and must never come from request input. Templates render in
+Jinja's sandbox, so a file dropped into a listed directory cannot run Python.
+
+The mailer searches the configured directories in order, then its bundled
+templates, independently for each file:
+
+| Email | Required text template | Optional HTML template | Context |
+|---|---|---|---|
+| Verification | `verify_email.txt` | `verify_email.html` | `app_name`, `link` |
+| Password reset | `reset_password.txt` | `reset_password.html` | `app_name`, `link` |
+| Invitation | `invite.txt` | `invite.html` | `app_name`, `link`, `invited_by_name`, `message` |
+
+An HTML-only override keeps the bundled text alternative. Both parts receive
+the same context; branding is read live, and the invitation message is trimmed
+(`None` when blank). HTML escapes interpolated values automatically; text
+remains unescaped. A missing HTML template sends text only. Invalid templates,
+missing includes and other rendering errors propagate and prevent delivery.
+Subjects, links, transport selection and email localization are unchanged.
+
+Hosts can supply compiled HTML from tools such as MJML; the framework does not
+compile it. Code constructing a mailer directly can pass the keyword-only
+`template_dirs` argument to `TemplatedMailer(base_url, ..., template_dirs=...)`
+or `build_mailer(settings, ..., template_dirs=...)`, using strings or `Path`
+objects. Startup passes `settings.mail_template_dirs` explicitly. Each mailer
+owns its template environment, so one application's overrides cannot affect
+another's.
+
 ## OAuth providers
 
 OAuth/OIDC sign-in is **DB-settings-driven**: a provider is enabled simply by setting its `client_id` + `client_secret` (and, for generic OIDC, a discovery URL) in the settings UI under Users at `/admin/settings/`. No code change or restart — `register_event_handlers` rebuilds the client cache (`app.state.users.oauth_clients`) on the `SettingsReloaded` event, so changes take effect live. Providers with no credentials are silently skipped.

modules/users/tests/test_users_mailer_templates.py

@@ -0,0 +1,148 @@
+"""Host template precedence, optional HTML, and startup configuration (#712)."""
+
+import json
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+from fastapi import FastAPI
+from jinja2 import TemplateNotFound, TemplateSyntaxError
+from jinja2.exceptions import SecurityError
+from simple_module_core.settings_hydrate import build_settings
+from users.mailer import build_mailer
+from users.mailer.templated import TemplatedMailer
+from users.settings import UsersSettings
+from users.startup import _build_mailer
+
+
+@pytest.mark.parametrize(
+    ("stem", "method", "args", "route"),
+    [
+        ("verify_email", "send_verification", (), "verify"),
+        ("reset_password", "send_password_reset", (), "reset-password"),
+        ("invite", "send_invite", ("<Alice>", " <note> "), "invite/accept"),
+    ],
+)
+async def test_html_only_override_keeps_bundled_text(
+    tmp_path, captured_mail, stem, method, args, route
+):
+    (tmp_path / f"{stem}.html").write_text(
+        "<p>{{ app_name }}|{{ link }}|{{ invited_by_name }}|{{ message }}</p>"
+    )
+    mailer = TemplatedMailer("https://example.com/", lambda: "<Acme>", template_dirs=[tmp_path])
+    await getattr(mailer, method)("a@example.com", "tok&x", *args)
+
+    (msg,) = captured_mail.messages
+    assert f"https://example.com/users/{route}?token=tok&x" in msg.text
+    assert "<Acme>" in msg.text
+    assert f"https://example.com/users/{route}?token=tok&amp;x" in msg.html
+    assert "&lt;Acme&gt;" in msg.html
+    if stem == "invite":
+        assert "&lt;Alice&gt;|&lt;note&gt;" in msg.html
+        assert "<Alice>" in msg.text and "<note>" in msg.text
+
+
+async def test_directory_order_and_independent_file_fallback(tmp_path, captured_mail):
+    first, second = tmp_path / "first", tmp_path / "second"
+    first.mkdir()
+    second.mkdir()
+    (first / "verify_email.txt").write_text("first {{ app_name }}")
+    (second / "verify_email.txt").write_text("second")
+    (second / "verify_email.html").write_text("<b>second {{ app_name }}</b>")
+    mailer = build_mailer(UsersSettings(), lambda: "<Acme>", template_dirs=[str(first), second])
+    await mailer.send_verification("a@example.com", "tok")
+    await mailer.send_password_reset("a@example.com", "reset")
+    verify, reset = captured_mail.messages
+    assert verify.text == "first <Acme>"
+    assert verify.html == "<b>second &lt;Acme&gt;</b>"
+    assert "/users/reset-password?token=reset" in reset.text
+    assert reset.html is None
+
+
+async def test_mailer_environments_are_isolated(tmp_path, captured_mail):
+    (tmp_path / "verify_email.txt").write_text("custom")
+    custom = TemplatedMailer("https://example.com", template_dirs=[tmp_path])
+    default = TemplatedMailer("https://example.com")
+    await custom.send_verification("a@example.com", "tok")
+    await default.send_verification("a@example.com", "tok")
+    await custom.send_verification("a@example.com", "tok")
+    assert [m.text for m in captured_mail.messages][::2] == ["custom", "custom"]
+    assert "Welcome to SimpleModule!" in captured_mail.messages[1].text
+    assert all(m.html is None for m in captured_mail.messages)
+
+
+@pytest.mark.parametrize(
+    ("source", "error"),
+    [
+        ("{% if %}", TemplateSyntaxError),
+        ('{% include "missing.html" %}', TemplateNotFound),
+        ('{% extends "missing.html" %}', TemplateNotFound),
+        ("{{ 1 / 0 }}", ZeroDivisionError),
+    ],
+)
+async def test_broken_html_prevents_delivery(tmp_path, captured_mail, source, error):
+    (tmp_path / "verify_email.html").write_text(source)
+    mailer = TemplatedMailer("https://example.com", template_dirs=[tmp_path])
+    with pytest.raises(error):
+        await mailer.send_verification("a@example.com", "tok")
+    assert not captured_mail.messages
+
+
+async def test_broken_text_is_not_hidden_by_bundled_fallback(tmp_path, captured_mail):
+    (tmp_path / "verify_email.txt").write_text("{% if %}")
+    mailer = TemplatedMailer("https://example.com", template_dirs=[tmp_path])
+    with pytest.raises(TemplateSyntaxError):
+        await mailer.send_verification("a@example.com", "tok")
+    assert not captured_mail.messages
+
+
+@pytest.mark.parametrize("kind", ["relative", "missing", "file"])
+def test_invalid_template_directory_is_skipped_with_warning(tmp_path, kind, caplog):
+    path = tmp_path / kind
+    if kind == "file":
+        path.write_text("not a directory")
+    if kind == "relative":
+        path = Path("relative")
+    settings = UsersSettings(mail_template_dirs=[str(path)])
+    state = SimpleNamespace()
+    _build_mailer(FastAPI(), state, settings)
+    assert state.mailer is not None
+    assert "Ignoring mail template path" in caplog.text
+
+
+async def test_startup_forwards_db_setting_and_keeps_branding_live(tmp_path, captured_mail):
+    (tmp_path / "verify_email.html").write_text("<p>{{ app_name }}</p>")
+    settings = build_settings(
+        UsersSettings, {"mail_template_dirs": (json.dumps([str(tmp_path)]), "json")}
+    )
+    app = FastAPI()
+    app.state.branding = SimpleNamespace(settings=SimpleNamespace(app_name="First"))
+    state = SimpleNamespace()
+    _build_mailer(app, state, settings)
+    app.state.branding.settings.app_name = "<Second>"
+    await state.mailer.send_verification("a@example.com", "tok")
+    (msg,) = captured_mail.messages
+    assert msg.html == "<p>&lt;Second&gt;</p>"
+    assert msg.subject == "Verify your email for <Second>"
+    assert "<Second>" in msg.text
+
+
+def test_template_setting_is_restart_required_and_defaults_are_independent(monkeypatch):
+    monkeypatch.setenv("SM_USERS_MAIL_TEMPLATE_DIRS", '["/ignored"]')
+    first, second = UsersSettings(), UsersSettings()
+    assert first.mail_template_dirs == []
+    first.mail_template_dirs.append("/trusted/templates")
+    assert second.mail_template_dirs == []
+    assert UsersSettings.model_fields["mail_template_dirs"].json_schema_extra == {
+        "requires_restart": True
+    }
+
+
+async def test_host_templates_render_sandboxed(tmp_path, captured_mail):
+    # The directory list is admin-editable, so a template must not reach Python
+    # internals (the classic ``__class__.__mro__`` escape).
+    (tmp_path / "verify_email.txt").write_text("{{ ''.__class__.__mro__[1].__subclasses__() }}")
+    mailer = TemplatedMailer("https://example.com", template_dirs=[tmp_path])
+    with pytest.raises(SecurityError):
+        await mailer.send_verification("a@example.com", "tok")
+    assert captured_mail.messages == []

modules/users/users/constants.py

@@ -15,6 +15,11 @@
 ADMIN_ROLE_DESCRIPTION = "Administrator"
 USER_ROLE_DESCRIPTION = "Standard user"
 
+# Auth email template filenames (HTML siblings are optional).
+MAIL_TEMPLATE_VERIFY = "verify_email.txt"
+MAIL_TEMPLATE_RESET = "reset_password.txt"
+MAIL_TEMPLATE_INVITE = "invite.txt"
+
 # Permission identifiers
 PERM_USERS_MANAGE = "users.manage"
 PERM_USERS_SELF_PROFILE = "users.self.profile"

modules/users/users/mailer/__init__.py

@@ -2,7 +2,8 @@
 
 from __future__ import annotations
 
-from collections.abc import Callable
+from collections.abc import Callable, Sequence
+from pathlib import Path
 from typing import Protocol, runtime_checkable
 
 from users.settings import UsersSettings
@@ -52,7 +53,13 @@ def mailer_delivers(mailer: object | None) -> bool:
 def build_mailer(
     settings: UsersSettings,
     app_name_provider: AppNameProvider | None = None,
+    *,
+    template_dirs: Sequence[str | Path] = (),
 ) -> Mailer:
     from users.mailer.templated import TemplatedMailer
 
-    return TemplatedMailer(base_url=settings.base_url, app_name_provider=app_name_provider)
+    return TemplatedMailer(
+        base_url=settings.base_url,
+        app_name_provider=app_name_provider,
+        template_dirs=template_dirs,
+    )

modules/users/users/mailer/templated.py

@@ -1,4 +1,4 @@
-"""Built-in mailer: renders the bundled templates and hands them to the mail sender.
+"""Built-in mailer: renders host or bundled templates and hands them to the mail sender.
 
 Transport (SMTP, console, ...) belongs to the ``mail`` module. The sender is
 looked up at send time so a settings change there takes effect without a
@@ -8,32 +8,55 @@
 from __future__ import annotations
 
 import importlib.resources
+import logging
+from collections.abc import Sequence
+from pathlib import Path
 from typing import TYPE_CHECKING, Any
 
 import jinja2
+import jinja2.sandbox
 from simple_module_core.mail import MailMessage, current_mail_sender
 
+from users.constants import MAIL_TEMPLATE_INVITE, MAIL_TEMPLATE_RESET, MAIL_TEMPLATE_VERIFY
+
 if TYPE_CHECKING:
     from users.mailer import AppNameProvider
 
-_TPL_VERIFY = "verify_email.txt"
-_TPL_RESET = "reset_password.txt"
-_TPL_INVITE = "invite.txt"
-
-
-# The bundled templates directory; plain text, so no autoescape.
-_template_env = jinja2.Environment(
-    loader=jinja2.FileSystemLoader(str(importlib.resources.files(__package__) / "templates")),
-    autoescape=False,
-)
+logger = logging.getLogger(__name__)
 
 
 class TemplatedMailer:
-    def __init__(self, base_url: str, app_name_provider: AppNameProvider | None = None) -> None:
+    def __init__(
+        self,
+        base_url: str,
+        app_name_provider: AppNameProvider | None = None,
+        *,
+        template_dirs: Sequence[str | Path] = (),
+    ) -> None:
         self._base = base_url.rstrip("/")
         from users.mailer import default_app_name
 
         self._app_name = app_name_provider or default_app_name
+        # An admin edits this setting in the UI; a bad entry must not stop the
+        # app from booting, so it is skipped with a warning instead.
+        directories = []
+        for directory in map(Path, template_dirs):
+            if directory.is_absolute() and directory.is_dir():
+                directories.append(directory)
+            else:
+                logger.warning(
+                    "Ignoring mail template path %s: not an existing absolute directory",
+                    directory,
+                )
+        # Sandboxed: the setting is editable from the admin UI, so a template
+        # dropped into a writable directory (an upload folder) must not be able
+        # to run arbitrary Python.
+        self._template_env = jinja2.sandbox.SandboxedEnvironment(
+            loader=jinja2.FileSystemLoader(
+                [*directories, str(importlib.resources.files(__package__) / "templates")]
+            ),
+            autoescape=jinja2.select_autoescape(enabled_extensions=("html",), default=False),
+        )
 
     @property
     def delivers_email(self) -> bool:
@@ -43,12 +66,16 @@ def delivers_email(self) -> bool:
     async def send_verification(self, email: str, token: str) -> None:
         app = self._app_name()
         link = f"{self._base}/users/verify?token={token}"
-        await self._send(email, f"Verify your email for {app}", _TPL_VERIFY, link=link)
+        await self._send(
+            email, f"Verify your email for {app}", MAIL_TEMPLATE_VERIFY, app_name=app, link=link
+        )
 
     async def send_password_reset(self, email: str, token: str) -> None:
         app = self._app_name()
         link = f"{self._base}/users/reset-password?token={token}"
-        await self._send(email, f"Reset your {app} password", _TPL_RESET, link=link)
+        await self._send(
+            email, f"Reset your {app} password", MAIL_TEMPLATE_RESET, app_name=app, link=link
+        )
 
     async def send_invite(
         self,
@@ -62,7 +89,8 @@ async def send_invite(
         await self._send(
             email,
             f"{invited_by_name} invited you to {app}",
-            _TPL_INVITE,
+            MAIL_TEMPLATE_INVITE,
+            app_name=app,
             link=link,
             invited_by_name=invited_by_name,
             # Stripped here rather than in the template: a message of nothing
@@ -71,7 +99,13 @@ async def send_invite(
         )
 
     async def _send(self, to: str, subject: str, template_name: str, **context: Any) -> None:
-        body = _template_env.get_template(template_name).render(
-            app_name=self._app_name(), **context
-        )
-        await current_mail_sender().send(MailMessage(to=to, subject=subject, text=body))
+        body = self._template_env.get_template(template_name).render(**context)
+        html_name = str(Path(template_name).with_suffix(".html"))
+        try:
+            html_template = self._template_env.get_template(html_name)
+        except jinja2.TemplateNotFound as exc:
+            if exc.name != html_name:
+                raise
+            html_template = None
+        html = html_template.render(**context) if html_template is not None else None
+        await current_mail_sender().send(MailMessage(to=to, subject=subject, text=body, html=html))

modules/users/users/settings.py

@@ -112,6 +112,15 @@ def _lowercase_samesite(cls, value: object) -> object:
 
     # Mail transport (SMTP, console) lives in the ``mail`` module.
     base_url: str = "http://localhost:8000"
+    mail_template_dirs: list[str] = Field(
+        default_factory=list,
+        description=(
+            "Absolute directories searched, in order, for verify_email / reset_password / "
+            "invite templates (.txt, optional .html) before the bundled ones. "
+            "Relative or missing paths are ignored."
+        ),
+        json_schema_extra={"requires_restart": True},
+    )
 
     # Login lockout (GH #530, ``users.auth_local.lockout``). Failures within the
     # window: ``login_rate_limit_failures`` from one IP for one username

modules/users/users/startup.py

@@ -96,7 +96,7 @@ def _app_name() -> str:
         name = getattr(getattr(branding, "settings", None), "app_name", None)
         return name or default_app_name()
 
-    state.mailer = build_mailer(s, _app_name)
+    state.mailer = build_mailer(s, _app_name, template_dirs=s.mail_template_dirs)
 
 
 def _build_limiters(state, s) -> None: