Typed client for the remaining modules + raw-fetch gate, frontend splits, UX polish (#504 #522 #575)

.github/workflows/pr.yml

@@ -57,6 +57,8 @@ jobs:
       - run: make ci-check-test-sleeps
       # `as unknown as` switches the type checker off; use typed page args / usePageProps.
       - run: make ci-check-no-double-cast
+      # Requests go through the typed client (packages/ui/src/lib/api.ts), not raw fetch().
+      - run: make ci-check-no-raw-fetch
       # The checked-in typed-key files must match what the installed modules'
       # catalogs emit. Catches a catalog edit committed without regenerating, and
       # (with the namespace guard in gen_i18n.py) a regeneration that dropped keys.

CHANGELOG.md

@@ -11,6 +11,24 @@ All notable changes to this project are documented in this file. The format is b
 
 ## [Unreleased]
 
+### Fixed
+- Retained logs: a NUL character no longer drops the whole log batch on Postgres.
+- Users: after a self-service password change the same browser kept its page session but
+  every `/api/users/me*` call (including "Sign out everywhere") answered 401, because the
+  change deleted the `sm_auth` access-token row that browser's cookie pointed at.
+  `POST /api/users/me/password` now re-issues that cookie under the new `session_version`
+  (keeping a remembered sign-in's window); other sessions and bearer tokens are still revoked.
+  The profile's password and sessions cards show the translated failure inline.
+- Users: the admin "Copy reset link" (and the bulk-invite fallback link) no longer points at
+  `http://localhost:8000` on installs that never set `users.base_url`: an explicitly set
+  `users.base_url` wins, then `HostSettings.public_base_url`, then the origin the admin is
+  browsing. Emailed links are unchanged.
+- UX polish (#575): `/tenants/settings` shows translated labels instead of raw keys
+  (`SettingDefinition.label_key` / `TenantSettingView.label_key`, set by `branding`; en + de);
+  signed-in error pages outside `/admin` (e.g. a 404 for an unknown URL) keep the app layout (sidebar, user menu); the tenant
+  invite form shows a translated inline error for an invalid email instead of the browser's
+  silent native validation.
+
 ### Removed
 - Committed QA scratch directories `qa-shots/`, `.qa/` and `.verify/` (159 files); they are now gitignored (`dataset/` stays) (#511).
 - Unused `Combobox` shadcn component and the `@base-ui/react` dependency (nothing imported it; Radix has no combobox, use `cmdk` `Command` + `Popover`). **Behaviour change** for anyone importing `@simple-module-py/ui/components/ui/combobox` (#521).
@@ -159,6 +177,17 @@ All notable changes to this project are documented in this file. The format is b
 - tenants and api_tokens pages call their APIs through the typed client and take
   typed component props (#504). **Behaviour change:** the api_tokens pages no longer
   receive the unused `api_url` prop.
+- users, background_tasks, branding, settings, file_storage and the first-run setup
+  wizard call their APIs through the typed client (#504), completing the migration.
+  **Behaviour change:** failed requests from these pages now show translated error
+  toasts (the server's `detail` or a localized fallback) instead of raw or generic
+  English text. `make lint` / CI reject raw `fetch(` in shipped `.ts`/`.tsx`
+  (`ci-check-no-raw-fetch`, `scripts/check_no_raw_fetch.py`) unless marked
+  `// raw-fetch-ok: <reason>`.
+- file_storage's public file routes (`/api/file-storage/public/{file_id}` and
+  `.../{filename}`) now have distinct OpenAPI operation ids (`file_storage_public_file`,
+  `file_storage_public_file_named`); the cosmetic `filename` is a path segment on the
+  named route only. Paths and responses are unchanged.
 - Typed page props (#505): `@simple-module-py/ui` registers `SharedProps` (now including `i18n`) with
   Inertia, so `usePage().props.auth` is typed, and exports `usePageProps<T>()` for components and
   layouts. Pages take typed component arguments. The `as unknown as` casts in module pages, `packages/ui`

Makefile

@@ -1,4 +1,4 @@
-.PHONY: install install-py install-js dev dev-api dev-ui build test test-py test-py-pg test-js test-e2e bench memray-run memray-flamegraph loadtest loadtest-seed loadtest-memray bench-nav lint doctor migrate migration downgrade migration-history migrations-roundtrip-pg docker-up docker-down kill new-module gen-pages openapi gen-i18n docker-build docker-app docker-compose-app sync-module-deps ci-python-lint ci-python-typecheck ci-js-lint ci-js-typecheck ci-check-file-size ci-check-test-sleeps ci-check-no-double-cast ci-check-hardcoded-strings ci-check-untranslated ci-check-metadata ci-check-locales ci-doc-paths ci-docs-build ci-i18n-fresh ci-api-types-fresh gen-api-types ci-css-build ci-doctor ci-build-packages worker beat worker-docker
+.PHONY: install install-py install-js dev dev-api dev-ui build test test-py test-py-pg test-js test-e2e bench memray-run memray-flamegraph loadtest loadtest-seed loadtest-memray bench-nav lint doctor migrate migration downgrade migration-history migrations-roundtrip-pg docker-up docker-down kill new-module gen-pages openapi gen-i18n docker-build docker-app docker-compose-app sync-module-deps ci-python-lint ci-python-typecheck ci-js-lint ci-js-typecheck ci-check-file-size ci-check-test-sleeps ci-check-no-double-cast ci-check-no-raw-fetch ci-check-hardcoded-strings ci-check-untranslated ci-check-metadata ci-check-locales ci-doc-paths ci-docs-build ci-i18n-fresh ci-api-types-fresh gen-api-types ci-css-build ci-doctor ci-build-packages worker beat worker-docker
 
 # UV_SYNC_FLAGS is empty locally; CI passes --locked so a stale uv.lock fails the build.
 UV_SYNC_FLAGS ?=
@@ -129,7 +129,7 @@ loadtest:                   ## Run locust against a server already on $(LOCUST_H
 loadtest-memray:            ## Start uvicorn under memray, load-test, emit flamegraph
 	scripts/loadtest_memray.sh $(LOCUST_ARGS)
 
-lint: ci-python-lint ci-python-typecheck ci-js-lint ci-js-typecheck ci-check-metadata ci-check-locales ci-doc-paths ci-i18n-fresh ci-api-types-fresh ci-check-file-size ci-check-test-sleeps ci-check-no-double-cast ci-check-hardcoded-strings ci-check-untranslated
+lint: ci-python-lint ci-python-typecheck ci-js-lint ci-js-typecheck ci-check-metadata ci-check-locales ci-doc-paths ci-i18n-fresh ci-api-types-fresh ci-check-file-size ci-check-test-sleeps ci-check-no-double-cast ci-check-no-raw-fetch ci-check-hardcoded-strings ci-check-untranslated
 
 # Kept granular so pr.yml can run them in parallel.
 ci-python-lint:
@@ -221,6 +221,9 @@ ci-check-test-sleeps:
 ci-check-no-double-cast:
 	uv run python scripts/check_no_double_cast.py
 
+ci-check-no-raw-fetch:
+	uv run python scripts/check_no_raw_fetch.py
+
 # Enforce that permissions, role names, Inertia page ids, and module dependency
 # names are declared as named constants rather than hardcoded string literals.
 ci-check-hardcoded-strings:

framework/hosting/simple_module_hosting/locales/de.json

@@ -18,6 +18,7 @@
         "description": "Datenbankschema auf die von diesem Code erwartete Version aktualisieren."
       }
     },
+    "request_failed": "Die Anfrage ist fehlgeschlagen. Bitte erneut versuchen.",
     "working": "In Bearbeitung…",
     "done": "Fertig. Wird neu geladen…"
   },

framework/hosting/simple_module_hosting/locales/en.json

@@ -18,6 +18,7 @@
         "description": "Bring the database schema up to the version this code expects."
       }
     },
+    "request_failed": "The request failed. Try again.",
     "working": "Working…",
     "done": "Done. Reloading…"
   },

framework/hosting/simple_module_hosting/locales/es.json

@@ -19,7 +19,8 @@
       }
     },
     "working": "Procesando…",
-    "done": "Hecho. Recargando…"
+    "done": "Hecho. Recargando…",
+    "request_failed": "La solicitud falló. Inténtalo de nuevo."
   },
   "errors": {
     "token_not_allowed": "Los tokens de API no se pueden usar en esta ruta. Usa una sesión iniciada o una ruta protegida por un permiso incluido en el token."

framework/hosting/simple_module_hosting/setup_wizard/components/ConnectionList.tsx

@@ -1,5 +1,6 @@
 import { keys, useT } from '@simple-module-py/i18n';
 import { Button } from '@simple-module-py/ui/components/ui/button';
+import { apiFetch } from '@simple-module-py/ui/lib/api';
 import { CheckCircle2, PlugZap, XCircle } from 'lucide-react';
 import { useState } from 'react';
 
@@ -32,22 +33,15 @@ export function ConnectionList({
   async function retest() {
     setBusy(true);
     setError(null);
-    try {
-      const resp = await fetch('/setup/test-connections', {
-        method: 'POST',
-        headers: { Accept: 'application/json', 'X-CSRF-Token': csrfToken },
-      });
-      // A 404 here means setup completed in another tab, and the body is not
-      // the JSON this expects. Without the check, `resp.json()` throws into an
-      // unhandled rejection and the button just stops responding.
-      if (!resp.ok) throw new Error(resp.statusText || String(resp.status));
-      const body = await resp.json();
-      setChecks(body.checks ?? []);
-    } catch (err) {
-      setError((err as Error).message);
-    } finally {
-      setBusy(false);
-    }
+    // A 404 here means setup completed in another tab; it arrives as a failed
+    // result instead of a JSON parse error.
+    const result = await apiFetch<{ checks?: CheckResult[] }>('/setup/test-connections', {
+      method: 'POST',
+      headers: { 'X-CSRF-Token': csrfToken },
+    });
+    if (result.ok) setChecks(result.data?.checks ?? []);
+    else setError(result.detail ?? t(keys.hosting.setup.request_failed));
+    setBusy(false);
   }
 
   return (

framework/hosting/simple_module_hosting/setup_wizard/components/StepForm.tsx

@@ -2,6 +2,7 @@ import { keys, useT } from '@simple-module-py/i18n';
 import { Button } from '@simple-module-py/ui/components/ui/button';
 import { Input } from '@simple-module-py/ui/components/ui/input';
 import { Label } from '@simple-module-py/ui/components/ui/label';
+import { apiFetch } from '@simple-module-py/ui/lib/api';
 import { useState } from 'react';
 
 /** One input of a step's form — `SetupField` on the server. */
@@ -75,31 +76,23 @@ export function StepForm({
       data[field.name] = typeof value === 'string' && value !== '' ? value : null;
     }
 
-    try {
-      const resp = await fetch(`/setup/steps/${encodeURIComponent(stepId)}`, {
-        method: 'POST',
-        // Accept: application/json matters. The host renders an Inertia error
-        // *page* for a 4xx unless the caller prefers JSON, and a bare fetch()
-        // sends Accept: */* — so without it the body is HTML, json() throws,
-        // and the operator never sees why the request was refused.
-        headers: {
-          'Content-Type': 'application/json',
-          Accept: 'application/json',
-          'X-CSRF-Token': csrfToken,
-        },
-        body: JSON.stringify(data),
-      });
-      if (!resp.ok) {
-        const body = await resp.json().catch(() => ({}));
-        throw new Error(errorMessage(body, resp.statusText || String(resp.status)));
-      }
+    // `/setup/*` is outside the OpenAPI document and its token is the page
+    // prop, not the session one, so it is passed explicitly (`apiFetch` leaves
+    // a header the caller set alone). `apiFetch` sends Accept: application/json,
+    // which matters: the host renders an Inertia error *page* for a 4xx unless
+    // the caller prefers JSON, and the operator would never see why.
+    const result = await apiFetch(`/setup/steps/${encodeURIComponent(stepId)}`, {
+      method: 'POST',
+      headers: { 'X-CSRF-Token': csrfToken },
+      json: data,
+    });
+    if (!result.ok) {
+      setError(errorMessage(result.error, t(keys.hosting.setup.request_failed)));
+    } else {
       setDone(true);
       window.location.href = '/';
-    } catch (err) {
-      setError((err as Error).message);
-    } finally {
-      setBusy(false);
     }
+    setBusy(false);
   }
 
   return (

framework/hosting/tests-js/hosting_SetupWizard.test.tsx

@@ -0,0 +1,93 @@
+import '@testing-library/jest-dom/vitest';
+import { configureI18n, keys } from '@simple-module-py/i18n';
+import { fireEvent, render, screen, waitFor } from '@testing-library/react';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+
+configureI18n({ locale: 'en', messages: {} });
+
+import { ConnectionList } from '../simple_module_hosting/setup_wizard/components/ConnectionList';
+import { StepForm } from '../simple_module_hosting/setup_wizard/components/StepForm';
+
+function stub(status: number, body: unknown) {
+  const fetchMock = vi.fn(async () => new Response(JSON.stringify(body), { status }));
+  vi.stubGlobal('fetch', fetchMock);
+  return fetchMock;
+}
+
+const action = {
+  submitLabel: 'Create admin',
+  fields: [
+    {
+      name: 'email',
+      label: 'Email',
+      type: 'text',
+      required: false,
+      autocomplete: '',
+      minLength: null,
+    },
+    {
+      name: 'note',
+      label: 'Note',
+      type: 'text',
+      required: false,
+      autocomplete: '',
+      minLength: null,
+    },
+  ],
+};
+
+describe('StepForm', () => {
+  afterEach(() => vi.unstubAllGlobals());
+
+  it('POSTs the fields as JSON with the page-prop CSRF token', async () => {
+    const fetchMock = stub(200, {});
+    render(<StepForm stepId="admin user" action={action} csrfToken="page-token" />);
+    fireEvent.change(screen.getByLabelText('Email'), { target: { value: 'a@b.co' } });
+    fireEvent.submit(screen.getByRole('button', { name: 'Create admin' }).closest('form')!);
+
+    await waitFor(() => expect(fetchMock).toHaveBeenCalled());
+    const [req] = fetchMock.mock.calls[0] as unknown as [Request];
+    expect(req.method).toBe('POST');
+    expect(new URL(req.url).pathname).toBe('/setup/steps/admin%20user');
+    expect(req.headers.get('X-CSRF-Token')).toBe('page-token');
+    expect(req.headers.get('Accept')).toBe('application/json');
+    expect(await req.json()).toEqual({ email: 'a@b.co', note: null });
+  });
+
+  it('joins the messages of a 422 detail list', async () => {
+    stub(422, { detail: [{ msg: 'too short' }, { msg: 'bad address' }] });
+    render(<StepForm stepId="s" action={action} csrfToken="t" />);
+    fireEvent.submit(screen.getByRole('button', { name: 'Create admin' }).closest('form')!);
+    expect(await screen.findByText('too short; bad address')).toBeInTheDocument();
+  });
+
+  it('falls back to a translated line when the body has no detail', async () => {
+    stub(500, null);
+    render(<StepForm stepId="s" action={action} csrfToken="t" />);
+    fireEvent.submit(screen.getByRole('button', { name: 'Create admin' }).closest('form')!);
+    expect(await screen.findByText(keys.hosting.setup.request_failed)).toBeInTheDocument();
+  });
+});
+
+describe('ConnectionList', () => {
+  afterEach(() => vi.unstubAllGlobals());
+
+  it('retests with the page-prop token and replaces the list', async () => {
+    const fetchMock = stub(200, {
+      checks: [{ name: 'redis', status: 'healthy', detail: 'ok' }],
+    });
+    render(<ConnectionList initial={[]} csrfToken="page-token" />);
+    fireEvent.click(screen.getByRole('button', { name: keys.hosting.setup.connections.retest }));
+    expect(await screen.findByText('redis')).toBeInTheDocument();
+    const [req] = fetchMock.mock.calls[0] as unknown as [Request];
+    expect(new URL(req.url).pathname).toBe('/setup/test-connections');
+    expect(req.headers.get('X-CSRF-Token')).toBe('page-token');
+  });
+
+  it('shows an error line when setup finished in another tab (404)', async () => {
+    stub(404, null);
+    render(<ConnectionList initial={[]} csrfToken="t" />);
+    fireEvent.click(screen.getByRole('button', { name: keys.hosting.setup.connections.retest }));
+    expect(await screen.findByText(keys.hosting.setup.request_failed)).toBeInTheDocument();
+  });
+});

framework/hosting/tests/test_error_page_shared_props.py

@@ -256,3 +256,19 @@ async def test_a_forbidden_page_url_is_relative_too(
         resp = await guarded_client.get(_GUARDED_PATH)
         assert resp.status_code == _FORBIDDEN
         assert _inertia_page(resp.text)["url"] == _GUARDED_PATH
+
+
+class TestSignedInNotFoundCarriesLayoutProps:
+    """The client wraps a signed-in unknown-URL 404 in the app layout (#575).
+
+    That needs ``auth.isAuthenticated`` and a non-empty menu in the page props;
+    pin that the host-level catch-all 404 delivers both.
+    """
+
+    async def test_unknown_url_has_signed_in_auth_and_a_menu(
+        self, authenticated_client: httpx.AsyncClient
+    ) -> None:
+        props = _inertia_page((await authenticated_client.get(_MISSING_PATH)).text)["props"]
+        assert props["auth"]["isAuthenticated"] is True
+        menus = props["menus"]
+        assert menus.get("sidebar") or menus.get("adminSidebar"), sorted(menus)

modules/background_tasks/background_tasks/log_sink.py

@@ -43,8 +43,13 @@
 _handler: QueueSinkHandler | None = None
 
 
+def _clean(value: object, limit: int) -> str:
+    """*value* as text with NULs removed (Postgres rejects them, losing the whole batch)."""
+    return str(value).replace("\x00", "")[:limit]
+
+
 def _text(value: object, limit: int) -> str | None:
-    return None if value in (None, "") else str(value)[:limit]
+    return None if value in (None, "") else _clean(value, limit)
 
 
 def _extra(record: logging.LogRecord) -> dict[str, Any]:
@@ -54,10 +59,10 @@ def _extra(record: logging.LogRecord) -> dict[str, Any]:
             continue
         if len(extra) >= LOG_EXTRA_MAX_KEYS:
             break
-        extra[key[:LOG_NAME_MAX_LENGTH]] = (
+        extra[_clean(key, LOG_NAME_MAX_LENGTH)] = (
             value
             if isinstance(value, bool | int | float)
-            else str(value)[:LOG_EXTRA_VALUE_MAX_LENGTH]
+            else _clean(value, LOG_EXTRA_VALUE_MAX_LENGTH)
         )
     return extra
 
@@ -67,8 +72,8 @@ def to_row(record: logging.LogRecord) -> RetainedLog:
     return RetainedLog(
         ts=datetime.fromtimestamp(record.created, UTC),
         level=record.levelno,
-        logger=record.name[:LOG_NAME_MAX_LENGTH],
-        message=str(record.msg)[:LOG_MESSAGE_MAX_LENGTH],
+        logger=_clean(record.name, LOG_NAME_MAX_LENGTH),
+        message=_clean(record.msg, LOG_MESSAGE_MAX_LENGTH),
         task_id=_text(getattr(record, "task_id", None), LOG_ID_MAX_LENGTH),
         job_id=_text(getattr(record, "job_id", None), LOG_ID_MAX_LENGTH),
         tenant_id=_text(getattr(record, "tenant_id", None), TENANT_ID_MAX_LENGTH),