You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 90e69f1
Browse filesBrowse the repository at this point in the historyBrowse files
- file_storage: stream S3 reads, get_range on backends, 206/416 Range
support on downloads; S3 get() is awaited like filesystem (#453)
- background_tasks: heartbeat() API + per-task stuck_after (#451);
retries publish after commit, enqueue_after_commit helper (#452)
- users: /auth/token persists the upgraded password hash (#457)
- hosting/cli: regenerate i18n keys into the installed npm package in
scaffolded apps, smpy host gen-i18n (#449)
- cli: scaffolded make kill stops only this app's ports (#461)
Closes#449Closes#451Closes#452Closes#453Closes#457Closes#461
Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
Copy file name to clipboardExpand all lines: docs/framework/i18n.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -283,6 +283,8 @@ The detection logic lives in `scripts/lib/untranslated-strings.mjs` behind its o
283
283
284
284
`packages/i18n/src/keys.generated.ts` is generated over every **installed** module, not just the ones the running host activated, and `make ci-js-typecheck` runs `tsc -p` for each `modules/*/tsconfig.json` and `packages/*/tsconfig.json` in the workspace. (A module shipping `.tsx` with no `tsconfig.json` fails the target outright, rather than being silently skipped — the same gap `SM017` reports.)
285
285
286
+
**Scaffolded apps.** There is no `packages/i18n` checkout in a scaffolded app: `@simple-module-py/i18n` is an npm dependency. When `packages/i18n/src` is absent, development boot (and `make gen-i18n` / `smpy host gen-i18n`) writes `keys.generated.ts` and `generated-resources.ts` into the nearest `node_modules/@simple-module-py/i18n/src` found by walking up from the project root (so a hoisted npm-workspace install is found; a flat host also checks `client_app/node_modules`). The shared `ui.*` and framework keys come from catalogs inside `simple_module_hosting`, and any keys already in the published file are kept (the published catalog covers modules this app does not install, so they are not treated as removals). `npm install` restores the published file; `make install` regenerates afterwards. With neither location present (a wheel-installed production app) boot skips silently; `make gen-i18n` fails loudly.
287
+
286
288
That distinction is load-bearing: with a filtered union, translating an inactive auth provider's pages (say `keycloak`, when this host runs `users`) would break its build on the next regeneration while the app itself ran fine. The runtime registry stays filtered — an inactive module's strings are typed, never served.
287
289
288
290
`t()` needs literal keys to typecheck; a key assembled at runtime won't resolve.
Copy file name to clipboardExpand all lines: docs/guide/installation.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -165,7 +165,7 @@ Pass `--loosen` to rewrite every constraint to `>=<latest>` instead.
165
165
Run `uv tool update-shell` (or restart your shell) so the tool's bin dir is on PATH.
166
166
167
167
**Port already in use.**
168
-
Free `:8000` and `:5050` before the next `make dev`. On Linux/macOS: `lsof -ti:8000,5050 | xargs kill -9`.
168
+
Run `make kill` to free the app's API and Vite ports (`:8000` and the port in `SM_VITE_DEV_URL`, default `:5050`) before the next `make dev`. It only touches those ports, never other apps' servers. To run two apps side by side, give each its own `API_PORT` (`make dev API_PORT=8010`) and `SM_VITE_DEV_URL`.
169
169
170
170
**Alembic complains about a revision mismatch.**
171
171
The DB is ahead of or behind the migration files. For a dev DB: `rm app.db && make migrate`. For Postgres: `docker compose down -v && docker compose up -d postgres && make migrate`.
Every dispatch / start / completion / failure / retry / revoke is recorded in the `background_tasks_task_execution` table by Celery signal handlers, so you can see history even after the Celery result backend has expired the result.
34
34
35
+
## Enqueueing from inside a transaction
36
+
37
+
`send_receipt.delay(...)` publishes *now*. If the request that called it is still writing the row the task will read, two things go wrong: a fast worker (or eager mode, which runs the task inside the publish) starts before the row is visible, and if the commit then fails the task is queued against a row that never existed. Publish after commit instead:
`enqueue_after_commit(db, task, *args, **kwargs)` sends `task` once `db`'s transaction commits and drops it if the transaction rolls back or the commit fails. It returns the Celery task id the message will carry, reserved up front, so you can store it on your own row in the same transaction.
52
+
53
+
-`task` is a task (`send_receipt`), a signature for routing options (`send_receipt.s().set(queue="mail", countdown=30)`), or a registered task name (`"orders.send_receipt"`, sent through `celery.current_app`).
54
+
- In a request (`get_db`), the publish runs from the session's `on_commit` hook — after the commit, before the response leaves.
55
+
- Any other session — a task body's sync session, a CLI script's own `async with session.begin()` — hangs it off SQLAlchemy's `after_commit` event instead, and drops it when the outermost transaction ends without committing. Publishing then blocks the committing thread for the length of one kombu publish.
56
+
- The task runs as the tenant bound when you called `enqueue_after_commit`, not whatever is bound at commit time.
57
+
- A publish that fails after the commit is logged, not raised: the transaction is already durable.
58
+
59
+
The admin retry buttons follow the same rule — the new `pending` row commits first, then the task is sent (GH #452). An orphaned `pending` row that was never sent is visible and retryable; a task with no row is neither.
60
+
61
+
## Long-running tasks: heartbeats and stuck thresholds
62
+
63
+
The beat sweep flips a `running` row to `stuck` once its `heartbeat_at` is older than the stuck threshold (`stuck_after_seconds`, default 300), which makes it retry-eligible. The signal handlers only stamp the heartbeat when a task starts and finishes, so a task that runs longer than the threshold must say it is still alive, or it can be retried — and run twice — while the first run is still working.
64
+
65
+
Call `heartbeat()` from the task body's loop:
66
+
67
+
```python
68
+
from background_tasks import heartbeat
69
+
from celery import shared_task
70
+
71
+
72
+
@shared_task(name="reports.rebuild")
73
+
defrebuild(report_id: int) -> None:
74
+
for chunk in chunks(report_id):
75
+
process(chunk)
76
+
heartbeat()
77
+
```
78
+
79
+
It refreshes `heartbeat_at` on the current execution's row (found through Celery's `current_task`), writing at most once every 30 seconds per task however often you call it, so calling it on every iteration is fine; `heartbeat(force=True)` skips the throttle. Outside a running task it does nothing and returns `False`, so shared helpers can call it unconditionally. It only touches a row that is still `running` — a heartbeat never revives a row the sweep already declared stuck.
80
+
81
+
A task whose steps are legitimately longer than the global threshold can declare its own, in seconds:
Celery copies extra decorator options onto the task class; the sweep reads `stuck_after` off the task registry of the worker running it (which autodiscovers every module's tasks), and falls back to `SM_BG_TASKS_STUCK_AFTER_SECONDS` / the default for tasks without one. No column or migration is involved, so the threshold always matches the deployed code. A shorter `stuck_after` works too, for tasks that should never take long. Keep any threshold well above 30 seconds, or the heartbeat throttle alone can let a live task look stale.
89
+
90
+
Not provided: an automatic heartbeat thread. A task that blocks in one long call (a single huge query, an external process) cannot heartbeat from inside it — give it a `stuck_after` that covers the call instead.
91
+
35
92
## Scheduling periodic work (beat)
36
93
37
94
A module schedules recurring work the same way it registers tasks — by shipping a `tasks.py` — and additionally exporting a module-level `BEAT_SCHEDULE` dict. `build_celery` merges every installed module's `BEAT_SCHEDULE` into the beat schedule at boot (identically in the web and worker processes), so `make beat` runs them alongside the built-ins:
@@ -145,7 +202,7 @@ DB-backed; defaults are in `BackgroundTasksSettings`. Several are marked `requir
|`stuck_after_seconds`|`300`| heartbeat-staleness threshold; a task's own `stuck_after` overrides it — see [Long-running tasks](#long-running-tasks-heartbeats-and-stuck-thresholds)|
149
206
|`stuck_sweep_interval_seconds`|`60`| beat cadence for the stuck sweep |
150
207
|`purge_interval_seconds`|`86_400`| beat cadence for old-row purge |
151
208
|`retention_days`|`14`| how long to keep terminal rows; range 1-3650 |
@@ -170,7 +227,7 @@ Bootstrap env-var equivalents (`SM_BG_TASKS_*`) only seed pydantic defaults at f
170
227
171
228
| Task | Schedule | What |
172
229
|---|---|---|
173
-
|`background_tasks.sweep_stuck_tasks`| every `stuck_sweep_interval_seconds`| flips `running` rows with stale heartbeats to `stuck`|
230
+
|`background_tasks.sweep_stuck_tasks`| every `stuck_sweep_interval_seconds`| flips `running` rows whose heartbeat is older than their task's `stuck_after` (else `stuck_after_seconds`) to `stuck`|
174
231
|`background_tasks.purge_old_executions`| every `purge_interval_seconds`| deletes terminal rows older than `retention_days`|
175
232
176
233
A `background_tasks.demo_echo` task is also registered for smoke tests; not scheduled.
@@ -202,7 +259,7 @@ Synchronous handlers in `signals.py`, all bound through Celery's `signals.<name>
202
259
|---|---|
203
260
|`before_task_publish`| inserts a `pending` row |
204
261
|`task_prerun`| flips to `running`, stamps `started_at` + `heartbeat_at`|
205
-
|`task_postrun`| refreshes `heartbeat_at`|
262
+
|`task_postrun`| refreshes `heartbeat_at`(between prerun and postrun, only `heartbeat()` from the task body does) |
206
263
|`task_success`| flips to `success`, stores `result`, sets `finished_at`|
207
264
|`task_failure`| flips to `failed`, stores `traceback` + `exception_type`, publishes `TaskFailed`|
208
265
|`task_retry`| flips to `retrying`, increments `retries`|
`get` and `get_range` must stream in bounded chunks (never buffer the whole
263
+
object) and raise `StorageNotFoundError` from the awaited call, before the
264
+
first byte. `get_range(key, start, end)` uses HTTP semantics — `end` is
265
+
inclusive — and receives an already-validated `0 <= start <= end < size`. A
266
+
backend without `get_range` still works: the endpoints then ignore `Range`.
267
+
238
268
Register it in your module's `register_settings` so the file_storage service can pick it up. (The shipped two backends are wired into a private `BACKEND_REGISTRY`; see `backends/__init__.py` for the pattern.)
A single dispatcher pair (`/api/users/auth/{provider}/login` + `/callback`) serves every provider; the client is resolved per request from the cache. The `/callback` returns a 303 redirect (not the stock fastapi-users 204) so Inertia lands on a real page, with the auth cookie attached. Find-or-create + email association goes through `UserManager.oauth_callback` (`associate_by_email=True`, `is_verified_by_default=True`); state CSRF uses the signed session cookie. Linked accounts are stored in `users_oauth_account`. A **newly provisioned** OAuth account is marked external (see below); an existing password account linked by email keeps its password and is left unchanged.
386
386
387
+
## Importing users from another system
388
+
389
+
`users` stores passwords through fastapi-users' default `PasswordHelper`, so you can import `hashed_password` values from another system and users keep their existing passwords (no reset needed). Accepted formats:
390
+
391
+
| Format | Prefix | Role |
392
+
|---|---|---|
393
+
| Argon2 (`argon2id`/`argon2i`/`argon2d`, PHC string) |`$argon2...`| Current scheme, used for all new hashes |
Other formats (PBKDF2, scrypt, SHA-based, plain text) are **not** recognised, and such users cannot sign in until they reset their password. A bcrypt hash is verified as-is and transparently re-hashed to Argon2 on the user's next successful sign-in, through both the session login (`/api/users/auth/login`) and the bearer-token login (`/api/users/auth/token`). Leave `hashed_password` as `NULL` for accounts that should only sign in through an external IdP (see below).
397
+
387
398
## External / SSO users
388
399
389
400
Users created through an external IdP (Google, GitHub, Microsoft/Entra ID, or generic OIDC) are provisioned with `is_external=True` and **no local password** (`hashed_password` is `NULL`). The OAuth callback's find-or-create only nulls the password + marks external for accounts it *creates* — an existing password account that gets linked by email keeps its password and stays non-external. External users sign in **only** through their IdP; roles are still assigned locally like any other user.
0 commit comments