Repository navigation
Runtime batch 2: task progress, retained logs, Sentry, leases, in-process outbox (#536 #537 #538 #539 #565) - #573
Merged
Merged
Conversation
heartbeat(progress=, message=, phase=) buffers under the existing throttle;
TaskExecution gains progress_* and an indexed owner_user_id stamped from the
sm_owner_id header (propagated through tasks that enqueue tasks, kept on
retry). GET /api/background_tasks/tasks/{id}/status is readable by the owner
in the active tenant or PERM_VIEW holders, 404 otherwise. Migration
bb906e1a1089.
Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
simple_module_core.leases: lease(name, ttl, owner=None) handle with token, strictly increasing fence, refresh()/release() and an async context manager (blocking/wait); semaphore(name, slots, ttl); LeaseBackend Protocol, InProcessLeaseBackend and configure_leases()/get_leases(). Backends: Redis (SET NX PX, Lua compare-and-set refresh/release, semaphore ZSET scored by Redis TIME, INCR fence) in simple_module_hosting; the framework-owned sm_lease table (DB clock, ON CONFLICT claims) in simple_module_db with migration d263ea727663, a new root labelled "leases". create_app and the Celery worker bootstrap select Redis, else the table, else in-process. An expired owner can neither refresh nor release. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
# Conflicts: # CHANGELOG.md # host/tests/test_migration_graph.py
# Conflicts: # CHANGELOG.md # docs/.vitepress/config.ts # framework/hosting/simple_module_hosting/app_builder.py
…eps app_builder under the cap Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
… and nested secrets from events Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…DB-backed retention (#565) Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
# Conflicts: # CHANGELOG.md # modules/background_tasks/background_tasks/constants.py
Opt-in DB log sink (log_sink_enabled), LogSink/QueueSinkHandler in hosting, Logs tab + admin Logs page, retention in purge_old_executions. Migration 1f5985daa10b. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
# Conflicts: # modules/background_tasks/background_tasks/constants.py # modules/background_tasks/background_tasks/module.py # modules/background_tasks/background_tasks/tasks.py
…deflake eager outbox test Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
- Every event field now goes through the recursive pass (tags, server_name, top-level stacktrace, threads, unknown future fields), not a field list. - Key matching: pwd/passphrase/jwt/bearer/card words, whole-word auth/pass/otp/pin/sid/sig/ssn/cvv/mfa, camelCase-aware (basicAuth). - [key, value] pairs in header/query lists, bytes values, quoted values with spaces, escaped JSON, whole Cookie:/Authorization: lines, userinfo with '@' or '/' in the password, JWTs and bare 32+ char tokens are redacted; db.params and frame vars dropped anywhere; user reduced to id. - The old key=value regex was quadratic (20 KB string: 72 s in before_send); all patterns are now boundary-anchored and bounded, strings capped at 64 KiB. - before_send drops the event if tagging/scrubbing raises, logging only the exception type. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…oker URL in startup log Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…; wrap redact docstring Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…ined-log sink log_sinks.redact() now uses sentry_scrub_text.scrub_text so URL passwords with @ or /, quoted values, escaped JSON, JWTs, long tokens and Cookie lines are masked identically on both paths; hosting tests no longer leak the Sentry flag. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
# Conflicts: # CHANGELOG.md # modules/background_tasks/background_tasks/locales/en.json
…r for log purge Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
… lease key namespaces Structured extras (dicts/lists/objects) bypassed redaction in the retained-log sink; they now go through the Sentry recursive scrubber. Redis holder and fence keys use separate prefixes so a lease named 'foo:fence' cannot collide with foo's counter. Also makes the Sentry route test independent of the git sha embedded in the release field. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…writes; tolerate NULL task args Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…GELOG for audit_exclude log skip Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
Resolve background_tasks conflicts keeping both sides: worker DB timeout settings (WorkerDatabaseSettings) alongside LogSinkFields/outbox_mode; runner configures DB options before leases and settings hydration, then re-applies them and installs the log sink; main's configure_database folded into _lifecycle.startup. CHANGELOG keeps all entries. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
… level in sink test, CHANGELOG model name Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
…ted() for SM031 Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Deploying simple-module-python with
|
| Latest commit: |
24bef27
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://f1631639.simple-module-python.pages.dev |
| Branch Preview URL: | https://feat-runtime-batch-2.simple-module-python.pages.dev |
CI's xdist workers share one Redis service; the teardown pattern sm:*:t-* deleted another worker's live semaphore, letting all 8 waiters in. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
re.match with $ accepted "ValueError\n"; use fullmatch. The too-long test case could not be stored on Postgres (varchar(255)); replace it with newline cases that can. Claude-Session: https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runtime batch 2: task progress, retained logs, optional Sentry, distributed leases, and an in-process outbox. Design:
docs/superpowers/specs/2026-10-21-runtime-batch-2-design.md.What changes
TaskExecutiongains progress columns andowner_user_id.heartbeat(progress=(i, n), message=, phase=)records progress.sm_owner_idheader.GET /api/background_tasks/tasks/{id}/status: only the owner can read it, or a holder ofPERM_VIEW. Anyone else gets a 404. The route requiresAuthenticated().bb906e1a1089.QueueSinkHandlerin hosting: batched, bounded queue, redacted, never re-logs its own writes.background_tasks_log_recordtable, migration1f5985daa10b.log_sink_enabled,log_sink_level,log_retention_days; old logs are purged by the same pass that purges old executions.simple_module_hosting[sentry]extra.init_sentrydoes nothing unlessSM_SENTRY_DSNis set.simple_module_core.leases:lease()andsemaphore()with fencing tokens.sm_lease, root revisiond263ea727663, labelleases) and in-process.outbox_modesetting (celeryorinprocess). It resolves toinprocessunder eager Celery.inprocessmode a sweep loop runs in each web process, so the outbox works without a worker.The migration graph test now covers the new
leasesroot and thebackground_tasksrevisions.Verification
make lint✓make test-py: 5382 passed, 36 skipped ✓make test-js: 676 passed ✓make ci-doctor✓uv lock --check✓The Redis variants of the lease tests run only in CI (no local
redis-server)./ship
Closes #536
Closes #537
Closes #538
Closes #539
Closes #565
https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4