Repository navigation
feat: isolate automatic uploads in a dedicated worker - #70
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A general supervisor processes every eligible directory, including older manual runs. A worker dedicated to new dashboard uploads needs its own scope so starting it cannot unexpectedly resume those runs or inherit a source-specific paid exception.
Add
orchestrator.supervisor --uploads-only --watch. It discovers automatic uploads after startup, preserves pause/terminal controls, skips malformed and symlinked manifests, and rechecks scope before starting an agent. Startup rejectsWANDER_STAGE_AUTHORIZATIONinstead of transferring an existing clip's exception to other uploads.Worker heartbeat version 2 records the processing mode. Upload checks and per-run progress match that capability: an automatic-only worker cannot make a manual request look serviceable. The updated reader preserves support for unrestricted version-one workers; old API readers ignore version-two records. Update API and worker together.
Validation includes an upload submitted through the real API after supervisor startup, a real RunSession with an inert harness, and malformed-neighbor cases that still allow a healthy run to finish. These tests make no provider calls. A local empty-queue rehearsal against the existing runs root exited successfully with all 39 run/state/journal/control files unchanged. Read-only worker preflight found credential configuration and Modal volume access available; account quotas and reconstruction quality remain unverified.
This PR prepares worker isolation and startup; it does not enable ongoing paid processing, change spending limits, reset existing attempts, or change reconstruction quality gates. The existing bottle worker remains scoped to its run.
Validation: all four CI jobs passed on
c30c94f, including 1,461 Python tests. The final local focused suite passed 174 tests; Ruff, formatting, and diff checks passed. An independent review found malformed-manifest isolation and manual-run compatibility issues; both were fixed and rechecked with no remaining findings.The owned local API now uses the prepared credentials and this revision. Direct API and admin proxy checks return HTTP 200; credential-presence checks pass and the general upload worker is accurately reported as unavailable until enabled. No paid provider calls were made.