Skip to content

Add durable Orrery publication and graph readers to the local provider - #4

Merged
anth-volk merged 6 commits into
mainfrom
feat/orrery-provider
Oct 12, 2026
Merged

anth-volk merged 6 commits into
mainfrom
feat/orrery-provider

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #3

Implementation

Keep one Python distribution, microcosm-emitter, with required telemetry and an internal completed-graph publication library. Resolve the existing extraction conflicts without restoring removed package names or database fallbacks.

  • Keep the generic host separate from telemetry, authentication and database imports. Load selected modules in one separate local process with independent workers, private startup configuration and bounded Unix-socket requests.
  • Keep build adapters socket-only. The separate local service owns telemetry interpretation, event construction, credential redaction, persistence, authentication, graph hashing and delivery. Valid supplied stage identifiers remain unchanged.
  • Preserve SQLAlchemy declarative ORM and Alembic. Upgrade the existing 20261007_01 queue to 20261008_02 for graph jobs while retaining producer sequences, event IDs and upload eligibility. Graph publication requires no telemetry run registration.
  • Preserve typed graph inventories and receipts, exact file bytes, retry delays and expiring worker reservations. Graph queries allow 30 seconds; telemetry acknowledgements, readiness and close requests retain their short limits. Failed startup or unacknowledged requests never claim data was saved by a fallback.
  • Add the existing Microcosm block() outcome and optional fail(error_code=...). The service closes the active stage and records a blocked run atomically, validates collector-required metadata before and after sanitization, and prevents later heartbeat or parent-exit handling from replacing a finished outcome. Microcosm remains responsible for failure classification.
  • Add close_service_on_finish=False for shared-process ownership. Completion, failure and refusal finish telemetry without shutting down the graph socket; explicit outer cleanup closes the host after graph requests. Standalone cleanup remains automatic by default.
  • Preserve existing permanent HTTP client-error handling when registering runs and uploading events: retain only the rejected producer's records locally, except retryable HTTP 408/429. HTTP 401 renews the short-lived credential; network and server failures remain retryable.
  • Keep the separate TypeScript reader package and interfaces. Build one Python wheel/source archive for one PyPI publisher; retain separately verified npm tarball publication.

No collector/dashboard source changes, new infrastructure, credentials, schema changes for outcomes, release or deployment. This does not incorporate unmerged emitter PRs #5–#7; it ports only the existing Microcosm outcome and collector-rejection behavior required by the consumer extraction.

Verification

  • Independent verification found all four approved items clean, including atomic rollback, thin-adapter boundaries, shared-process cleanup ordering and per-producer HTTP rejection handling.
  • Final post-format affected client/module/delivery/subprocess/boundary tests: 135 passed on Python 3.13 and 135 passed on Python 3.14. The earlier full noncollector run passed 180 tests before the last HTTP-rejection change; it is not represented as a final full-suite run.
  • Exact final wheel installed noneditably: all nine collector contract tests passed against pinned collector 280459c8d782658b3fb6098b27e9acba11af29d8 and disposable PostgreSQL 16. Coverage includes blocked metadata, caller-supplied failure codes, rejected identities and lost-ack duplicate delivery, with synthetic credentials and loopback networking only.
  • One wheel/source archive and strict metadata checks passed. Clean installed host, telemetry-only, graph-only and combined-process checks, packaged migrations, blocked outcomes and coded failures passed on Python 3.13 and 3.14.
  • Full Ruff lint/format, Bash syntax, actionlint and diff checks passed. CI/helper pins match; collector qualification installs the downloaded built wheel before tests. No new environment variable or secret is required.
  • The unchanged TypeScript reader previously passed all 25 tests, formatting, types, build and isolated tarball imports/browser-target compilation during conflict resolution. No browser automation or visual inspection.

Publication and infrastructure status

Release validation and qualification precede approval-protected publication of the tested Python artifacts. The separate npm job waits for Python publication and publishes the tested reader tarball. Only publication jobs receive OIDC permission; CI supplies no long-lived publishing credential.

The existing GitHub pypi approval environment and release-tag restriction were previously verified. The Python pending publisher is user-confirmed; private registry settings and the npm package's initial publication/trusted-publisher configuration remain unverified. The live collector's deployed revision and hosted graph API readiness also remain unverified: the compatibility pin proves source behavior, not production deployment.

No registry bootstrap, release, package publication, hosted deployment or consumer activation occurred in this work. No queued GitHub checks are being monitored.

anth-volk and others added 3 commits October 9, 2026 01:47
A job whose preserved directory was removed no longer stops the Orrery
module: it stays pending with preserved_files_missing and backs off, and
claims take the earliest-due job first so other jobs proceed. The latest
publication.status.json copy is written after the queue commit, so a
missing directory cannot roll back the recorded result. The module's tick
reports an unexpected local error once instead of letting the host stop
the module. Type the module's queue and delivery attributes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from codex/extract-telemetry-packages to main October 9, 2026 15:39
@anth-volk
anth-volk marked this pull request as ready for review October 12, 2026 19:42
@anth-volk
anth-volk merged commit 81dafb9 into main Oct 12, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Own immutable graph publication and Orrery graph reads

1 participant