Skip to content

Delegate local telemetry and Orrery publication to provider packages (deferred) - #1153

Draft
anth-volk wants to merge 7 commits into
feat/minimal-orrery-publicationfrom
feat/provider-delegation
Draft

anth-volk wants to merge 7 commits into
feat/minimal-orrery-publicationfrom
feat/provider-delegation

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1152

Implemented and tested now; intentionally defer merge and activation. Stacked on #1151, replacing embedded delivery with a thin Microcosm metadata adapter. Depends on PolicyEngine/microcosm-emitter#4, stacked on that repository’s #2. Orrery runs-app read consumer: PolicyEngine/microcosm-runs#4.

Microcosm retains actual recorded-execution graph generation, explicit public-file selection, publication-before-HF ordering and artifact receipt snapshots. One provider host runs independent telemetry and graph workers. Authentication, persistence, retries, resource sampling and socket lifecycle belong to provider packages. Removes embedded service and duplicate tests from this branch; they remain recoverable from Git history. Calibration dashboard unchanged.

Local validation:

  • Complete engine-free suite: 15,004 passed, 136 skipped; successful process exit.
  • 18 adapter/export focused tests and full-build focused checks pass.
  • Both existing synthetic UK command-line integration tests pass on this branch and Publish UK Orrery graphs with the existing local build service #1151.
  • Seven wheels build and pass archive inspection; latest adapter wheel rebuilt and inspected after final changes.
  • Lint, changed-file formatting and test-plan verification pass.
  • The preceding minimal implementation also passes its complete engine-free suite.

The first broad attempt encountered a non-UTF-8 joblib fixture in an extra local integration environment. That generated environment was moved outside the source tree; the affected module and the complete suite now pass. No source-test workaround was added. No new cross-stack integration tests or CI monitoring.

Before merge: merge and publish all four provider Python distributions after registry-owner trust confirmation; remove exact Git-source overrides from microcosm-build/pyproject.toml; regenerate uv.lock and its approved worker-identity digest; rerun checks. A successful local Microcosm wheel build alone does not make unpublished dependencies installable. Preserve versioned queues; old unversioned queues must first be upgraded by preceding Microcosm code. No new runtime variables, secrets, hosted services, package releases or deployments.

History-only correction: restacked the single migration commit onto the corrected #1151, which is now based on #1144. The final source tree is identical to the previously tested version; Git tree comparison and range-diff pass. No test-suite rerun was needed for this history-only correction.

Publication types (2026-10-09)

Restack on #1151's explicit publication contracts and import the provider's named inventory/result types directly, without retaining a duplicate local type module. Pin all four provider packages to 06e47847983ee67ccdb6a596062251322d07f793, using the independently verified canonical repository URL PolicyEngine/microcosm-emitter. Regenerate uv.lock and its approved worker-identity digest.

Latest validation: 105 targeted adapter/export/staging/full-build checks pass before the final static-check corrections; the final 18 adapter/export checks pass again. Explicit namespace-aware static checks for the adapter and exporter pass, along with lint, formatting, test-plan verification and the rebuilt adapter wheel. The complete 15,004-test engine-free result above predates this type-focused update and was not rerun. Merge and activation remain deferred. No CI monitoring, merge, release or deployment.

anth-volk and others added 6 commits October 9, 2026 18:11
A failed UK build now publishes its graph evidence to orrery-failure/ in the
attempt evidence directory instead of the output directory, so an earlier
successful bundle and its completion marker stay consistent. Interrupts keep
local evidence and the queue entry without waiting for an upload or staging
to Hugging Face, and a failure while saving evidence no longer replaces the
build's own error.

The graph worker survives local queue errors. A job whose preserved directory
was removed stays pending with preserved_files_missing; the status file is
written after the queue commit, and claims take the earliest-due job first.

Remove the unused socket publication action, share origin validation with the
collector, return typed collector credentials and graph finalization results,
type the ORM publication columns, and replace private cross-module access with
public accessors. Update the publication guide and add regression tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk force-pushed the feat/minimal-orrery-publication branch from 25e4ceb to 6f93d06 Compare October 9, 2026 14:36
@anth-volk
anth-volk force-pushed the feat/provider-delegation branch from 0aa9d48 to 2e6b0ab Compare October 9, 2026 14:57
@anth-volk
anth-volk force-pushed the feat/minimal-orrery-publication branch from 6f93d06 to 79d1624 Compare October 9, 2026 15:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant