Repository navigation
feat: add crash-safe ownership accumulation engine - #82
Merged
Merged
Conversation
Implements the ACCUMULATE half of the checkpoint architecture on the B1 substrate: preflight -> HEAD probe -> pinned Commit validation -> guarded init -> versioned scope membership (depth, scope_path_hash) -> pinned history(path:) pages -> grouped (period x work_key) deltas -> own_checkpoint_page CAS, one scope at a time until resolve-ready. Exactly-once is owned by the DB CAS — a committed page replayed with the old cursor returns 'stale' and never double-counts. Slices are bounded (~40s wall clock, ~300 measured GraphQL points, 500-point rate reserve) and a soft per-repo lease inside history_cursor serializes workers; both boundaries stop only at committed page boundaries. ACCUMULATE only: p_identities is always [], no RESOLVE pass, no promotion, no retained snapshots, no identity staging, no production wiring — the single entrypoint stays behind the hard-off OWNERSHIP_INGEST gate. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Two B2 correctness gaps before merge review: - page-deltas: first-observation-wins is_bot collapsed contradictory classifications for one work_key inside a page, hiding drift from the stored-row DB guard. A page-wide frozen work_key -> is_bot map now fails closed via OwnershipIdentityError on any conflicting observation (any order, any month); the DB guard remains the cross-page barrier. - accumulate: GraphQL telemetry was only counted inside the history page loop. One shared per-slice budget now records cost/remaining/resetAt for every request — HEAD probe, pin validation, scope-prep pinned revalidation, history pages, and the identical retry — and gates each subsequent request on cost budget, 500-point reserve, and wall clock. The retry is budget-gated too and its telemetry replaces the failed shape's. Coverage: page-reducer drift matrix (same/reverse/cross-month), engine-level no-checkpoint/no-cursor-move proof, cross-page stored-row rejection, per-request cost accumulation, retry double-cost + authoritative remaining, reserve-stop, and cost-budget-stop — plus real Postgres proofs E11/E12 in the recovery drill. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A thrown own_checkpoint_page call does not prove rollback — Postgres may have applied the deltas, advanced the durable cursor, and committed while the response was lost. Treating every RPC exception as "not committed" could double-apply a page or fail a healthy build. The engine now reconciles the sync row before deciding: a proven commit is adopted; a row still exactly at the attempted position is retried once with the identical payload (the B1 CAS makes 'applied' and 'stale' both safe); a canonical transition elsewhere is followed, never overwritten. Error marking is now position-aware: phase='error' can land only while the durable row matches the exact attempted position (version, full history_cursor, cursor, lease), so a stale worker cannot poison a newer cursor or the ACCUMULATE->RESOLVE handoff. Proven by unit tests UA-UG and real-Postgres proofs E13-E15 in the PG17 recovery drill. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A continuing page (hasNextPage=true) that returns the request cursor — or an empty endCursor — would checkpoint the same boundary and re-page the identical slice forever, double-counting deltas on every commit. Exactly- once assumes each accepted next cursor is a strictly new page boundary. Validate progression before own_checkpoint_page: a continuing page must return a non-empty endCursor different from the request cursor. Violations fail closed through the position-aware error path with OWNERSHIP_HISTORY_FAILED — no deltas persist, the durable cursor stays at the last valid boundary. Final pages are exempt: B1 flips the pass to resolve regardless of the terminal cursor value. Covered by unit tests CA-CE and real-Postgres proof E16 in the PG17 recovery drill. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 1C-B2 — the ACCUMULATE half of the ownership checkpoint engine, on the merged B1 substrate (PR #81, migration 013).
own_checkpoint_pageCAS,own_build_preflight/own_build_init, versioned scope membership, and the hard-offOWNERSHIP_INGEST_*gate.depth, scope_path_hash) → pinnedhistory(path:)pages → grouped(period_start × work_key)deltas →own_checkpoint_page. One scope at a time, ending atpass='resolve'— the B3 handoff.Repository.object(oid: $pinnedOid), verifies__typename === 'Commit'and returnedoid === pinnedOid. One retry on pinned-object failure, then fail-closed — never repins mid-build.'stale'— no double-count. GitHub requests and reduction may repeat; the checkpoint decides.p_identitiesis always[]; no identity staging, nogithub_node_id/display_loginpersistence anywhere.ownership_contributors,ownership_contributor_build_work, andownership_snapshotsall stay empty (asserted in real-PG proofs).runOwnershipAccumulate) sits behind the hard-off gate before any network or DB work; nothing schedules it — no cron, webhook, or UI.Implementation shape
lib/ownership/accumulate.mjs— transport/db-agnostic engine (page loop, lease, §5.6b resumability, dispatch)lib/ownership/accumulate-runtime.mjs— supabase adapter + octokit transport + dormant gated entrypointlib/ownership/page-deltas.mjs— pure page→deltas reducer (reuses canonical classifier +monthStartUtc)lib/ownership/queries.mjs— pinned production queries (ownershipHeadProbeQuery,pinnedObjectQuery,pinnedHistoryPageQuery) + response-contract tripwirescripts/ownership-pg-db.mjs,scripts/ownership-work-projection.mjs,scripts/ownership-engine-proofs.mjs— real-PG drill legstest/ownership-accumulate.test.mjs— 28 unit tests (deltas, payload privacy, pinned-query statics, crash/replay A–K, lease, differential, adversarial partitions)Validation
npm audit(0 vulns), gitleaks cleanTest plan
reduceHistoryToMonthlyGenerated with Devin