ops: reconciliation must be a union that preserves published-but-store-lost records - #211
Merged
Merged
Conversation
…e-lost records New plutus_agent/reconcile_union.py: union reconciliation between the tracked ledger (published JSONL records) and the store (usage_events). - published-but-store-lost (tracked only): preserved and recovered into the store; the recovery reason is recorded on the record (reconciliation_note, new v17 column, excluded from the hash-chain canonical form so existing chains stay byte-identical) and in the new reconciliation_events journal table. - store-only: preserved; never deleted without the explicit operator flag (drop_missing=True). - both sides: compared by row hash; fresh store state wins on conflict and the divergence reason is journaled - no silent status rewrite. - dry run by default (apply=False): surfaces every action, including all would-be drops, before any mutation. - recovered rows re-insert with NULL chain fields (pre-chain prefix), the original row hash is kept in the journal. - tests: 6 new tests covering both one-sided cases, the both-sides conflict, dry-run-before-mutation, and the operator-flag deletion gate. Full suite: 617 passed, 3 skipped; 2 pre-existing failures (engine secret test, live-balance collect test) reproduced on clean main at c0e4514 - unrelated to this change. SCHEMA_VERSION 16 -> 17 with the three hardcoded version asserts updated. Closes #207
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements union reconciliation between the tracked ledger and the store (ledger #207).
New
plutus_agent/reconcile_union.py—reconcile_union(conn, tracked_records, *, apply=False, drop_missing=False):apply=False): surfaces every action — including all would-be drops (would_droplist) — before any mutation; dry runs write nothing to the store and nothing to the journal.reconciliation_note, new v17 column) and in the newreconciliation_eventsjournal table. Recovered rows re-insert with NULL chain fields (verifyreports them as pre-chain — the store never attests a hash it didn't produce), and the originalrow_hashis preserved in the journal.drop_missing=True).reconciliation_noteis excluded from the hash-chain canonical form, so existing chains are byte-identical after the migration (SCHEMA_VERSION 16→17, additive column + additive journal table).Tests (actual output)
python -m pytest tests/test_reconcile_union.py -q→6 passed in 0.22spython -m pytest -q→ 617 passed, 3 skipped, 2 failed — the 2 failures (test_engine.py::test_save_strips_env_secret,test_plutus.py::test_live_balance_populated) were reproduced on clean main at c0e4514 (stash check) and are pre-existing/unrelated.test_cache_write.py/test_external_ref.pyupdated 16→17.git diff --check→ cleanAcceptance criteria
Closes #207