Skip to content

ops: reconciliation must be a union that preserves published-but-store-lost records - #211

Merged
tcconnally merged 1 commit into
mainfrom
fix/issue-207-reconcile-union
Aug 4, 2026
Merged

ops: reconciliation must be a union that preserves published-but-store-lost records#211
tcconnally merged 1 commit into
mainfrom
fix/issue-207-reconcile-union

Conversation

@tcconnally

Copy link
Copy Markdown
Contributor

Summary

Implements union reconciliation between the tracked ledger and the store (ledger #207).

New plutus_agent/reconcile_union.pyreconcile_union(conn, tracked_records, *, apply=False, drop_missing=False):

  • Dry run by default (apply=False): surfaces every action — including all would-be drops (would_drop list) — before any mutation; dry runs write nothing to the store and nothing to the journal.
  • Published-but-store-lost (tracked only) → preserved and recovered into the store; the recovery reason is recorded on the record (reconciliation_note, new v17 column) and in the new reconciliation_events journal table. Recovered rows re-insert with NULL chain fields (verify reports them as pre-chain — the store never attests a hash it didn't produce), and the original row_hash is preserved in the journal.
  • Store-only → preserved; never deleted without the explicit operator flag (drop_missing=True).
  • Both sides → compared by row hash; fresh store state wins on conflict, divergence reason journaled, store untouched — no silent status rewrite.
  • reconciliation_note is excluded from the hash-chain canonical form, so existing chains are byte-identical after the migration (SCHEMA_VERSION 16→17, additive column + additive journal table).

Tests (actual output)

  • New: python -m pytest tests/test_reconcile_union.py -q6 passed in 0.22s
    • both one-sided cases (tracked-only recovery with reason; store-only preservation)
    • both-sides conflict (fresh store wins, reason journaled, no rewrite)
    • dry-run surfaces would-be drops before mutation
    • no deletion without the explicit operator flag
  • Full suite: python -m pytest -q617 passed, 3 skipped, 2 failed — the 2 failures (test_engine.py::test_save_strips_env_secret, test_plutus.py::test_live_balance_populated) were reproduced on clean main at c0e4514 (stash check) and are pre-existing/unrelated.
  • Version asserts in test_cache_write.py / test_external_ref.py updated 16→17.
  • git diff --check → clean

Acceptance criteria

  • Dry-run reconciliation surfaces all would-be drops before any mutation
  • A record present only in the tracked ledger is preserved and its recovery reason is recorded (on the record + journal)
  • A record present only in the store is preserved
  • No silent status rewrite: every divergence records its resolution reason (journal)
  • Unit tests for both one-sided cases and the both-sides conflict case

Closes #207

…e-lost records

New plutus_agent/reconcile_union.py: union reconciliation between the
tracked ledger (published JSONL records) and the store (usage_events).

- published-but-store-lost (tracked only): preserved and recovered into the
  store; the recovery reason is recorded on the record
  (reconciliation_note, new v17 column, excluded from the hash-chain
  canonical form so existing chains stay byte-identical) and in the new
  reconciliation_events journal table.
- store-only: preserved; never deleted without the explicit operator flag
  (drop_missing=True).
- both sides: compared by row hash; fresh store state wins on conflict and
  the divergence reason is journaled - no silent status rewrite.
- dry run by default (apply=False): surfaces every action, including all
  would-be drops, before any mutation.
- recovered rows re-insert with NULL chain fields (pre-chain prefix), the
  original row hash is kept in the journal.
- tests: 6 new tests covering both one-sided cases, the both-sides conflict,
  dry-run-before-mutation, and the operator-flag deletion gate.

Full suite: 617 passed, 3 skipped; 2 pre-existing failures (engine secret
test, live-balance collect test) reproduced on clean main at c0e4514 -
unrelated to this change. SCHEMA_VERSION 16 -> 17 with the three hardcoded
version asserts updated.

Closes #207
@tcconnally
tcconnally merged commit 1a7be9a into main Aug 4, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ops: reconciliation must be a union that preserves published-but-store-lost records

1 participant