Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f301693
feat: add offline archive root relocation
Sinity Aug 13, 2026
2275ab2
fix: permit relocation receipt recovery
Sinity Aug 13, 2026
4d988e3
test: type relocation train evidence
Sinity Aug 13, 2026
ee1f4fa
docs: point operator examples at relocated archive
Sinity Aug 13, 2026
17f5730
fix: require source continuity before root relocation
Sinity Aug 13, 2026
5fe8c4d
test: type relocation continuity evidence
Sinity Aug 13, 2026
3fc3d5d
fix: expose relocation subcommands to inventory
Sinity Aug 13, 2026
b529110
feat: recover authenticated historical source continuity
Sinity Aug 13, 2026
903a755
fix: prove complete historical source delta
Sinity Aug 13, 2026
69715f9
fix: harden historical continuity receipts
Sinity Aug 13, 2026
d011f2b
fix: harden archive relocation proof boundaries
Sinity Aug 13, 2026
e102d5f
test: reject archive relocation proof substitutions
Sinity Aug 13, 2026
d954b6e
fix: bind archive relocation recovery evidence
Sinity Aug 13, 2026
062a371
fix: preserve continuity through archive relocation
Sinity Aug 13, 2026
8cd1316
fix: relocate active index pointers with archive roots
Sinity Aug 13, 2026
73d1516
test: cover historical source continuity recovery
Sinity Aug 14, 2026
a3c447b
test: type recovery interruption seams
Sinity Aug 14, 2026
77001a5
fix: bind recovery evidence through apply
Sinity Aug 14, 2026
b246e55
test: type pointer publication regression
Sinity Aug 14, 2026
b7f873a
test: satisfy pointer type contract
Sinity Aug 14, 2026
30b0123
fix: bind archive relocation recovery authority
Sinity Aug 14, 2026
41f6fed
fix: bind no-rebind relocation continuity
Sinity Aug 14, 2026
6c288a3
docs: correct moved-root relocation backup flow
Sinity Aug 14, 2026
2561771
fix: pin retained continuity refresh proofs
Sinity Aug 14, 2026
ef4c08c
docs: require complete relocation backup tiers
Sinity Aug 14, 2026
22fc5cf
fix: chain archive relocation authority
Sinity Aug 14, 2026
8dee193
docs: align relocation recovery contract
Sinity Aug 14, 2026
bf61c7d
fix: close relocation admission gaps
Sinity Aug 14, 2026
c3cc3d2
fix: make continuity recovery resumable
Sinity Aug 14, 2026
80b3445
fix: chain archive relocation authority
Sinity Aug 14, 2026
acb9e49
docs: sync relocation output schemas
Sinity Aug 14, 2026
d3c9cd2
fix: close relocation authority proof gaps
Sinity Aug 14, 2026
8fb8c83
fix: seal relocation authority transitions
Sinity Aug 14, 2026
42ae727
fix: seal index generation leaf identities
Sinity Aug 14, 2026
39507e6
fix(ops): finish relocation continuity repairs
Sinity Aug 14, 2026
e2a9fa5
fix(ops): bind relocation continuity authority
Sinity Aug 14, 2026
79ad235
fix(ops): preserve legacy maintenance recovery authority
Sinity Aug 14, 2026
5fddbe5
docs: clarify relocation backup evidence
Sinity Aug 14, 2026
f61b272
fix(ops): preserve prepared relocation recovery
Sinity Aug 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions devtools/command_catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -574,9 +574,9 @@ def to_dict(self) -> dict[str, object]:
"archive and never writes into polylogue/scenarios/ on its own."
),
examples=(
"devtools demo real-slice-screen --archive-root /realm/db/polylogue "
"devtools demo real-slice-screen --archive-root /realm/state/polylogue "
"--session claude-code-session:<id>:<agent> --out .agent/scratch/real-slice",
"devtools demo real-slice-screen --archive-root /realm/db/polylogue "
"devtools demo real-slice-screen --archive-root /realm/state/polylogue "
"--refs-file refs.txt --out .agent/scratch/real-slice",
),
),
Expand Down
16 changes: 16 additions & 0 deletions devtools/render_cli_output_schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
from polylogue.archive.query.metadata import terminal_query_cli_surfaces, terminal_query_source_list
from polylogue.cli.commands.maintenance._migrate_tier import MigrateTierResultPayload
from polylogue.operations.action_contracts import ActionAffordanceListPayload
from polylogue.operations.archive_root_relocation import ArchiveRootRelocationResult
from polylogue.operations.historical_source_continuity_recovery import HistoricalSourceContinuityRecoveryResult
from polylogue.surfaces.payloads import (
ArchiveDebtListPayload,
ImportExplainPayload,
Expand Down Expand Up @@ -270,6 +272,20 @@ class CliOutputSchema:
model=MigrateTierResultPayload,
surfaces=("polylogue ops maintenance migrate-tier --output-format json",),
),
CliOutputSchema(
name="archive-root-relocation-result",
title="Archive Root Relocation Result",
description=("Result from the offline archive-root relocation apply command."),
model=ArchiveRootRelocationResult,
surfaces=("polylogue ops maintenance archive-root-relocation apply --output-format json",),
),
CliOutputSchema(
name="historical-source-continuity-recovery-result",
title="Historical Source Continuity Recovery Result",
description=("Result from the one-purpose pre-#3868 historical source continuity recovery apply command."),
model=HistoricalSourceContinuityRecoveryResult,
surfaces=("polylogue ops maintenance source-continuity-recovery apply --output-format json",),
),
CliOutputSchema(
name="machine-error",
title="Machine Error Envelope",
Expand Down
13 changes: 12 additions & 1 deletion devtools/verify_distribution_surface.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
"polylogue.mcp.cli",
"polylogue.archive.query.expression",
)
PACKAGE_RESOURCES = (("polylogue.operations", "historical-source-continuity-operation-20260807.json"),)


class DistributionVerificationError(RuntimeError):
Expand Down Expand Up @@ -98,6 +99,10 @@ def _verify_wheel_surface(wheel: Path) -> None:
names = set(archive.namelist())
if "polylogue/_build_info.py" not in names:
raise DistributionVerificationError(f"{wheel.name} is missing polylogue/_build_info.py")
for package, resource in PACKAGE_RESOURCES:
resource_path = f"{package.replace('.', '/')}/{resource}"
if resource_path not in names:
raise DistributionVerificationError(f"{wheel.name} is missing package resource {resource_path}")
entry_points = _read_entry_points(archive)
for script in RUNTIME_SCRIPTS:
if f"{script} =" not in entry_points:
Expand Down Expand Up @@ -135,7 +140,13 @@ def _smoke_installed_wheel(wheel: Path, install_dir: Path) -> None:
def _probe_runtime_imports(python: Path, install_dir: Path, env: dict[str, str]) -> None:
"""Import runtime entrypoint modules from the installed wheel environment."""
modules_literal = repr(RUNTIME_IMPORT_PROBES)
code = f"import importlib\nmodules = {modules_literal}\nfor name in modules:\n importlib.import_module(name)\n"
resources_literal = repr(PACKAGE_RESOURCES)
code = (
"import importlib\nfrom importlib import resources\n"
f"modules = {modules_literal}\nresources_to_read = {resources_literal}\n"
"for name in modules:\n importlib.import_module(name)\n"
"for package, resource in resources_to_read:\n assert resources.files(package).joinpath(resource).read_text(encoding='utf-8')\n"
)
_run((str(python), "-I", "-c", code), cwd=install_dir, env=env)


Expand Down
71 changes: 63 additions & 8 deletions docs/archive-backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ The configured archive root contains these durable paths:
| `index.db` | Parsed sessions, messages, FTS/search indexes, graph rows, and derived read models. | Rebuildable from `source.db`; include in full evidence backups for faster restore, but cache-exclude profiles may omit it. |
| `embeddings.db` | Vector rows, embedding status, and catch-up metadata. | Back up when present. It is rebuildable, but expensive and may require provider cost. |
| `user.db` | Human/user/agent overlays stored as assertions, immutable annotation schema definitions and batch provenance, settings, and context-delivery receipts. | Always back up. This tier is irreplaceable user state. |
| `ops.db` | Daemon cursors, attempts, convergence debt, stage events, and operational telemetry. | Disposable. Include only in diagnostics bundles or incident snapshots. |
| `audit.db` | Append-only mutation authority, authorizations, attempts, receipts, and continuity heads. | Always back up. Relocation full-evidence backups require it. |
| `ops.db` | Daemon cursors, attempts, convergence debt, stage events, and operational telemetry. | Disposable for ordinary restore profiles, but required by the exact relocation full-evidence tier contract. |
| `blob/` | Content-addressed binary payloads keyed by SHA-256. | Back up referenced blobs with `source.db`/`user.db`; do not prune by age alone. |

`polylogue ops maintenance archive-plan --output-format json` is the machine-readable
Expand All @@ -28,7 +29,7 @@ Use these profiles when choosing what to copy:

| Profile | Include | Exclude | Use case |
| --- | --- | --- | --- |
| Full evidence | `source.db`, `index.db`, `embeddings.db`, `user.db`, referenced `blob/`, and optional `ops.db` snapshot. | Temporary SQLite `*-wal`/`*-shm` only after a clean checkpoint. | Fastest complete restore with raw evidence, read models, vectors, and overlays. |
| Full evidence | All six archive tiers: `source.db`, `index.db`, `embeddings.db`, `user.db`, `ops.db`, and `audit.db`, plus referenced `blob/`. | Temporary SQLite `*-wal`/`*-shm` only after a clean checkpoint. | Complete relocation authority and the fastest restore with raw evidence, read models, vectors, overlays, audit authority, and operational state. |
| User overlays | `user.db` and any assertion/note evidence blobs referenced by user-owned rows. | `index.db`, `ops.db`, rebuildable search/derived models. | Protect irreplaceable human/agent state before resets or schema rebuilds. |
| Rebuildable-cache exclude | `source.db`, `user.db`, referenced `blob/`, optionally `embeddings.db`. | `index.db`, `ops.db`, derived/cache artifacts. | Small backup that can rebuild parsed/indexed data locally. |
| Diagnostics bundle | `ops.db`, `archive-plan` JSON, `daemon-workload-probe` JSON, logs, and readonly status outputs. | Private raw blobs unless explicitly needed for the incident. | Bug reports and incident triage without over-sharing archive contents. |
Expand All @@ -37,6 +38,59 @@ When SQLite WAL files are present, either stop the daemon or run an explicit
checkpoint before copying. Copying only `*.db` while an uncheckpointed `*-wal`
contains recent writes creates an incomplete backup.

## Offline archive-root relocation

An inode-preserving filesystem move is the only supported way to change a configured archive root without restoring or rebuilding it. Stop the daemon and move the complete root without copying its database files. Set `POLYLOGUE_ARCHIVE_ROOT` to the moved root before creating relocation backup evidence.

If the current released source train lacks continuity authority for historical source changes, first run the `source-continuity-recovery` plan and apply sequence documented in [Maintenance Operations](maintenance.md#recovering-the-one-historical-liveness-receipt-shape). Its authenticated pre- and post-backup evidence belongs to the retired path and is used only for that bridge. After the bridge commits, or immediately after the move when no bridge is required, create and verify a fresh complete backup at the moved root:

```bash
POLYLOGUE_ARCHIVE_ROOT=/new/archive/root \
polylogue ops backup \
--output-dir /safe/operator/location/relocation-backup \
--profile full_evidence --verify
```

For relocation, the profile name alone is insufficient. The moved root must already contain every `ArchiveTier`, and the new backup manifest must contain this exact set with no omitted tiers:

```json
{
"profile": "full_evidence",
"included_tiers": [
"source.db",
"index.db",
"embeddings.db",
"user.db",
"ops.db",
"audit.db"
],
"omitted_tiers": []
}
```

The relocation validator compares `included_tiers` as a set, so JSON list order is not significant. It rejects a missing `audit.db`, a missing `ops.db`, any extra tier, or any non-empty `omitted_tiers` value. Use the `manifest.json` printed by the backup command to create the bound relocation plan. `--old-root` names the retired pre-move root only for the identity transition and active-index pointer mapping:

```bash
POLYLOGUE_ARCHIVE_ROOT=/new/archive/root \
polylogue ops maintenance archive-root-relocation plan \
--old-root /old/archive/root \
--backup-manifest /safe/operator/location/relocation-backup/PACKAGE/manifest.json \
--output /safe/operator/location/relocation-plan.json --output-format json
```

Apply only the exact self-hash printed in that plan:

```bash
POLYLOGUE_ARCHIVE_ROOT=/new/archive/root \
polylogue ops maintenance archive-root-relocation apply \
--plan /safe/operator/location/relocation-plan.json \
--authorize PLAN_SHA256 --output-format json
```

The route reads every SQLite file immutably and refuses copied files, WAL sidecars, moved-root backup receipts that do not authenticate the current tier paths, changed bytes/schema/version/tier inventory, fresh-bootstrap authority, or any incomplete released durable-train chain. A live source train whose historical content differs from the current source must first carry receipt-backed source-continuity authority. For the one pre-#3868 liveness receipt shape, create that authority with `source-continuity-recovery` using authenticated pre/post backups and a fresh zero-orphan census. That bridge is a separate offline transition, not an exception inside relocation. Relocation records both configured and resolved paths. A configured `index.db` active-generation symlink is permitted only through the existing `ArchiveLocation` resolver; the plan binds its resolved generation, every retained generation's absolute metadata and tier links, and apply remaps those exact objects before publishing the active pointer. Apply writes no SQLite rows, blobs, or sidecars. It CAS-revises released `source`, `user`, and `audit` train manifests when identity or continuity proof requires it, retains the exact plan, and records a prepared then committed receipt under `.maintenance-state/archive-root-relocations/`. Repeated relocations and intervening source refreshes must form one unbranched chain through typed predecessor authority and exact before/after manifest hashes. A prepared receipt blocks daemon startup and prints a shell-quoted exact retained-plan resume command. Live application and post-move observation remain operator evidence outside this code path.

For a deployed archive, run these commands only from the Nix package built from the post-merge commit selected for deployment. Record that merge SHA and the resulting Nix store path in the operator receipt, verify the daemon executable resolves to that exact package, and keep `POLYLOGUE_ARCHIVE_ROOT` set to the configured deployed root. Do not resume a stopped daemon with an older deployed package or a branch checkout: its durable-train vocabulary may predate the relocation transition.

## Restore Rules

Restore into an isolated archive root first:
Expand Down Expand Up @@ -141,8 +195,9 @@ sqlite3 <restored>/source.db "PRAGMA user_version; SELECT count(*) FROM raw_sess

# Sane-lag comparison against the live archive (restored counts must be <=
# live counts, and the gap should track the age of the chosen archive):
sqlite3 /realm/db/polylogue/user.db "SELECT count(*) FROM assertions;"
sqlite3 /realm/db/polylogue/source.db "SELECT count(*) FROM raw_sessions;"
archive_root="${POLYLOGUE_ARCHIVE_ROOT:?set the configured archive root}"
sqlite3 "$archive_root/user.db" "SELECT count(*) FROM assertions;"
sqlite3 "$archive_root/source.db" "SELECT count(*) FROM raw_sessions;"
```

**Negative control (deliberately corrupted restore must fail loudly)** —
Expand All @@ -169,10 +224,10 @@ schema `user_version=4`, `source.db` carried 17,839 `raw_sessions` rows at
snapshot's 17-day age. The corruption negative control correctly failed with
`database disk image is malformed (11)`.

**CRITICAL FINDING — the live durable tier currently has NO Borg coverage.**
`/realm/db/polylogue` (where `source.db`/`user.db` actually live; `/realm/data/captures/polylogue/*.db`
are symlinks to it) was converted to its own nested Btrfs subvolume on
2026-07-06 (`btrfs subvolume list /realm` shows `ID 3862 ... path db/polylogue`).
**CRITICAL FINDING — the durable tier then under review had NO Borg coverage.**
The configured archive root (resolved from `POLYLOGUE_ARCHIVE_ROOT`) was a nested
Btrfs subvolume at the time of the drill. Its location is configuration, not a
fixed runtime path; inspect the resolved root before repeating this evidence.
btrbk/Borg snapshot the **parent** `/realm` subvolume only; a nested
subvolume shows up as an **empty directory** in every snapshot and archive —
confirmed directly: `borg list <latest realm archive> db/polylogue` returns
Expand Down
4 changes: 2 additions & 2 deletions docs/devtools.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,8 +233,8 @@ These are the commands worth remembering during normal repo work:
## Cursor-authority reconciliation

`polylogue ops maintenance cursor-authority-reconcile` is a dry-run-by-default
repair route for exactly one proven cursor-ahead source. It reads the fixed
`/realm/db/polylogue` archive root, requires the daemon to be stopped, and
repair route for exactly one proven cursor-ahead source. It reads the
configured `POLYLOGUE_ARCHIVE_ROOT` (using its resolved archive root), requires the daemon to be stopped, and
writes a plan containing path and raw identifiers only as digests. Apply
requires that immutable plan, a freshly verified `full_evidence` backup
manifest with blob rollback evidence, and a new receipt path. The apply route
Expand Down
24 changes: 24 additions & 0 deletions docs/maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,30 @@ Restart health and runtime-consumer convergence are the final lifecycle proof
and are recorded by the durable train lifecycle API, not inferred from this
command's migration result alone.

## Relocating an archive root

Use `ops maintenance archive-root-relocation` only after an offline inode-preserving root move. `--old-root` names the retired pre-move root for the identity transition and active-index pointer mapping. Create a fresh verified `full_evidence` backup after setting `POLYLOGUE_ARCHIVE_ROOT` to the moved root. The relocation plan authenticates that backup against the moved root and revalidates its device/inode inventory there; it never asks a moved-root backup to authenticate the nonexistent retired path. A current source train with post-release source content must first have receipt-backed source-continuity authority; relocation verifies and rebinds that authority but never creates it. Planning is read-only. Applying revalidates all evidence, CAS-revises only the released `source`, `user`, and `audit` durable-train manifests that require relocation authority, and remaps every retained index generation's sealed `archive_root`, `index_path`, and absolute tier symlinks before publishing the active pointer. Mixed exact before/after generation states are accepted only for this plan's crash resume. The exact plan and prepared/committed receipts remain retained. Apply never opens SQLite read-write, changes a row, rebuilds, reindexes, or repairs startup state.

```bash
POLYLOGUE_ARCHIVE_ROOT=/new/archive/root polylogue ops maintenance archive-root-relocation plan --old-root /old/archive/root --backup-manifest /path/to/manifest.json --output /safe/relocation-plan.json --output-format json
POLYLOGUE_ARCHIVE_ROOT=/new/archive/root polylogue ops maintenance archive-root-relocation apply --plan /safe/relocation-plan.json --authorize PLAN_SHA256 --output-format json
```

If apply stops after recording a prepared receipt, daemon startup fails closed and names an exact command whose plan path is the retained plan bound by that receipt. Rerun that command with the same authorization after restoring offline ownership. Resume accepts only the exact post-CAS manifest hashes and relocation proof chain sealed for that plan. Do not use this operation for a copy, restore, new archive, migration, or live service move.

### Recovering the one historical liveness receipt shape

`source-continuity-recovery` is a one-purpose bridge for a committed pre-#3868 blob-reference-liveness receipt that lacks the modern manifest digest and post-orphan fields. It does not make ordinary liveness receipts permissive. It requires an authenticated pre-mutation backup at the old source path, an authenticated post-mutation backup at that same old path, exact pre/post `blob_refs` delta proof, and a fresh zero-orphan census against the configured moved root. It creates no SQLite rows; apply CAS-revises only the current released source train and retained receipts.

```bash
POLYLOGUE_ARCHIVE_ROOT=/new/archive/root polylogue ops maintenance source-continuity-recovery plan --old-root /old/archive/root --mutation-receipt /safe/liveness.jsonl --pre-backup-manifest /safe/pre/manifest.json --post-backup-manifest /safe/post/manifest.json --output /safe/continuity-plan.json --output-format json
POLYLOGUE_ARCHIVE_ROOT=/new/archive/root polylogue ops maintenance source-continuity-recovery apply --plan /safe/continuity-plan.json --authorize PLAN_SHA256 --output-format json
```

After this bridge commits, create and verify a fresh `full_evidence` backup at the moved root. Use that moved-root manifest with the separate archive-root-relocation plan/apply transition while `--old-root` continues to name the retired pre-move root. Before publishing a prepared bridge receipt, apply retains the exact sealed plan under `.maintenance-state/historical-source-continuity-recovery-plans/`; the blocking daemon error names that retained path in its executable resume command.

Later authenticated source maintenance writes a typed refresh receipt that binds its predecessor authority and the exact durable-train manifest hashes before and after the refresh. Repeated relocations and intervening refreshes therefore validate as one unbranched transition chain ending at the exact current manifest; matching only the current source hash or archive identity is not authority.

### Rebuild deployment-currency preflight

Before a managed `rebuild-index`, confirm that the package selected for the
Expand Down
Loading