test(lineage): bind topology proof to live seams - #3850
Conversation
Problem: topology status and method claims lacked a reusable candidate/live census, and unresolved-parent composition had no report-level proof through the archive read seam. What changed: extend lineage-validation with effective topology states, method and cycle-evidence counts, bounded unresolved-parent read checks, candidate index selection, and a receipt digest. Exercise the helper through the production cycle-quarantine writer and mutation fixtures. Compatibility/migration: preserve nullable status for ordinary resolved and unresolved rows; no schema or archive writes are introduced. Co-Authored-By: Claude <noreply@anthropic.com>
Problem: the topology implementation checkpoint must distinguish candidate proof from unavailable live evidence. What changed: record the candidate census results, production-route cycle evidence, read-safety proof, and the exact unexercised live step in the established evidence-report format. Compatibility/migration: this is a read-only evidence record and makes no archive or Beads changes. Co-Authored-By: Claude <noreply@anthropic.com>
Problem: the evidence report was rejected by the generated docs-surface gate because it lacked a registry entry. What changed: register the topology live-proof residue under the evidence tier and regenerate docs/README.md. Compatibility/migration: documentation-only change; no runtime or archive state changes. Co-Authored-By: Claude <noreply@anthropic.com>
Problem: The topology census could pass with hand-built rows, a self-hash without source identity, or an empty unresolved sample.\n\nWhat changed: Build a writer-backed candidate fixture, bind each report to database and SQLite sidecar identities across a held read transaction, and make unresolved-reader sampling report not_observed instead of passing vacuously. Add explicit candidate override and mutation tests.\n\nCompatibility/migration: The nullable raw status contract remains unchanged; effective resolved, unresolved, repaired, and quarantined states are still reported separately. Co-Authored-By: Claude <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 19 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
📝 WalkthroughWalkthroughThe PR adds topology-link census validation to lineage reports. It checks link states, methods, cycle evidence, unresolved-parent reads, snapshot stability, and receipt hashes. It adds tests and documents the live-proof evidence. ChangesTopology lineage validation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Validator
participant IndexDatabase
participant ArchiveReader
participant Report
Validator->>IndexDatabase: Capture initial snapshot
Validator->>IndexDatabase: Census session_links
Validator->>ArchiveReader: Sample unresolved-parent sessions
ArchiveReader-->>Validator: Read safety results
Validator->>IndexDatabase: Capture final snapshot
Validator->>Report: Write topology evidence and receipt hash
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@devtools/lineage_validation.py`:
- Around line 309-330: Update the missing-columns return branch of
census_topology_links to include raw_status_empty_count,
unknown_effective_statuses, and unresolved_count with the same default values
and types used by the successful branch. Preserve the existing unchecked
response while ensuring both branches emit an identical key set for
lineage.topology and summary.json consumers.
- Around line 43-58: Update _snapshot_identity to compute each file’s SHA-256
digest incrementally in fixed-size chunks instead of using path.read_bytes().
Preserve the existing file metadata and digest output, including behavior for
missing paths, while avoiding loading the entire file into memory.
- Around line 657-661: Move the _snapshot_identity(index_db) call that
initializes snapshot_before to after open_readonly_connection(index_db), before
BEGIN, so both snapshots are taken with the read connection open. Keep the
existing snapshot_after comparison and connection lifecycle unchanged.
In `@docs/evidence/polylogue-topology-live-proof-2026-08-06.md`:
- Line 16: Update
test_lineage_validation_proves_writer_candidate_and_snapshot_identity to assert
topology["raw_status_empty_count"] == 3, directly validating the documented raw
NULL status count for the writer-backed candidate.
In `@tests/unit/devtools/test_lineage_validation.py`:
- Around line 348-355: Update tests/unit/devtools/test_lineage_validation.py
lines 348-355 to use frozen_clock for both build_report calls, add an
unchanged-database rerun asserting the receipt is reproducible, then assert the
session_links mutation changes it; update
docs/evidence/polylogue-topology-live-proof-2026-08-06.md line 9 to document
reproducibility at a fixed capture time and divergence after database changes,
or attribute content binding to snapshot_identity.sha256 until that behavior is
verified.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 93ef2cf2-6e9a-422d-9f50-580cdaf14d36
📒 Files selected for processing (6)
devtools/docs_surface.pydevtools/lineage_validation.pydocs/README.mddocs/evidence/polylogue-topology-live-proof-2026-08-06.mdtests/unit/devtools/test_lineage_validation.pytests/unit/storage/test_topology_cycle_quarantine_live.py
| index_db = (args.index_db or config.db_path).expanduser().resolve() | ||
| snapshot_before = _snapshot_identity(index_db) | ||
| conn = open_readonly_connection(index_db) | ||
| try: | ||
| conn.execute("BEGIN") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Determine the journal mode applied to the index tier and the read connection.
set -euo pipefail
echo "=== READ_CONNECTION_PRAGMA_STATEMENTS ==="
rg -n -A 20 'READ_CONNECTION_PRAGMA_STATEMENTS\s*[:=]' --type=py
echo "=== journal_mode / WAL settings in storage ==="
rg -n -i 'journal_mode' --type=py
echo "=== index tier DDL / spec definition ==="
fd -t f 'archive_tier_spec|bootstrap.py' -x rg -n -C 5 'journal_mode|wal' {}Repository: Sinity/polylogue
Length of output: 196
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== locate lineage_validation.py ==="
fd -t f 'lineage_validation\.py' .
echo "=== relevant function outline/context ==="
python3 - <<'PY'
from pathlib import Path
p=next(Path('.').rglob('lineage_validation.py'))
text=p.read_text()
for tok in ['def build_report', 'def _snapshot_identity', 'def open_readonly_connection']:
idx=text.find(tok)
if idx!=-1:
print(f'--- {tok} at {idx} ---')
start=max(0,text.rfind('\n',0,idx-100))
end=min(len(text), text.find('\ndef ', idx+10000))
if end<=idx: end=len(text)
for n,line in enumerate(text[start:end].splitlines(), text[:start].count('\n')+1):
if n>=max(1, (text[:idx].count('\n')+1)-40) and n<=text[:end].count('\n')+80:
print(f'{n}: {line}')
PY
echo "=== search sqlite/pragmas/schema specs ==="
rg -n -i 'sqlite|journal_mode|pragma|wal|readonly|index_db|_snapshot_identity' --type=py devtools . 2>/dev/null | head -200Repository: Sinity/polylogue
Length of output: 43989
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== connection_profile.py ==="
p=../polylogue/storage/sqlite/connection_profile.py
if [ -f "$p" ]; then
cat -n "$p" | sed -n '1,180p'
else
fd -t f 'connection_profile.py' . -x sh -c 'echo "--- $1 ---"; cat -n "$1" | sed -n "1,180p"' sh {}
fi
echo "=== archive tier files with journal settings ==="
rg -n -i 'journal_mode|PRAGMA|CREATE TABLE|CREATE INDEX|archive_tiers|index\.db|OPEN_URI|mode=ro|readonly' polylogue storage 2>/dev/null | head -300Repository: Sinity/polylogue
Length of output: 48693
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== connection_profile.py remaining ==="
cat -n polylogue/storage/sqlite/connection_profile.py | sed -n '180,280p'
echo "=== bootstrap/index tier spec relevant sections ==="
rg -n -C 8 'journal_mode|PRAGMA|ARCHIVE_TIER_SPECS|index\.db|filename|db_path|wal|WAL' polylogue/storage/sqlite/archive_tiers polylogue/storage/sqlite/connection.py polylogue/readiness/__init__.py --type=py
echo "=== SQLite open_readonly_connection behavior probe ==="
python3 - <<'PY'
import sqlite3
from pathlib import Path
tmp_dir = Path.home() / '.polylogue_lineage_probe'
tmp_dir.mkdir(exist_ok=True)
path = tmp_dir / 'index.db'
path.unlink(missing_ok=True)
for mode in ['no-wal', 'set-wal']:
if mode == 'set-wal':
conn = sqlite3.connect(str(path))
conn.execute('PRAGMA journal_mode=WAL')
conn.execute('SELECT 1').fetchone()
conn.commit()
conn.close()
before = {name: path.stat().st_mtime_ns if path.joinpath(name).exists() else None for name in ('-shm', '-wal')}
uri = f'file:{path}?mode=ro'
conn = sqlite3.connect(uri, uri=True)
after_readonly = {name: path.joinpath(name).exists() for name in ('-shm', '-wal')}
conn.execute('PRAGMA user_version')
after_pragma = {name: path.joinpath(name).exists() for name in ('-shm', '-wal')}
conn.execute('BEGIN')
after_begin = {name: path.joinpath(name).exists() for name in ('-shm', '-wal')}
conn.rollback()
conn.close()
after_close = {name: path.joinpath(name).exists() for name in ('-shm', '-wal')}
print(mode, "before", before, "after_readonly", after_readonly, "after_pragma", after_pragma, "after_begin", after_begin, "after_close", after_close)
path.unlink(missing_ok=True)
tmp_dir.rmdir()
PYRepository: Sinity/polylogue
Length of output: 50373
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
# Read-only SQLite sidecar probe without installing extra dependencies or running repository code.
import sqlite3
from pathlib import Path
import tempfile
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "index.db"
for label, setup in [
("no-wal", lambda: None),
("set-wal", lambda: sqlite3.connect(str(path)).execute("PRAGMA journal_mode=WAL").fetchall()),
]:
setup()
def exists(*names):
return {name: path.joinpath(name).exists() for name in names}
before = exists("index.db", "index.db-wal", "index.db-shm", "index.db-journal")
uri = f"file:{path}?mode=ro"
conn = sqlite3.connect(uri, uri=True)
after_readonly = exists("index.db-wal", "index.db-shm")
conn.execute("PRAGMA user_version")
after_pragma = exists("index.db-wal", "index.db-shm")
conn.execute("BEGIN")
after_begin = exists("index.db-wal", "index.db-shm")
conn.rollback()
conn.execute("PRAGMA journal_mode")
after_pragma2 = exists("index.db-wal", "index.db-shm")
conn.close()
after_close = exists("index.db-wal", "index.db-shm")
print(label, before, after_readonly, after_pragma, after_begin, after_pragma2, after_close)
PY
echo "=== focused connection_profile implementation ==="
sed -n '1,260p' polylogue/storage/sqlite/connection_profile.py | cat -n
echo "=== focused init/write journal settings ==="
sgrep -n 'PRAGMA journal_mode|initialize_archive_database|ARCHIVE_DDL_BY_TIER|WAL_JOURNAL' \
polylogue/storage/sqlite/bootstrap.py polylogue/storage/sqlite/connection_profile.py polylogue/storage/sqlite/write.py \
--max-count 50 || trueRepository: Sinity/polylogue
Length of output: 283
🌐 Web query:
SQLite documentation DELETE-JOURNAL mode create temporary rollback journal after opening database read-only mode delete-journal mode
💡 Result:
In SQLite, the behavior of rollback journal files when a database is opened in read-only mode depends on whether the database requires recovery from a previous, incomplete transaction [1][2]. If the database is in a consistent state and no recovery is required, SQLite does not create a rollback journal file when opening a connection in read-only mode, regardless of the journal_mode setting [3][4]. However, if the database has a "hot journal" (indicating a crash or power loss occurred during a previous write transaction), SQLite must perform recovery to ensure consistency [1][2]. Recovery requires writing to the database file to roll back the interrupted transaction, which necessitates write permission on both the database file and its containing directory [1][2]. Consequently, you cannot open a database that requires recovery in read-only mode [1]. When operating in the default DELETE journal mode, SQLite does not create a rollback journal file simply by opening the connection [3]. A rollback journal is only created if and when a write transaction is initiated [3][5]. Because a read-only connection is prohibited from initiating write transactions, it will not create a rollback journal [3][6]. It is important to distinguish this from WAL (Write-Ahead Logging) mode. Unlike DELETE mode, WAL mode requires the presence of a -shm (shared memory) file and sometimes a -wal file even for read-only connections, as these files are used to manage concurrency and locking [2][7][6]. Therefore, while a read-only connection in DELETE mode typically requires no auxiliary files, a read-only connection in WAL mode may still require write access to the directory to create or manage these support files unless the database is marked as immutable or the files already exist [8][7]. For databases on truly read-only media, it is recommended to ensure the database is in a clean, consistent state and, if necessary, set the journal_mode to DELETE before finalizing the read-only image [8][9].
Citations:
- 1: https://stackoverflow.com/questions/7957778/opening-a-sqlite3-db-on-a-read-only-filesystem-with-a-journal-file
- 2: https://sqlite.org/forum/forumpost/030179b7ca7b4177
- 3: https://sqlite.org/pragma.html
- 4: https://stackoverflow.com/questions/26092994/how-to-turn-off-the-sqlite-journal-for-access-from-a-read-only-filesystem
- 5: https://sqlite.org/isolation.html
- 6: https://sqlite.org/forum/forumpost/8ff20f4255306ca6?t=c
- 7: https://www.sqlite.org/wal.html
- 8: https://sqlite.org/wal.html
- 9: https://sqlite.org/forum/info/8debe446d2b5dbe3
🌐 Web query:
Python sqlite3 open_readonly_connection uri mode=ro journal_mode=NONE PRAGMA journal_mode=NONE
💡 Result:
To open a SQLite database in read-only mode using Python's sqlite3 module, you must use the URI connection string format with the uri=True parameter [1][2]. Example: con = sqlite3.connect("file:path/to/database.db?mode=ro", uri=True) In this configuration: - The mode=ro query parameter explicitly instructs SQLite to open the database in read-only mode [3][4]. - Setting uri=True is required for the library to interpret the database string as a URI [1]. Regarding PRAGMA journal_mode=NONE: - The correct command is PRAGMA journal_mode=OFF [5]. There is no journal_mode=NONE [5]. - Setting PRAGMA journal_mode=OFF disables the rollback journal entirely [5][6]. This is a dangerous setting that disables atomic commit and rollback capabilities; if the application crashes during a transaction, the database file will likely become corrupted [5][6]. - PRAGMA settings are connection-specific and must be executed after establishing the connection [5][7]. They cannot be passed as part of the URI connection string [3][4]. Note: When you open a database in read-only mode (mode=ro), you cannot modify the database, and attempting to change the journal_mode via a PRAGMA will generally fail because the database is read-only [1][5]. [1][3][5]
Citations:
- 1: https://docs.python.org/3/library/sqlite3.html
- 2: https://adhdecode.com/articles/sqlite/sqlite-readonly-mode-connection/
- 3: https://www.sqlite.org/uri.html
- 4: https://sqlite.org/c3ref/open.html
- 5: https://sqlite.org/pragma.html
- 6: https://www.sqlite.org/tempfiles.html
- 7: https://stackoverflow.com/questions/16998452/how-to-keep-journal-mode-for-every-transaction
🌐 Web query:
Python sqlite3 sqlite3.Connection open_readonly_connection?
💡 Result:
To open an SQLite database in read-only mode using Python's sqlite3 module, you must use the Uniform Resource Identifier (URI) syntax within the sqlite3.connect function [1][2]. There is no direct "open_readonly_connection" attribute or method [3]; instead, you configure the connection via the URI query parameters [1]. To establish a read-only connection, pass the database path as a URI string with the mode=ro parameter and set the uri argument to True [2][4]: import sqlite3 # Connect to the database in read-only mode conn = sqlite3.connect("file:your_database.db?mode=ro", uri=True) Attempting to execute any write operations (such as INSERT, UPDATE, or CREATE) on a connection opened this way will raise an sqlite3.OperationalError [2][5]. Additionally, for existing connections, you can execute the command PRAGMA query_only = ON; to prevent modifications on that specific connection [6]. This PRAGMA is useful if you cannot re-establish the connection using the URI format [6]. If you require even stricter enforcement, such as working with immutable database files on read-only media, you can use the immutable=1 query parameter in the URI (e.g., file:your_database.db?mode=ro&immutable=1) [1].
Citations:
- 1: https://sqlite.org/uri.html
- 2: https://docs.python.org/3/library/sqlite3.html
- 3: https://stackoverflow.com/questions/77641477/how-can-i-open-a-database-in-read-only-mode-by-sqlite3-in-python
- 4: https://adhdecode.com/articles/sqlite/sqlite-readonly-mode-connection/
- 5: https://www.slingacademy.com/article/python-311-sqlite3-how-to-open-a-database-in-read-only-mode/
- 6: https://stackoverflow.com/questions/10205744/opening-sqlite3-database-from-python-in-read-only-mode
Make the snapshot identity stable against open_readonly_connection.
snapshot_before is recorded before open_readonly_connection(index_db), then snapshot_after is recorded while the read connection is still open. Live index.db is WAL-mode, so the read connection is opened before the BEGIN; WAL reader sidecar state can change between these snapshots and produce index snapshot changed during the read-only census. Capture the before snapshot after opening the read connection, or exclude reader-only SQLite sidecar state from the comparison.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@devtools/lineage_validation.py` around lines 657 - 661, Move the
_snapshot_identity(index_db) call that initializes snapshot_before to after
open_readonly_connection(index_db), before BEGIN, so both snapshots are taken
with the read connection open. Keep the existing snapshot_after comparison and
connection lifecycle unchanged.
| | effective topology states | `resolved=2`, `unresolved=1` | | ||
| | empty effective states | `0` | | ||
| | empty methods | `0` | | ||
| | raw nullable status values | `3` ordinary NULLs, reported transparently | |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Pin the raw NULL status count with an assertion, or qualify the claim.
The table records 3 ordinary NULL raw status values for the candidate fixture. No test in this PR asserts raw_status_empty_count for the writer-backed candidate. test_lineage_validation_proves_writer_candidate_and_snapshot_identity asserts the effective status counts and the method counts, but not the raw status count. The effective-state derivation maps a NULL status and an explicit 'unresolved' status to the same value, so the existing assertions cannot distinguish the two cases for the orphan link.
Add assert topology["raw_status_empty_count"] == 3 to that test so the documented number is backed by the same evidence lane as the other rows.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/evidence/polylogue-topology-live-proof-2026-08-06.md` at line 16, Update
test_lineage_validation_proves_writer_candidate_and_snapshot_identity to assert
topology["raw_status_empty_count"] == 3, directly validating the documented raw
NULL status count for the writer-backed candidate.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bdd2eeeca3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| files.append({"path": str(path), "present": False}) | ||
| continue | ||
| stat = path.stat() | ||
| digest = hashlib.sha256(path.read_bytes()).hexdigest() |
There was a problem hiding this comment.
When this census is run against the intended live or candidate archive and index.db or its WAL is multi-gigabyte, path.read_bytes() allocates the entire file before hashing it. Because _snapshot_identity runs both before and after the census, the command can be killed by memory pressure and fail to produce the receipt it exists to generate. Hash each file incrementally in bounded chunks while collecting its identity metadata.
Useful? React with 👍 / 👎.
|
|
||
| def _snapshot_identity(index_db: Path) -> dict[str, Any]: | ||
| """Describe the database files that make up one read-only index snapshot.""" | ||
| paths = [index_db, Path(f"{index_db}-wal"), Path(f"{index_db}-shm"), Path(f"{index_db}-journal")] |
There was a problem hiding this comment.
On the intended WAL-mode live archive, opening the read-only connection and performing the first query can itself update reader marks in index.db-shm. Because the before hash is captured before the connection opens while the after hash is captured inside the read transaction, a completely quiescent database can be reported as changed solely due to this census reader, making external_counts_citable false for the primary live-archive use case. Establish the SQLite snapshot before taking the baseline, or exclude mutable wal-index bookkeeping from the content identity.
Useful? React with 👍 / 👎.
| SELECT COUNT(*) | ||
| FROM session_links | ||
| WHERE TRIM(status) = 'quarantined' | ||
| AND json_extract(evidence_json, '$.reason') = 'cycle_rejected' | ||
| """, |
There was a problem hiding this comment.
When a quarantined live row contains malformed evidence_json, json_extract raises and aborts the entire report even though the schema permits arbitrary TEXT; conversely, a payload containing only {"reason":"cycle_rejected"} is counted as valid cycle evidence despite lacking the writer's cycle path. Since this is an integrity census, guard with json_valid and require the expected evidence fields so corrupt rows produce a non-citable verdict rather than a crash or a false proof.
Useful? React with 👍 / 👎.
| SELECT CASE | ||
| WHEN NULLIF(TRIM(status), '') IS NOT NULL THEN TRIM(status) | ||
| WHEN resolved_dst_session_id IS NOT NULL THEN 'resolved' | ||
| ELSE 'unresolved' | ||
| END AS effective_status, |
There was a problem hiding this comment.
When a stale or corrupt row has status='quarantined' but retains a non-NULL resolved_dst_session_id, this CASE prioritizes the status and reports an apparently valid quarantined edge. If its evidence has the expected reason, no verdict check rejects it, while the production prefix-sharing reader can still compose the resolved parent because that read query keys on resolved_dst_session_id and does not exclude quarantined status. Count this contradictory state as an integrity failure, including any corresponding stale parent projection, so a supposed cycle break cannot remain traversable.
AGENTS.md reference: AGENTS.md:L111-L112
Useful? React with 👍 / 👎.
Problem: topology receipts hashed whole files, emitted variant census shapes, and accepted malformed or contradictory quarantine evidence. Reader composition also treated a quarantined resolved edge as traversable, while the receipt mutation test changed its capture time on every run. What changed: stream snapshot hashing from a transaction-bound read, stabilize the census schema, validate cycle evidence structurally, report contradictory quarantine rows, and exclude quarantined edges from lineage readers and repair traversal. Add fixed-clock receipt reproducibility and production-route mutation tests. Compatibility/migration: no schema or archive data changes. Live census evidence remains explicitly unobserved.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f870283b86
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| AND inheritance = 'prefix-sharing' | ||
| AND resolved_dst_session_id IS NOT NULL | ||
| AND branch_point_message_id IS NOT NULL | ||
| AND COALESCE(TRIM(status), '') != 'quarantined' |
There was a problem hiding this comment.
When re-ingesting session A with parent B would close a cycle and A's payload begins with messages copied from B, _extract_prefix_tail runs before _resolve_outbound_session_links detects and quarantines the cycle, so A is already stored as a divergent tail. This new predicate then refuses to compose B, causing both sync and async reads of A to silently omit the sliced prefix. Unlike the earlier contradictory-row case, the production writer can create this state with resolved_dst_session_id still NULL; detect the cycle before slicing or restore the removed prefix when quarantining.
AGENTS.md reference: AGENTS.md:L93-L97
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| conn.execute("BEGIN") | ||
| snapshot_before = _snapshot_identity(index_db) |
There was a problem hiding this comment.
On a live WAL database where the first read updates a wal-index reader mark, BEGIN is deferred and does not establish the SQLite snapshot; the first database read remains _user_version, after snapshot_before has hashed the SHM file. The census can therefore still report a quiescent archive as changed solely because its own first query modified index.db-shm. Fresh evidence beyond the earlier comment is that the added mitigation only executes deferred BEGIN; force a harmless read before taking the baseline or exclude mutable reader-mark bookkeeping.
AGENTS.md reference: AGENTS.md:L163-L166
Useful? React with 👍 / 👎.
## Summary This follow-up combines the schema-currency and topology-proof lineages from PRs #3856 and #3850, then repairs the remaining RW acceptance gaps. Rebuild admission now checks every durable tier at each ownership-sensitive boundary, missing durable tiers have an atomic recovery path, topology quarantine preserves child content, and proof receipts bind a stable SQLite snapshot. Ref #3850 and #3856. ## Problem The predecessor branches had concrete execution blockers. Their fixtures omitted the audit tier, the recovery runbook omitted audit initialization, daemon bulk could consume a source page after its last durable-currency check, and local empty-source output assumed replay counters that are absent by design. Cycle quarantine could also run after prefix extraction, so a rejected parent could silently remove copied child content. Finally, the topology census hashed files before proving which SQLite snapshot it read and accepted several ambiguous or fabricated topology evidence shapes. ## Solution The schema lane now enumerates every canonical durable migration tier, rechecks currency after archive ownership and again before daemon page selection, preserves the empty-source CLI receipt, and documents source, user, and audit recovery. `migrate-tier --initialize-missing` constructs the canonical fresh database in memory, copies it directly into an anonymous inode, fsyncs it, and publishes it with atomic no-replace semantics. Existing or concurrently created targets remain untouched. The topology lane classifies cycles before prefix-tail extraction and preserves the full child transcript for quarantined or over-budget parent walks. Stored cycle evidence must close through the asserted edge and match the current projection. Unresolved and quarantined readers remain child-local. The census groups unresolved samples by complete edge identity, distinguishes walk exhaustion from cycle proof, and rejects stale or contradictory projections without penalizing a valid alternate resolved edge. The receipt route begins a read transaction and establishes its snapshot before the baseline file hash. A second observer checks `data_version` for concurrent WAL commits, while before and after chunked hashes bind `index.db` and its sidecars without whole-file allocation. Five independent adversarial passes reviewed the complete branch. Four produced repairs. The final pass raised only the intentionally empty replay mapping, which the exact PR #3856 review permits the implemented status branch to handle and which the production CLI regression covers. ## Acceptance criteria | ID | Criterion | Result | Evidence or residue | | --- | --- | --- | --- | | S1 | Gate every canonical durable tier before receipt consumption or candidate creation. | Satisfied | Early local guard, post-ownership local guard, daemon transaction guard, daemon page-selection guard, HTTP guard, and all-tier fixtures. | | S2 | Recheck currency after archive ownership. | Satisfied | Local and daemon ownership-race tests inject a mismatch after ownership and prove refusal before mutation. | | S3 | Preserve empty-source CLI behavior and include audit recovery. | Satisfied | Exact plain-output regression and maintenance commands for source, user, and audit. | | S4 | Initialize only a missing durable tier without exposing replaceable staged bytes. | Satisfied | Canonical in-memory image, anonymous inode, fsync, atomic no-replace link, identity check, directory fsync, and concurrent-creator tests. | | S5 | Avoid production database mutation in this lane. | Satisfied | All runtime tests use temporary archives. No live command was run. | | T1 | Preserve copied child content when a proposed parent is quarantined. | Satisfied | Production writer and sync plus async reader regression; removing the pre-slice classification loses the asserted transcript. | | T2 | Distinguish proven cycles from bounded-walk exhaustion. | Satisfied | Typed writer outcome, projection-backed evidence validation, and a 1,024-hop acyclic fixture. | | T3 | Keep unresolved and quarantined composition child-local. | Satisfied for temporary archives | Production readers plus same-child multi-edge census coverage. Live sampling remains unobserved. | | T4 | Bind topology evidence to the exact stable read snapshot. | Satisfied for temporary WAL fixtures | Snapshot established before hashing, observer `data_version` check, chunked database and sidecar hashes, quiescent and concurrent WAL tests. | | T5 | Record exact live topology evidence before claiming operational completion. | Partial | `polylogue-live-operation-receipts` remains open for the approved live census. | ## Verification - `devtools test tests/unit/cli/test_archive_maintenance_cli.py -k 'migrate_tier_cli or rebuild_index_preflight or rebuild_index_empty_source_still or rebuild_index_rejects_daemon_schema_preflight'`: 12 passed, 55 deselected. - `devtools test tests/unit/cli/test_archive_maintenance_cli.py -k 'migrate_tier_cli_initializes_only_an_absent_durable_tier or migrate_tier_cli_missing_initialization_refuses_an_existing_tier or migrate_tier_cli_missing_initialization_loses_publish_race_without_replacement or migrate_tier_cli_exposes_no_named_staging_inode_before_publication'`: 4 passed, 66 deselected. - `devtools test tests/unit/daemon/test_bulk_rebuild_ownership.py -k 'rejects_schema_mismatch_before_transaction_bookkeeping or rechecks_schema_currency_after_ownership'`: 2 passed, 2 deselected. - `devtools test tests/unit/daemon/test_bulk_rebuild.py -k daemon_bulk_pass_rechecks_schema_currency_in_page_selection_hold`: 1 passed, 8 deselected. - `devtools test tests/unit/maintenance/test_rebuild_status.py -k falls_back_to_the_daemon_well_known_operation_id_by_default`: 1 passed, 5 deselected. - `devtools test tests/unit/storage/test_topology_cycle_quarantine_live.py tests/unit/devtools/test_lineage_validation.py`: 20 passed at `6e2996668`. - `devtools test tests/unit/devtools/test_lineage_validation.py`: 16 passed after the final multi-edge repair. - `devtools verify --quick`: 24 steps, exit 0 in 105.41 seconds at `d34e663cb` (`20260806T111901Z-quick-119573-0540dbf0`). - `devtools verify --seed-testmon --skip-slow`: static and policy steps passed, but the inherited full-suite seed ended with 258 failures, 19,631 passes, 3 skips, and 8 errors across more than 100 files. None of the acceptance files failed. The seed remains incomplete, so the default affected selector could not run. ## Live receipt still required No production database was opened or mutated. After merge and deployment, `polylogue-live-operation-receipts` must run the read-only topology census against the exact active or approved candidate index generation and retain its snapshot-bound receipt. `polylogue-a7gmk` must also verify the shipped package and migrate every durable tier to current before the managed rebuild. This PR does not assign `polylogue-9qnzy` because its AC explicitly requires no code changes under that bead. <!-- polylogue-pr-scope:v1 { "assigned_beads": [ "polylogue-topology-live-proof" ], "beads_digest": "ac5775b4d1215a3b38f0070c4e694bef389803aacd4a4d355f4d8173d3ce9b06", "dispositions": [ { "bead_id": "polylogue-topology-live-proof", "disposition": "partial", "evidence": [ { "kind": "commit", "ref": "d34e663cbfb1cd7eb628ab51f48f6574e5c90b1b" }, { "kind": "test", "ref": "tests/unit/devtools/test_lineage_validation.py and tests/unit/storage/test_topology_cycle_quarantine_live.py" }, { "kind": "command", "ref": "devtools verify --quick: 24 steps, exit 0, run 20260806T111901Z-quick-119573-0540dbf0" }, { "kind": "receipt", "ref": "docs/evidence/polylogue-topology-live-proof-2026-08-06.md: synthetic candidate only; live receipt explicitly unobserved" } ], "successors": [ "polylogue-live-operation-receipts" ] } ], "head_sha": "d34e663cbfb1cd7eb628ab51f48f6574e5c90b1b", "scope_digest": "80257ffd894ba2700bce1c520f1e5465e95cef06376bba24ee9039635fe9867b", "version": 1 } --> --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude <noreply@anthropic.com>
|
This branch is fully represented in merged PR #3858. The ancestry comparison shows all five #3850 commits reproduced there, plus the topology and schema-gate follow-up repairs. The live-proof obligation remains represented by polylogue-topology-live-proof and polylogue-live-operation-receipts. This stale duplicate PR is being removed from the active frontier. |
1 participant
Summary
Bind topology proof reports to writer-backed candidate fixtures, immutable database and sidecar identities, and explicit unresolved or cycle-quarantined states. The slice records candidate evidence while keeping the live census open.
Problem
Topology checks could hash whole files, emit different shapes on schema failure, accept malformed quarantine evidence, and let contradictory quarantined edges reach composition. The reindex proof needs a reproducible receipt that distinguishes candidate evidence from a live archive census.
Solution
Stream snapshot hashing from a transaction-bound read, make the census schema stable, validate cycle evidence structurally, report quarantined rows that still resolve parents, and exclude quarantined edges from synchronous and asynchronous lineage composition. Add fixed-clock receipt reproducibility, production-route anti-vacuity mutations, and documentation for the explicit not-observed live result.
Verification
direnv exec . devtools test tests/unit/devtools/test_lineage_validation.py tests/unit/storage/test_topology_cycle_quarantine_live.pypassed: 15 passed in 7.20s.direnv exec . devtools verify --quickpassed with Ruff format/check, strict mypy, generated render, layering, policy, and schema checks green. Commits:a31a95a1b,89f7d1cb5,b78e05bb4,bdd2eeeca34a6c8da083acc444784f1547f5b3ca,f870283b8.Follow-ups
The live archive census was not run because the current daemon and schema deployment are incompatible. The receipt therefore does not claim live convergence. Run the harness against the approved candidate or live generation and retain its receipt under
polylogue-live-operation-receipts.Ref polylogue-topology-live-proof
Summary by CodeRabbit
New Features
Documentation
Tests