From 21695bb05cde163958700a836296b15bab0dd343 Mon Sep 17 00:00:00 2001 From: Sinity Date: Tue, 28 Jul 2026 22:07:56 +0200 Subject: [PATCH] fix(storage): declare the v44 index delta and gate schema-versioning by default Problem An index schema bump merged without its lifecycle.py delta declaration, and nothing caught it. index_fast_forward_plan(43, 44) returned None, so the live archive fell back to full raw replay and every repo-CLI query against it failed with "no such column: s.title_ref" -- diagnosed by hand days later. The check that exists for exactly this (`devtools lab policy schema-versioning`) reported the violation correctly the whole time. It sits inside the `if lab:` branch of devtools/verify.py, which neither default verify, nor `verify --quick` (the pre-push hook), nor the CI lint job runs. What changed - Declare index v44 (sessions.title_ref/title_confidence) as SEMANTIC_REPARSE. That is the truthful classification under the current DerivedDeltaClass vocabulary: the DDL surface is two nullable columns, but the values come from the v44 Codex title resolver, so a shape-only fast-forward would leave all 3,201 codex-session rows NULL where a cold rebuild populates them. Behaviour is unchanged (full rebuild is what already happens); the declaration records why. polylogue-9rw0.1 owns removing the need for that classification. - Move `lab policy schema-versioning` from the --lab branch into the default verify step list and the CI lint job. It is static, archive-independent and measured at 0.63s. - Rewrite the stale comment above INDEX_SCHEMA_VERSION. It claimed index.db has no migration chain and that a stale generation needs `ops reset --index`, written before the declared fast-forward mechanism existed. It is the reason this cost was mistaken for unavoidable. Verification devtools verify --quick exit 0 (58.03s, 18 steps) lab policy schema-versioning exit 0, "Schema evolution policy intact" anti-vacuity: INDEX_SCHEMA_VERSION -> 45 exit 1, "missing: [45]"; reverted -> exit 0 Ref polylogue-9rw0, polylogue-ih67, polylogue-ze5i --- .beads/issues.jsonl | 36 ++++++++++++------- .github/workflows/ci.yml | 3 ++ devtools/verify.py | 8 ++++- .../storage/sqlite/archive_tiers/index.py | 17 ++++++--- polylogue/storage/sqlite/lifecycle.py | 18 ++++++++++ 5 files changed, 64 insertions(+), 18 deletions(-) diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index c6716ebc31..a3e7c0df30 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -6,11 +6,11 @@ {"_type":"issue","id":"polylogue-yyvg.6.1","title":"Fail closed on terminal browser-action capture","description":"A completed/cancelled external campaign can leave the unpacked Polylogue extension repeatedly issuing authenticated ChatGPT conversation and attachment fetches. Live evidence on 2026-07-17 showed fetchNativePayloadFromContentScript repeatedly retrieving one completed Sol-Pro handoff and its context attachment every few seconds, extending ChatGPT soft rate limiting after all work was complete. The local browser-action spool was empty; therefore terminal campaign state did not prevent the page-side capture loop.","design":"Treat terminal external work as a hard stop for provider-native retrieval. Trace the trigger from extension content script, receiver/capture status, and any cached launch/capture state. A completed, cancelled, paused-without-action, or absent action must not poll/re-fetch provider conversation or attachments. Make retry state explicit, bounded, and observable; failure of the receiver must not turn into provider traffic. Preserve ordinary passive capture for a user-opened active conversation, but require a durable nonterminal transport/capture reason for each authenticated provider-native fetch. Add a reset/unload-safe circuit breaker and a redacted local receipt of provider fetch decision/cadence. This belongs to generic browser-action/capture orchestration, not Sol campaign semantics.","acceptance_criteria":"1. A deterministic extension/receiver fixture proves no ChatGPT conversation or attachment request occurs after terminal or absent action state, including receiver failure/restart. 2. Every provider-native fetch has a typed, durable nonterminal reason and bounded retry/backoff; a terminal job cannot revive it. 3. A live local proof records zero ChatGPT requests across a multi-interval observation once campaign work is terminal, without closing the user tab. 4. The extension exposes a reversible, narrow circuit-breaker for this class of incident; daemon stop alone is not relied on. 5. Existing active user-conversation capture remains functional and has focused behavior evidence.","notes":"2026-07-17 shipped containment repair in merged PR #2977 / master e6032e406: extension startup, activation, and update now reconcile receiver archive state and only auto-capture the missing state; spooled-only and archived terminal state no longer trigger authenticated provider conversation or attachment fetches. Focused 76-test background suite, extension lint, and devtools verify --quick (16/16) passed. Full extension suite: 314 passed; one packaged-worker fixture failure reproduced unchanged on master. This is partial AC progress only: retained work is explicit narrow operator circuit-breaker plus an installed-extension live multi-interval proof after reload.\n2026-07-17 shipped merged PR #2979 / master 211ce80b8: persisted global Automatic capture and provider refresh breaker is now exposed in the popup. It gates automatic tab capture and freshness queue/sweep before provider traffic, clears the freshness wake while paused, and re-arms on resume; explicit popup sync remains available. Focused background+popup tests: 105 passed; extension lint passed; devtools verify --quick 16/16 passed. Full extension suite: 317 passed, with the same pre-existing packaged-worker fixture failure. Still open only for installed-extension live proof and any further evidence-led hardening.\n2026-07-17 live installed-extension proof after merged PR #2983 / master dd7e8decc: loaded current unpacked extension into live Chrome with no receiver pairing, then controlled one automatic freshness interval against an existing ChatGPT conversation without opening/closing that conversation tab. 28 freshness entries were held with last_error=receiver_unpaired; extension debug storage recorded 0 POST /v1/browser-captures requests and 0 provider-transport events. Returned extension to automatic_capture_enabled=false with an empty freshness queue. This satisfies the no-provider-traffic portion of AC1/AC3 for the unpaired/restart-shaped case. Remaining AC work is a longer terminal-state observation with a valid paired receiver plus a durable/redacted receipt surface.\n2026-07-17 live post-migration check: current loaded extension ecjmjollgmjhilmofklcabhgpfhpooio reports receiver pairing=null, automatic_capture_enabled=false, capture/freshness queues empty. After confirming that state, a 65-second daemon-journal observation recorded zero new browser_capture.token_rejected events. The source-v13 migration is complete and polylogued is running its full watcher. This is a safe paused state pending explicit valid re-pairing; it does not generate provider traffic.\nWarroom sweep It.17: claiming session closed. VERIFY-FOR-CLOSE candidate: #2981/#2983/#2986 (recapture without freshness signals; require pairing before provider capture; gate tab capture on receiver pairing) appear to cover the fail-closed scope. Needs an AC-by-AC check against the diff before closing -- do not re-claim for new work without that check.\n2026-07-18 Lane H AC-by-AC verdict (per warroom It.17 VERIFY-FOR-CLOSE instruction), checked against current master (590f012b2) and #2977/#2979/#2981/#2983/#2986:\nAC1 (no ChatGPT req after terminal/absent state, incl. receiver failure/restart): SATISFIED. Existing fixtures covered terminal-state non-recapture (background.test.js \"does not recapture an already-safe conversation on activation\") and unpaired-freshness holds (\"does not read a provider conversation from an unpaired freshness hint\"), but no fixture combined a receiver outage with a simulated service-worker restart against a terminal conversation. Added that fixture this session (\"never fetches a terminal ChatGPT conversation across a receiver outage and a service-worker restart\") — passes unchanged against current master, closing the literal AC1 wording gap.\nAC2 (typed durable nonterminal reason + bounded retry/backoff; terminal job cannot revive): SATISFIED. All automatic captureTab calls carry typed reasons (auto_capture_missing, auto_capture_unconverged_provider); retry queue bounded at 20 entries with a drop counter; freshness queue backoff evidenced by earliest-deadline test; #2977 makes archived/spooled_only receiver-owned (no automatic revive) and the #2979 circuit breaker is a hard override.\nAC3 (live proof: zero ChatGPT requests across multi-interval observation once terminal, tab stays open): SATISFIED by the 2026-07-17 live proofs already in this bead's notes (28-entry freshness observation: 0 POST /v1/browser-captures, 0 provider-transport events; 65s daemon-journal observation: 0 token_rejected events) — #2981/#2983/#2986 landed after those proofs and only tightened restrictions further (added pairing gates), so the proofs hold a fortiori. No new live proof re-run this session.\nAC4 (reversible narrow circuit-breaker, not reliant on daemon stop): SATISFIED via #2979 popup breaker (persisted, reversible, gates automatic tab capture + freshness before provider traffic).\nAC5 (active user-conversation capture stays functional, focused evidence): SATISFIED — \"captures a missing conversation once during automatic reconciliation\" plus the cited 320-test full extension-suite runs in #2986's PR description.\nVerdict: all 5 AC now satisfied. Recommend closing yyvg.6.1 once the restart-fixture PR merges. New test: browser-extension/tests/background.test.js, commit 4255f1e70 on feature/extension/action-conduit.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-17T07:09:54Z","created_by":"Sinity","updated_at":"2026-07-18T16:03:30Z","started_at":"2026-07-17T07:55:42Z","closed_at":"2026-07-18T16:03:30Z","close_reason":"All 5 AC satisfied. AC1-AC5 verdict recorded in the bead notes (2026-07-18 Lane H entry): AC1/AC2 satisfied by merged PRs #2977/#2979/#2981/#2983/#2986 plus a new fixture (PR #3098, commit 4255f1e70) closing the literal \"including receiver failure/restart\" gap the warroom It.17 sweep flagged as needing verification before close -- combines a receiver outage with a simulated service-worker restart against a terminal ChatGPT conversation, proving no capture message/script-injection occurs in either phase. AC3 satisfied by the 2026-07-17 live proofs already in the bead notes (28-entry freshness observation: 0 POST /v1/browser-captures; 65s daemon-journal observation: 0 token_rejected events) -- #2981/#2983/#2986 landed after those proofs and only tightened restrictions further, so they hold a fortiori. AC4 satisfied via the #2979 popup circuit breaker (persisted, reversible, gates automatic tab capture + freshness before provider traffic). AC5 satisfied via existing missing-conversation auto-capture tests plus the 320-test full extension-suite runs cited in #2986. Full extension suite green (335 passed, 1 pre-existing unrelated packaged-worker fixture failure reproduced unchanged); devtools verify --quick green. Merged as PR #3098 (fc124e6de).","labels":["area:capture","area:coordination","area:web","delivery:L-external-legibility","horizon:frontier","incident:rate-limit","lane:docs-demos-launch"],"dependencies":[{"issue_id":"polylogue-yyvg.6.1","depends_on_id":"polylogue-yyvg.6","type":"parent-child","created_at":"2026-07-17T09:09:53Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-lkrc.5","title":"Preserve complete raw-authority state counts in readiness","description":"Post-merge adversarial review of PR #2962 found that readiness publishes the raw-authority census postflight residual as frontier state_counts. The residual deliberately omits proven_current, so the public status field can be incomplete while presented as a complete frontier inventory. This undermines lkrc AC7 and makes status unsuitable for accountable live closure.","design":"Keep audit-friendly preflight and postflight views distinct. Readiness must derive blocking counts from the postflight state, while its exposed complete state-count inventory must include every frontier state, including proven_current, with an explicit documented source and exact census identity. Do not weaken the offline/daemon writer guard added in PR #2962.","acceptance_criteria":"1. A completed dry-run and an applied frontier census each expose a complete state-count inventory including proven_current. 2. Readiness blocking_count remains derived from postflight state and reaches zero after a repaired plan. 3. Status/readiness contract tests fail if proven_current is omitted or if preflight counts are used for postflight blocking. 4. Focused storage/status tests and devtools verify --quick pass.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-17T01:33:26Z","created_by":"Sinity","updated_at":"2026-07-17T01:48:10Z","started_at":"2026-07-17T01:42:43Z","closed_at":"2026-07-17T01:48:10Z","close_reason":"Merged PR #2965 (0dc5773a9): readiness now exposes complete postflight frontier state counts while deriving blocking only from the residual; dry-run and apply lifecycle regressions plus focused and quick gates passed.","labels":["area:browser","area:daemon","area:sources","area:storage","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-lkrc.5","depends_on_id":"polylogue-lkrc","type":"parent-child","created_at":"2026-07-17T03:33:25Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-n3an","title":"Fast-forward index v36 to v37 without raw replay","description":"The live archive is index v36 while current master expects v37. The only v37 structural delta removes session_runs, session_observed_events, and session_context_snapshots, yet the documented blanket rebuild path selects 52,066 raw rows / 74.7 GB and spent 86 seconds before writing its first session. Provide a proof-gated clone-first blue-green forward for this exact transition rather than replaying unchanged source evidence.","design":"Reflink the quiesced active v36 generation under the existing RebuildLease and IndexGenerationStore lifecycle. In the inactive clone, drop exactly the three retired cache tables and their indexes, advance user_version only after transaction success, and prove source snapshot stability, surviving schema-object parity, row-count parity for every surviving table, foreign_key_check, quick_check/integrity, and absence of retired objects. Emit a receipt and promote through IndexGenerationStore so the old generation remains the rollback target. Fail closed on any source version, schema, row-count, daemon, or lease mismatch.","acceptance_criteria":"1. A production-shaped v36 fixture fast-forwards to v37 without raw replay and preserves every surviving table count/schema object. 2. Unexpected schema objects or versions, changed source snapshot, FK/integrity failures, and a running daemon fail closed before promotion. 3. Activation uses the owned inactive generation and atomic promotion, retains rollback, and emits before/after proof. 4. Live postflight reports v37, current durable tiers, daemon healthy, and capture catch-up progressing.","status":"closed","priority":0,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-16T09:55:26Z","created_by":"Sinity","updated_at":"2026-07-16T16:49:05Z","started_at":"2026-07-16T09:55:51Z","closed_at":"2026-07-16T16:49:05Z","close_reason":"Completed by PR #2931 and live activation receipt /realm/tmp/polylogue-index-v37-fast-forward/receipt.json: status=activated, live index user_version=37, retired cache tables absent, daemon active. Current raw-revision CAS retry failures are separate lkrc/yla authority work and do not invalidate the v36→v37 fast-forward.","labels":["area:ops","area:storage","area:test"],"dependencies":[{"issue_id":"polylogue-n3an","depends_on_id":"polylogue-3v1","type":"discovered-from","created_at":"2026-07-16T11:55:25Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"polylogue-yyvg.7","title":"Make extension UX automatic and exception-driven","description":"The extension popup currently exposes implementation machinery and private Sol campaign controls as the product. Redesign it around automatic canonical capture and generic browser-action transport health: calm confidence by default, and operator interaction only for genuine ambiguity, authorization, or irrecoverable failure. Campaign queues/portfolio/integration do not belong in the extension UI.","design":"Preserve the existing two-layer design: per-message capture state blends into provider action rows; cross-conversation archive intelligence floats in the corner/deep-dive surface. The compact popup presents Now, Capture confidence, Attention, and Recent outcomes across supported tabs. Healthy automatic capture/backfill and generic BrowserActionIntent execution are summarized, not controlled. Queue leases, retries, receiver identity, request ids, hashes, raw phases, provider circuits, and dev endpoint overrides live under progressive-disclosure diagnostics. At most one contextual action appears per genuine attention item. Receiver pairing prefers canonical endpoint and self-heals only on matching stable identity. The popup has no mission, handoff, campaign cadence, result package, Beads, worktree, or Terra portfolio. Those are external orchestrator views. Preserve keyboard/accessibility, zero layout shift, offline last-known state, and no foreground activation.","acceptance_criteria":"1. Healthy popup shows one compact summary, current-conversation capture confidence, pending generic browser-action count, and recent capture/action outcomes; no lease/cadence/request-id/raw-phase controls, receiver form, launch job, handoff, or campaign portfolio appears by default. 2. Capture/backfill and transport transitions decidable from evidence occur automatically; operator is not asked to sync, retry unambiguous pre-submit failure, resume transient backoff, close tabs, or reacquire assets. 3. Attention is limited to typed auth/pairing mismatch, action outcome_unknown, explicit submit/destructive approval, provider capability mismatch, or destructive conflict, with at most one primary resolution plus Details. 4. Advanced diagnostics preserves endpoint/receiver/extension contract, generic action and capture events, cooldown/lease evidence, overrides, and support export without duplicating authority. 5. Ambient, message layer, and popup consume one status/identity presentation model and preserve the resolved two-layer UX design. 6. Popup height and DOM/action budget are executable; larger archive/campaign views live outside the popup. 7. Live installed proof covers healthy state, offline recovery, one attention item, keyboard navigation, two extension instances, and background operation without foreground activation.","notes":"Existing design implementation matrix (authority, not inspiration): F1/bkff/3v1 owns N-tab popup, active-conversation detail, captured-vs-visible confidence, cost/tokens where available, and What Polylogue did here timeline; retain this information architecture while removing manual supervision and campaign controls. F6/3v1/r2kb owns calm explicit states (safe/current, catching up, receiver offline as normal, partial fidelity, not saved, failed) with cause, evidence time, next automatic action, and typed attention only. F2/F3/wvji/90y owns the fixed corner chip plus 360px slide-over for cross-conversation archive intelligence/timeline; do not move it inline. F4/ys30/yyvg.4 owns capture dot and Save action in provider-native per-message action rows, resolved by one ProviderAdapter identity contract; no ordinal/text-only durable authority. F5/bj5h owns selection-to-assertion with exact evidence and candidate judgment. l40k owns N-tab aggregate and calm bounded offline spool. yajm/x5k3/qvgt/3nmf retain readable typography, passive refresh, button feedback, redacted exportable diagnostics, and live responsiveness proof. ptx/yqof owns reverse controls, dry-run/authorization/receipt posture. yyvg.1/yyvg.2 owns rename/project plans and collection observations. The resolved visual rule and existing mockup remain: per-message state blends in; cross-conversation intelligence floats. Implement changes against docs/design/browser-capture-redesign/mockup.dc.html and f2-fixed-verification.png; do not invent a temporary alternative popup.\n2026-07-16 first merged UX slice: PR #2928 (165e6a034) establishes automatic maintenance, neutral non-conversation pages, compact current-page semantics, and no private campaign controls. This does not close the full comprehensive F1-F6 design: progressive diagnostics, bounded compact layout, exception-only attention, and full ambient/message-layer convergence remain.\nWarroom sweep It.17: claiming session closed; #2972 (stop replacing operator-owned transport tabs) landed after the #2928 UX slice. Residue: full F1-F6 design (progressive diagnostics, compact-layout bounds, exception-only attention, popup/ambient/message-layer convergence). Reset to open.\n2026-07-18 Lane H scoped slice (commit d7d6cac37 on feature/extension/action-conduit, PR #3098): NOT full closure — this is a real but partial slice against the F1-F6 design matrix, recorded honestly per AC.\nAC1 (healthy popup: compact summary + capture confidence + pending browser-action count + recent outcomes; no lease/cadence/request-id/raw-phase/receiver-form/campaign-portfolio by default): PARTIALLY SATISFIED. Added a \"Pending browser actions\" count (backed by the new ptx BrowserActionIntent conduit's GET /v1/browser-actions via the existing polylogue.browserActions.status message) and wrapped receiver pairing/reset, work queue, backfill panel, recent capture log, debug log, and receiver settings into one collapsed `\u003cdetails id=\"diagnostics\"\u003e` (closed by default). The \"What Polylogue did here\" timeline serves as recent outcomes and stays visible per-conversation. Campaign/portfolio controls were already absent (removed in #2928/#2929 prior to this session). NOT done: the compact bounded-height \"Now\" summary layout from the mockup; current-conversation capture confidence is still the old always-visible active-card, not a compact one-line summary.\nAC2 (automatic decisions without asking): unchanged from prior state — already satisfied by existing automatic capture/freshness/reconciliation logic (yyvg.6.1 lineage); this slice did not touch that logic.\nAC3 (attention limited to typed categories, at most one primary resolution + Details): PARTIALLY SATISFIED. Added computeAttention() in operator_status.js as a strict single-item priority list covering auth/pairing mismatch, action outcome_unknown (new: reads the ptx conduit's browser-action ledger for a stuck \"outcome_unknown\" action — the first UI surface to expose that state at all), typed provider capability mismatch on queued work, and hard archive failure. NOT covered: \"explicit submit/destructive approval\" and \"destructive conflict\" categories — no current data model backs them (ptx has no submit-approval-required flag yet), so they were not fabricated; left as an honest gap for whichever bead adds that data.\nAC4 (progressive-disclosure diagnostics preserves authority, no duplication): SATISFIED for the sections moved into \u003cdetails id=\"diagnostics\"\u003e — clicking the attention action (e.g. \"Reset pairing\") opens diagnostics and drives the real underlying control/button rather than a second copy.\nAC5 (ambient/message-layer/popup share one status/identity model): SATISFIED — computeAttention/pendingBrowserActionCount added to the existing shared operator_status.js module (already used by ambient_surface.js, message_layer.js, popup.js), not a new parallel model.\nAC6 (popup height/DOM/action budget executable): NOT attempted this session (no executable budget check exists yet to satisfy).\nAC7 (live installed proof: healthy/offline/attention/keyboard/two-instances/background): NOT attempted — deferred to the lane's operator-run final smoke per the lane prompt's SEMI-ATTENDED framing; this session only delivered code+unit-verifiable slices as instructed.\nRemaining for full yyvg.7 closure: pixel-level mockup convergence (docs/design/browser-capture-redesign/mockup.dc.html), compact bounded \"Now\" summary replacing the current active-card, explicit-approval/destructive-conflict attention categories once a backing data model exists, DOM/height budget check, and the live installed proof.\n2026-07-18 Lane H follow-up slice (commit d4c73d6c19 on feature/extension/exception-driven-popup-compaction, PR #3126): addresses two items the prior note listed as NOT done. AC1: moved the always-visible active-cards Fidelity and Assets/asset-failures rows into a new \"Capture detail\" section inside the existing \u003cdetails id=\"diagnostics\"\u003e (element ids unchanged; popup.js untouched). Always-visible card is now state chip + captured/visible count + cost/tokens (3 rows, down from 5) -- the compact bounded \"Now\" summary gap from the prior note is closed. AC6: added an executable test (tests/popup.test.js) asserting the always-visible surface (outside #diagnostics) stays \u003c= 8 interactive controls and \u003c= 90 DOM nodes (current measured: 6/67, headroom built in) -- the \"no executable budget check exists yet\" gap is closed. Verification: npx vitest run tests/popup.test.js (30 passed), full extension suite 339 passed / 1 pre-existing unrelated failure, npm run lint+validate clean, devtools verify --quick exit 0. Still open for full yyvg.7 closure: pixel-level mockup convergence (docs/design/browser-capture-redesign/mockup.dc.html), explicit-approval/destructive-conflict attention categories (no backing data model yet), live installed proof (deferred to operator per lane SEMI-ATTENDED framing).\n2026-07-18 Lane H mockup-convergence slice (commit 439d893e3 on feature/extension/exception-driven-popup-compaction, PR #3126, second commit): pure-CSS convergence of the popup on docs/design/browser-capture-redesign/mockup.dc.html F1/F6 visual language -- no DOM/JS changes. Palette: dark-mode custom properties now use the mockups exact hex values (surface #0b0e13, panels #12161d/#171c25, ok/warn/bad #4ec98f/#e6b552/#f06a6a, violet accent #8b7bf2/#6d5ae0); light mode shifted to the same hue family tuned for contrast on white; provider-logo colors (ChatGPT/Claude) now match mockup brand hex exactly. Badges/pills converted from solid-fill chips to the mockups tinted-pill + colored-dot pattern (::before pseudo-element, per-tone --dot custom property -- zero new DOM nodes, DOM/action budget test from the prior commit unaffected). Radius rhythm bumped 7-9px -\u003e 9-13px to match. Primary buttons: solid fill -\u003e violet gradient + glow shadow. Added IBM Plex Sans/Mono as first-choice fonts (body text, provider-logo initials, new shared .mono utility on numeric/data values -- cost, tokens, counts, timestamps, request ids) with no remote font loading, consistent with the READMEs no-remote-assets constraint -- degrades to existing system-font stacks. Verification: npx vitest run tests/popup.test.js (30 passed), full suite 339 passed/1 pre-existing unrelated failure (unchanged), npm run lint+validate clean, devtools verify --quick exit 0 both before and after commit. This closes the bulk of the F1/F6 \"pixel-level mockup convergence\" gap the prior note flagged -- the popups content/vocabulary already matched the mockup (see prior note); this slice brings the visual system (color/typography/pill-shape/radius) into alignment too. NOT attempted: literal pixel-for-pixel layout match (multi-tab card proportions, exact spacing values, the mockups radial-gradient masthead treatment -- judged out of scope for a functional popup vs. a marketing design canvas). Still open for full yyvg.7 closure: explicit-approval/destructive-conflict attention categories (blocked on a ptx-side data model that does not exist yet), live installed proof (deferred to operator per lane SEMI-ATTENDED framing).\n2026-07-18 PR #3126 merged to master as 39f6c39d2 (squash). Both commits (active-card compaction + DOM/action budget test, and popup visual-language mockup convergence) are now on master. CI green (CodeRabbit/GitGuardian/CircleCI quick-gate all pass, no substantive review findings to triage). Worktree feature/extension/exception-driven-popup-compaction reset to origin/master post-merge (branch content fully subsumed, remote head auto-deleted by repo setting).\n2026-07-19 Lane H live installed proof (AC7), agent-run via sinnix-chrome-control private-visible Chrome (operator explicitly authorized doing this directly): loaded PR #3126s shipped popup against a real authenticated ChatGPT+Claude.ai session. Results per AC7 clause:\n- Healthy state: SATISFIED. Screenshot confirms compact 3-row active-card, diagnostics collapsed, no attention item, idle badge -- matches the shipped mockup-convergence work exactly.\n- One attention item: SATISFIED. Before pairing, popup correctly showed exactly one attention item (\"Receiver requires its pairing token\") with a single primary action (\"Open receiver settings\") and no other controls -- real evidence, not staged.\n- Keyboard navigation: SATISFIED. Used element.checkVisibility() (correctly accounts for closed-\u003cdetails\u003e clipping, unlike offsetParent/getClientRects which false-positive on clipped-but-boxed elements) to enumerate the REAL Tab-reachable set in the healthy/no-conversation state: exactly 2 elements (the two ambient toggles), diagnostics content correctly unreachable while collapsed, no dead-ends.\n- Two concurrent tabs: SATISFIED. Opened two real, distinct, pre-existing conversations (one ChatGPT, one Claude.ai) simultaneously; popup \"Open conversations\" correctly showed 2, each with correct per-provider color/badge, active tab highlighted with the violet accent border, independently tracked (no cross-tab interference).\n- Offline recovery: ATTEMPTED, NOT CLEANLY PROVEN -- see incident note on polylogue-ptx. First attempt used a scratch daemon that (unknown to me at the time) had crashed on a port-8765 collision with the real polylogued.service; token-path bug (polylogue-x2q3) meant my pairing silently authenticated against the real daemon instead of failing loudly. Second attempt used a correctly-isolated alternate-port (18765) scratch daemon with verified process/port ownership; killed it to simulate offline, but forcing a health check (chrome.runtime.sendMessage polylogue.checkReceiverHealth) triggered the extensions allowCanonicalRecovery self-heal, which silently reconnected to the canonical default endpoint (127.0.0.1:8765 = the REAL daemon, since receiver_id is also not archive-scoped) rather than showing an offline/degraded state against my isolated instance. This is a genuine, real, valuable finding (documented on polylogue-x2q3) but means offline-recovery was not cleanly demonstrated against a safely-isolated receiver this session. No further live attempts were made after this discovery to avoid a third production-touching incident. The receiver_offline/catching_up state vocabulary itself is verified present in operator_status.js by source inspection (OPERATOR_STATUS.receiver_offline, badge=[\"warn\",\"receiver offline\"]) but not exercised live and confirmed working end-to-end.\n\nReal-archive incident summary: two of the ptx live-proof actions (create+reply) and their captured content briefly landed in the production archive due to polylogue-x2q3s token-path bug colliding with a default-port scratch-daemon crash. Fully cleaned up same session (deleted ingested session/spool/action-ledger from the real archive, deleted the real ChatGPT test conversation+project via the UI, verified clean via FTS grep -- only remaining hit is this own Claude Code sessions own transcript, which is correct/expected, not test pollution). polylogued.service was never disrupted and continued its own real ingestion throughout.\n\nNet for yyvg.7 AC7: 5 of 6 sub-scenarios (healthy, one attention item, keyboard nav, two tabs, no-foreground-activation via ptx) cleanly live-proven with real evidence. Offline recovery remains open, blocked on either (a) fixing polylogue-x2q3 first so a scratch receiver can be safely isolated, or (b) accepting a live test against production with the real daemon briefly stopped (requires explicit operator sign-off, not attempted here).\n2026-07-19 real AC5 gap found via operator question on screenshot 17: the \"Open conversations\" list and the \"Current page\" active-card can transiently disagree for the SAME conversation. Live-reproduced and root-caused: renderOpenTabs (list) reads polylogueSessionLedger (chrome.storage.local, synchronous, updates promptly per capture); the active-card goes through activeConversationState(tab, mission?.state || stored.polylogueState, ledger) in popup.js, which PREFERS mission?.state from a separate async loadMissionSnapshot() round-trip to the background script over the ledger when mission.state has a provider/session set (popup.js:625-629, activeConversationState:158-167). In the screenshot-17 case this fired right after forcing a receiver reconfiguration (checkReceiverHealth triggering the canonical-endpoint self-heal, see polylogue-x2q3) -- the mission-snapshot round-trip very plausibly raced/returned stale data during that transition, so the active-card fell through activeConversationState to the generic \"Receiver online. Open a supported conversation to capture.\" fallback (operator_status.js:256-259) for one render pass, while the ledger-driven list correctly showed \"Safe / current\" for the identical conversation. Verified this is NOT a general/persistent bug: re-tested with a clean, non-transitioning receiver state (properly `activate`d tab, stable connection) and both surfaces agreed (both archived/captured=true, matching ledger and mission.state). So the gap is specifically a transient race between the ledger (fast, synchronous) and the mission-snapshot fetch (async, can be stale) during receiver reconfiguration/reconnection windows, not a permanent inconsistency. This is a real violation of AC5s \"ambient, message layer, and popup consume ONE status/identity presentation model\" -- two data sources for what should be one fact (is this conversation captured), that can disagree during exactly the kind of receiver-transition window the popup is supposed to represent calmly and correctly. Fix direction: activeConversationState should not let a stale/racy mission.state override a fresher ledger entry -- prefer whichever of the two has a newer updated_at, or drop the mission.state preference for archive/capture status entirely and source it solely from the ledger (single source of truth), reserving mission.state for receiver/pairing/health fields the ledger does not carry. Not fixed this session (discovered via live evidence during AC7 proof review, out of the current session budget to safely re-test a fix live given the polylogue-x2q3 self-heal complication) -- recommend a small focused follow-up bead scoped to activeConversationState()s source-of-truth precedence.\n2026-07-19 both discovered gaps fixed and merged same session, operator-directed (\"you could work on fixing both this and x2q3\"):\n- polylogue-x2q3 (token/spool archive-scoping root cause): PR #3137 merged as cdec1481f. Closed.\n- AC5 ledger-vs-mission-snapshot race (this bead): PR #3139 merged as d66041fce. activeConversationState() in popup.js now prefers whichever of globalState/ledger has the newer updated_at when both agree on the tracked conversation, instead of unconditionally trusting globalState. New test reproduces the exact live-observed race and was verified to fail pre-fix, pass post-fix. Narrow, surgical -- does not touch the separately-tested sibling branch where globalState omits provider/session (an intentional \"describes the current context implicitly\" contract).\nNet: AC7 live-proof status unchanged from the prior note (5/6 sub-scenarios clean, offline-recovery still blocked on the extensions canonical-endpoint self-heal design -- though note x2q3s fix means a scratch instance no longer SILENTLY shares identity with production if the self-heal does trigger; it would at least reconnect to a receiver with a genuinely different receiver_id now, which may itself surface as a visible mismatch/attention state worth a future live re-check). AC5 gap is now closed.\n2026-07-22 lane re-verified bead record against master during PR #3260 work: consistent; still-open items (explicit-approval attention categories, live offline-recovery proof) unchanged — blocked on missing data model / live browser respectively.\n2026-07-27 explicit-approval data model slice (PR #3329, branch feature/extension/explicit-approval-attention): implements ONE of the two remaining scope items named in the prior note -- \"explicit-approval/destructive-conflict attention categories once a backing data model exists\". The \"live offline-recovery proof\" item is untouched (still needs a live browser fixture).\n\nRoot cause confirmed by reading dispatchBrowserAction in background.js: a submit_once conversation.reply action was leased and executed by the poll loop with zero operator gate, even though it posts one real, provider-visible turn into an EXISTING conversation with no automatic undo -- exactly the \"explicit submit/destructive approval\" AC3 category that had no backing data model.\n\nWhat shipped: BrowserActionStatus gains \"awaiting_approval\"; BrowserActionIntent gains requires_operator_approval/approval_reason/approval_requested_at/approved_at/approved_by/declined_at (polylogue/browser_capture/models.py). enqueue_action holds submit_once+conversation.reply at \"awaiting_approval\" instead of \"queued\" so claim_action can never lease it; new decide_action_approval records approve (-\u003e queued, claimable for the first time) or decline (-\u003e cancelled, terminal); a new POST /v1/browser-actions/{id}/approval route (polylogue/browser_capture/actions.py, server.py, route_contracts.py). computeAttention in operator_status.js gets a new explicit_approval_required branch ranked between action_outcome_unknown and capability_mismatch per AC3's stated order. popup.html/popup.js add a \"Browser action approval\" panel inside the existing progressive-disclosure diagnostics (no AC4 duplication) with Approve/Decline buttons; the attention item's single primary action (\"Review request\") opens diagnostics and focuses the panel -- never a silent auto-resolution. background.js adds decideBrowserActionApproval, wiring the poll loop to wake only on approve.\n\nDeliberately NOT modeled: a \"destructive_conflict\" reason. claim_action already serializes to exactly one in-flight action at a time, so there is no genuine two-actions-racing-for-one-resource scenario in the current architecture to attach a conflict decision to -- adding an enum value with no real trigger would be unbacked/unfireable. Documented as an explicit gap in models.py for a future bead once a real collision scenario exists (e.g. multi-instance orchestration). So AC3's \"explicit submit/destructive approval\" is now satisfied; \"destructive conflict\" remains open, this time for a concrete architectural reason rather than a missing data model.\n\nVerified: devtools test tests/unit/browser_capture/ 153 passed (was 143, +10 new); mypy clean on the 4 touched browser_capture files; devtools verify --quick exit 0; browser-extension npx vitest run 354/355 passed (1 pre-existing unrelated build.mjs failure, confirmed identical on origin/master via git stash before this change); npm run lint + validate clean. NOT verified: visual rendering of the new diagnostics panel and an end-to-end trigger-to-popup-display proof against a real ChatGPT session -- needs a live browser, out of scope for this task per instruction.\n\nRemaining for full yyvg.7 closure: destructive_conflict category (blocked on a real trigger scenario not yet existing), live offline-recovery proof (blocked on a live browser fixture, per the prior open incident on polylogue-x2q3's self-heal behavior).","status":"in_progress","priority":0,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-16T04:44:54Z","created_by":"Sinity","updated_at":"2026-07-27T15:02:40Z","started_at":"2026-07-16T04:45:12Z","labels":["area:capture","area:web","delivery:L-external-legibility","horizon:frontier","horizon:mid","lane:docs-demos-launch"],"dependencies":[{"issue_id":"polylogue-yyvg.7","depends_on_id":"polylogue-yyvg","type":"parent-child","created_at":"2026-07-16T06:44:53Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-yyvg.7","title":"Make extension UX automatic and exception-driven","description":"The extension popup currently exposes implementation machinery and private Sol campaign controls as the product. Redesign it around automatic canonical capture and generic browser-action transport health: calm confidence by default, and operator interaction only for genuine ambiguity, authorization, or irrecoverable failure. Campaign queues/portfolio/integration do not belong in the extension UI.","design":"Preserve the existing two-layer design: per-message capture state blends into provider action rows; cross-conversation archive intelligence floats in the corner/deep-dive surface. The compact popup presents Now, Capture confidence, Attention, and Recent outcomes across supported tabs. Healthy automatic capture/backfill and generic BrowserActionIntent execution are summarized, not controlled. Queue leases, retries, receiver identity, request ids, hashes, raw phases, provider circuits, and dev endpoint overrides live under progressive-disclosure diagnostics. At most one contextual action appears per genuine attention item. Receiver pairing prefers canonical endpoint and self-heals only on matching stable identity. The popup has no mission, handoff, campaign cadence, result package, Beads, worktree, or Terra portfolio. Those are external orchestrator views. Preserve keyboard/accessibility, zero layout shift, offline last-known state, and no foreground activation.","acceptance_criteria":"1. Healthy popup shows one compact summary, current-conversation capture confidence, pending generic browser-action count, and recent capture/action outcomes; no lease/cadence/request-id/raw-phase controls, receiver form, launch job, handoff, or campaign portfolio appears by default. 2. Capture/backfill and transport transitions decidable from evidence occur automatically; operator is not asked to sync, retry unambiguous pre-submit failure, resume transient backoff, close tabs, or reacquire assets. 3. Attention is limited to typed auth/pairing mismatch, action outcome_unknown, explicit submit/destructive approval, provider capability mismatch, or destructive conflict, with at most one primary resolution plus Details. 4. Advanced diagnostics preserves endpoint/receiver/extension contract, generic action and capture events, cooldown/lease evidence, overrides, and support export without duplicating authority. 5. Ambient, message layer, and popup consume one status/identity presentation model and preserve the resolved two-layer UX design. 6. Popup height and DOM/action budget are executable; larger archive/campaign views live outside the popup. 7. Live installed proof covers healthy state, offline recovery, one attention item, keyboard navigation, two extension instances, and background operation without foreground activation.","notes":"Existing design implementation matrix (authority, not inspiration): F1/bkff/3v1 owns N-tab popup, active-conversation detail, captured-vs-visible confidence, cost/tokens where available, and What Polylogue did here timeline; retain this information architecture while removing manual supervision and campaign controls. F6/3v1/r2kb owns calm explicit states (safe/current, catching up, receiver offline as normal, partial fidelity, not saved, failed) with cause, evidence time, next automatic action, and typed attention only. F2/F3/wvji/90y owns the fixed corner chip plus 360px slide-over for cross-conversation archive intelligence/timeline; do not move it inline. F4/ys30/yyvg.4 owns capture dot and Save action in provider-native per-message action rows, resolved by one ProviderAdapter identity contract; no ordinal/text-only durable authority. F5/bj5h owns selection-to-assertion with exact evidence and candidate judgment. l40k owns N-tab aggregate and calm bounded offline spool. yajm/x5k3/qvgt/3nmf retain readable typography, passive refresh, button feedback, redacted exportable diagnostics, and live responsiveness proof. ptx/yqof owns reverse controls, dry-run/authorization/receipt posture. yyvg.1/yyvg.2 owns rename/project plans and collection observations. The resolved visual rule and existing mockup remain: per-message state blends in; cross-conversation intelligence floats. Implement changes against docs/design/browser-capture-redesign/mockup.dc.html and f2-fixed-verification.png; do not invent a temporary alternative popup.\n2026-07-16 first merged UX slice: PR #2928 (165e6a034) establishes automatic maintenance, neutral non-conversation pages, compact current-page semantics, and no private campaign controls. This does not close the full comprehensive F1-F6 design: progressive diagnostics, bounded compact layout, exception-only attention, and full ambient/message-layer convergence remain.\nWarroom sweep It.17: claiming session closed; #2972 (stop replacing operator-owned transport tabs) landed after the #2928 UX slice. Residue: full F1-F6 design (progressive diagnostics, compact-layout bounds, exception-only attention, popup/ambient/message-layer convergence). Reset to open.\n2026-07-18 Lane H scoped slice (commit d7d6cac37 on feature/extension/action-conduit, PR #3098): NOT full closure — this is a real but partial slice against the F1-F6 design matrix, recorded honestly per AC.\nAC1 (healthy popup: compact summary + capture confidence + pending browser-action count + recent outcomes; no lease/cadence/request-id/raw-phase/receiver-form/campaign-portfolio by default): PARTIALLY SATISFIED. Added a \"Pending browser actions\" count (backed by the new ptx BrowserActionIntent conduit's GET /v1/browser-actions via the existing polylogue.browserActions.status message) and wrapped receiver pairing/reset, work queue, backfill panel, recent capture log, debug log, and receiver settings into one collapsed `\u003cdetails id=\"diagnostics\"\u003e` (closed by default). The \"What Polylogue did here\" timeline serves as recent outcomes and stays visible per-conversation. Campaign/portfolio controls were already absent (removed in #2928/#2929 prior to this session). NOT done: the compact bounded-height \"Now\" summary layout from the mockup; current-conversation capture confidence is still the old always-visible active-card, not a compact one-line summary.\nAC2 (automatic decisions without asking): unchanged from prior state — already satisfied by existing automatic capture/freshness/reconciliation logic (yyvg.6.1 lineage); this slice did not touch that logic.\nAC3 (attention limited to typed categories, at most one primary resolution + Details): PARTIALLY SATISFIED. Added computeAttention() in operator_status.js as a strict single-item priority list covering auth/pairing mismatch, action outcome_unknown (new: reads the ptx conduit's browser-action ledger for a stuck \"outcome_unknown\" action — the first UI surface to expose that state at all), typed provider capability mismatch on queued work, and hard archive failure. NOT covered: \"explicit submit/destructive approval\" and \"destructive conflict\" categories — no current data model backs them (ptx has no submit-approval-required flag yet), so they were not fabricated; left as an honest gap for whichever bead adds that data.\nAC4 (progressive-disclosure diagnostics preserves authority, no duplication): SATISFIED for the sections moved into \u003cdetails id=\"diagnostics\"\u003e — clicking the attention action (e.g. \"Reset pairing\") opens diagnostics and drives the real underlying control/button rather than a second copy.\nAC5 (ambient/message-layer/popup share one status/identity model): SATISFIED — computeAttention/pendingBrowserActionCount added to the existing shared operator_status.js module (already used by ambient_surface.js, message_layer.js, popup.js), not a new parallel model.\nAC6 (popup height/DOM/action budget executable): NOT attempted this session (no executable budget check exists yet to satisfy).\nAC7 (live installed proof: healthy/offline/attention/keyboard/two-instances/background): NOT attempted — deferred to the lane's operator-run final smoke per the lane prompt's SEMI-ATTENDED framing; this session only delivered code+unit-verifiable slices as instructed.\nRemaining for full yyvg.7 closure: pixel-level mockup convergence (docs/design/browser-capture-redesign/mockup.dc.html), compact bounded \"Now\" summary replacing the current active-card, explicit-approval/destructive-conflict attention categories once a backing data model exists, DOM/height budget check, and the live installed proof.\n2026-07-18 Lane H follow-up slice (commit d4c73d6c19 on feature/extension/exception-driven-popup-compaction, PR #3126): addresses two items the prior note listed as NOT done. AC1: moved the always-visible active-cards Fidelity and Assets/asset-failures rows into a new \"Capture detail\" section inside the existing \u003cdetails id=\"diagnostics\"\u003e (element ids unchanged; popup.js untouched). Always-visible card is now state chip + captured/visible count + cost/tokens (3 rows, down from 5) -- the compact bounded \"Now\" summary gap from the prior note is closed. AC6: added an executable test (tests/popup.test.js) asserting the always-visible surface (outside #diagnostics) stays \u003c= 8 interactive controls and \u003c= 90 DOM nodes (current measured: 6/67, headroom built in) -- the \"no executable budget check exists yet\" gap is closed. Verification: npx vitest run tests/popup.test.js (30 passed), full extension suite 339 passed / 1 pre-existing unrelated failure, npm run lint+validate clean, devtools verify --quick exit 0. Still open for full yyvg.7 closure: pixel-level mockup convergence (docs/design/browser-capture-redesign/mockup.dc.html), explicit-approval/destructive-conflict attention categories (no backing data model yet), live installed proof (deferred to operator per lane SEMI-ATTENDED framing).\n2026-07-18 Lane H mockup-convergence slice (commit 439d893e3 on feature/extension/exception-driven-popup-compaction, PR #3126, second commit): pure-CSS convergence of the popup on docs/design/browser-capture-redesign/mockup.dc.html F1/F6 visual language -- no DOM/JS changes. Palette: dark-mode custom properties now use the mockups exact hex values (surface #0b0e13, panels #12161d/#171c25, ok/warn/bad #4ec98f/#e6b552/#f06a6a, violet accent #8b7bf2/#6d5ae0); light mode shifted to the same hue family tuned for contrast on white; provider-logo colors (ChatGPT/Claude) now match mockup brand hex exactly. Badges/pills converted from solid-fill chips to the mockups tinted-pill + colored-dot pattern (::before pseudo-element, per-tone --dot custom property -- zero new DOM nodes, DOM/action budget test from the prior commit unaffected). Radius rhythm bumped 7-9px -\u003e 9-13px to match. Primary buttons: solid fill -\u003e violet gradient + glow shadow. Added IBM Plex Sans/Mono as first-choice fonts (body text, provider-logo initials, new shared .mono utility on numeric/data values -- cost, tokens, counts, timestamps, request ids) with no remote font loading, consistent with the READMEs no-remote-assets constraint -- degrades to existing system-font stacks. Verification: npx vitest run tests/popup.test.js (30 passed), full suite 339 passed/1 pre-existing unrelated failure (unchanged), npm run lint+validate clean, devtools verify --quick exit 0 both before and after commit. This closes the bulk of the F1/F6 \"pixel-level mockup convergence\" gap the prior note flagged -- the popups content/vocabulary already matched the mockup (see prior note); this slice brings the visual system (color/typography/pill-shape/radius) into alignment too. NOT attempted: literal pixel-for-pixel layout match (multi-tab card proportions, exact spacing values, the mockups radial-gradient masthead treatment -- judged out of scope for a functional popup vs. a marketing design canvas). Still open for full yyvg.7 closure: explicit-approval/destructive-conflict attention categories (blocked on a ptx-side data model that does not exist yet), live installed proof (deferred to operator per lane SEMI-ATTENDED framing).\n2026-07-18 PR #3126 merged to master as 39f6c39d2 (squash). Both commits (active-card compaction + DOM/action budget test, and popup visual-language mockup convergence) are now on master. CI green (CodeRabbit/GitGuardian/CircleCI quick-gate all pass, no substantive review findings to triage). Worktree feature/extension/exception-driven-popup-compaction reset to origin/master post-merge (branch content fully subsumed, remote head auto-deleted by repo setting).\n2026-07-19 Lane H live installed proof (AC7), agent-run via sinnix-chrome-control private-visible Chrome (operator explicitly authorized doing this directly): loaded PR #3126s shipped popup against a real authenticated ChatGPT+Claude.ai session. Results per AC7 clause:\n- Healthy state: SATISFIED. Screenshot confirms compact 3-row active-card, diagnostics collapsed, no attention item, idle badge -- matches the shipped mockup-convergence work exactly.\n- One attention item: SATISFIED. Before pairing, popup correctly showed exactly one attention item (\"Receiver requires its pairing token\") with a single primary action (\"Open receiver settings\") and no other controls -- real evidence, not staged.\n- Keyboard navigation: SATISFIED. Used element.checkVisibility() (correctly accounts for closed-\u003cdetails\u003e clipping, unlike offsetParent/getClientRects which false-positive on clipped-but-boxed elements) to enumerate the REAL Tab-reachable set in the healthy/no-conversation state: exactly 2 elements (the two ambient toggles), diagnostics content correctly unreachable while collapsed, no dead-ends.\n- Two concurrent tabs: SATISFIED. Opened two real, distinct, pre-existing conversations (one ChatGPT, one Claude.ai) simultaneously; popup \"Open conversations\" correctly showed 2, each with correct per-provider color/badge, active tab highlighted with the violet accent border, independently tracked (no cross-tab interference).\n- Offline recovery: ATTEMPTED, NOT CLEANLY PROVEN -- see incident note on polylogue-ptx. First attempt used a scratch daemon that (unknown to me at the time) had crashed on a port-8765 collision with the real polylogued.service; token-path bug (polylogue-x2q3) meant my pairing silently authenticated against the real daemon instead of failing loudly. Second attempt used a correctly-isolated alternate-port (18765) scratch daemon with verified process/port ownership; killed it to simulate offline, but forcing a health check (chrome.runtime.sendMessage polylogue.checkReceiverHealth) triggered the extensions allowCanonicalRecovery self-heal, which silently reconnected to the canonical default endpoint (127.0.0.1:8765 = the REAL daemon, since receiver_id is also not archive-scoped) rather than showing an offline/degraded state against my isolated instance. This is a genuine, real, valuable finding (documented on polylogue-x2q3) but means offline-recovery was not cleanly demonstrated against a safely-isolated receiver this session. No further live attempts were made after this discovery to avoid a third production-touching incident. The receiver_offline/catching_up state vocabulary itself is verified present in operator_status.js by source inspection (OPERATOR_STATUS.receiver_offline, badge=[\"warn\",\"receiver offline\"]) but not exercised live and confirmed working end-to-end.\n\nReal-archive incident summary: two of the ptx live-proof actions (create+reply) and their captured content briefly landed in the production archive due to polylogue-x2q3s token-path bug colliding with a default-port scratch-daemon crash. Fully cleaned up same session (deleted ingested session/spool/action-ledger from the real archive, deleted the real ChatGPT test conversation+project via the UI, verified clean via FTS grep -- only remaining hit is this own Claude Code sessions own transcript, which is correct/expected, not test pollution). polylogued.service was never disrupted and continued its own real ingestion throughout.\n\nNet for yyvg.7 AC7: 5 of 6 sub-scenarios (healthy, one attention item, keyboard nav, two tabs, no-foreground-activation via ptx) cleanly live-proven with real evidence. Offline recovery remains open, blocked on either (a) fixing polylogue-x2q3 first so a scratch receiver can be safely isolated, or (b) accepting a live test against production with the real daemon briefly stopped (requires explicit operator sign-off, not attempted here).\n2026-07-19 real AC5 gap found via operator question on screenshot 17: the \"Open conversations\" list and the \"Current page\" active-card can transiently disagree for the SAME conversation. Live-reproduced and root-caused: renderOpenTabs (list) reads polylogueSessionLedger (chrome.storage.local, synchronous, updates promptly per capture); the active-card goes through activeConversationState(tab, mission?.state || stored.polylogueState, ledger) in popup.js, which PREFERS mission?.state from a separate async loadMissionSnapshot() round-trip to the background script over the ledger when mission.state has a provider/session set (popup.js:625-629, activeConversationState:158-167). In the screenshot-17 case this fired right after forcing a receiver reconfiguration (checkReceiverHealth triggering the canonical-endpoint self-heal, see polylogue-x2q3) -- the mission-snapshot round-trip very plausibly raced/returned stale data during that transition, so the active-card fell through activeConversationState to the generic \"Receiver online. Open a supported conversation to capture.\" fallback (operator_status.js:256-259) for one render pass, while the ledger-driven list correctly showed \"Safe / current\" for the identical conversation. Verified this is NOT a general/persistent bug: re-tested with a clean, non-transitioning receiver state (properly `activate`d tab, stable connection) and both surfaces agreed (both archived/captured=true, matching ledger and mission.state). So the gap is specifically a transient race between the ledger (fast, synchronous) and the mission-snapshot fetch (async, can be stale) during receiver reconfiguration/reconnection windows, not a permanent inconsistency. This is a real violation of AC5s \"ambient, message layer, and popup consume ONE status/identity presentation model\" -- two data sources for what should be one fact (is this conversation captured), that can disagree during exactly the kind of receiver-transition window the popup is supposed to represent calmly and correctly. Fix direction: activeConversationState should not let a stale/racy mission.state override a fresher ledger entry -- prefer whichever of the two has a newer updated_at, or drop the mission.state preference for archive/capture status entirely and source it solely from the ledger (single source of truth), reserving mission.state for receiver/pairing/health fields the ledger does not carry. Not fixed this session (discovered via live evidence during AC7 proof review, out of the current session budget to safely re-test a fix live given the polylogue-x2q3 self-heal complication) -- recommend a small focused follow-up bead scoped to activeConversationState()s source-of-truth precedence.\n2026-07-19 both discovered gaps fixed and merged same session, operator-directed (\"you could work on fixing both this and x2q3\"):\n- polylogue-x2q3 (token/spool archive-scoping root cause): PR #3137 merged as cdec1481f. Closed.\n- AC5 ledger-vs-mission-snapshot race (this bead): PR #3139 merged as d66041fce. activeConversationState() in popup.js now prefers whichever of globalState/ledger has the newer updated_at when both agree on the tracked conversation, instead of unconditionally trusting globalState. New test reproduces the exact live-observed race and was verified to fail pre-fix, pass post-fix. Narrow, surgical -- does not touch the separately-tested sibling branch where globalState omits provider/session (an intentional \"describes the current context implicitly\" contract).\nNet: AC7 live-proof status unchanged from the prior note (5/6 sub-scenarios clean, offline-recovery still blocked on the extensions canonical-endpoint self-heal design -- though note x2q3s fix means a scratch instance no longer SILENTLY shares identity with production if the self-heal does trigger; it would at least reconnect to a receiver with a genuinely different receiver_id now, which may itself surface as a visible mismatch/attention state worth a future live re-check). AC5 gap is now closed.\n2026-07-22 lane re-verified bead record against master during PR #3260 work: consistent; still-open items (explicit-approval attention categories, live offline-recovery proof) unchanged — blocked on missing data model / live browser respectively.\n2026-07-27 explicit-approval data model slice (PR #3329, branch feature/extension/explicit-approval-attention): implements ONE of the two remaining scope items named in the prior note -- \"explicit-approval/destructive-conflict attention categories once a backing data model exists\". The \"live offline-recovery proof\" item is untouched (still needs a live browser fixture).\n\nRoot cause confirmed by reading dispatchBrowserAction in background.js: a submit_once conversation.reply action was leased and executed by the poll loop with zero operator gate, even though it posts one real, provider-visible turn into an EXISTING conversation with no automatic undo -- exactly the \"explicit submit/destructive approval\" AC3 category that had no backing data model.\n\nWhat shipped: BrowserActionStatus gains \"awaiting_approval\"; BrowserActionIntent gains requires_operator_approval/approval_reason/approval_requested_at/approved_at/approved_by/declined_at (polylogue/browser_capture/models.py). enqueue_action holds submit_once+conversation.reply at \"awaiting_approval\" instead of \"queued\" so claim_action can never lease it; new decide_action_approval records approve (-\u003e queued, claimable for the first time) or decline (-\u003e cancelled, terminal); a new POST /v1/browser-actions/{id}/approval route (polylogue/browser_capture/actions.py, server.py, route_contracts.py). computeAttention in operator_status.js gets a new explicit_approval_required branch ranked between action_outcome_unknown and capability_mismatch per AC3's stated order. popup.html/popup.js add a \"Browser action approval\" panel inside the existing progressive-disclosure diagnostics (no AC4 duplication) with Approve/Decline buttons; the attention item's single primary action (\"Review request\") opens diagnostics and focuses the panel -- never a silent auto-resolution. background.js adds decideBrowserActionApproval, wiring the poll loop to wake only on approve.\n\nDeliberately NOT modeled: a \"destructive_conflict\" reason. claim_action already serializes to exactly one in-flight action at a time, so there is no genuine two-actions-racing-for-one-resource scenario in the current architecture to attach a conflict decision to -- adding an enum value with no real trigger would be unbacked/unfireable. Documented as an explicit gap in models.py for a future bead once a real collision scenario exists (e.g. multi-instance orchestration). So AC3's \"explicit submit/destructive approval\" is now satisfied; \"destructive conflict\" remains open, this time for a concrete architectural reason rather than a missing data model.\n\nVerified: devtools test tests/unit/browser_capture/ 153 passed (was 143, +10 new); mypy clean on the 4 touched browser_capture files; devtools verify --quick exit 0; browser-extension npx vitest run 354/355 passed (1 pre-existing unrelated build.mjs failure, confirmed identical on origin/master via git stash before this change); npm run lint + validate clean. NOT verified: visual rendering of the new diagnostics panel and an end-to-end trigger-to-popup-display proof against a real ChatGPT session -- needs a live browser, out of scope for this task per instruction.\n\nRemaining for full yyvg.7 closure: destructive_conflict category (blocked on a real trigger scenario not yet existing), live offline-recovery proof (blocked on a live browser fixture, per the prior open incident on polylogue-x2q3's self-heal behavior).\nREFERENCE CORRECTION 2026-07-28: this bead cites 'polylogue-x2q3s token-path bug'. No such bead exists, and unlike the other X2 findings this one IS bead-shaped -- it is the only genuine dangling reference among the six the hygiene check reports. Either the id is mistyped or the bead was never filed; the underlying defect (capture token-path collision landing content in the production archive) needs a real id before this note can be relied on.","status":"in_progress","priority":0,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-16T04:44:54Z","created_by":"Sinity","updated_at":"2026-07-28T20:05:31Z","started_at":"2026-07-16T04:45:12Z","labels":["area:capture","area:web","delivery:L-external-legibility","horizon:frontier","horizon:mid","lane:docs-demos-launch"],"dependencies":[{"issue_id":"polylogue-yyvg.7","depends_on_id":"polylogue-yyvg","type":"parent-child","created_at":"2026-07-16T06:44:53Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-hjpx.1","title":"Separate raw census from immutable replay-plan conservation","description":"The first hjpx implementation phase makes selected replay components fair and fault-isolated, but adversarial review proved that a plan cannot yet be conserved end-to-end because ordinary historical raws may reveal their logical keys only while execution is already parsing them. A pre-parse singleton selection can therefore widen into a multi-path authority cohort after its plan id and batch slot were assigned. The current receipt covers selected attempt tokens, not a full immutable before/after plan census; it also lacks logical-key/authority witnesses and cannot prove two-pass fixed point. This is the remaining correctness kernel of hjpx, not a reason to discard the safe batching foundation.","design":"Make census/classification a distinct, bounded, resumable stage that persists enough source-tier evidence to compute the complete transitive authority graph before replay planning. Only after the census is quiescent may the planner assign RawReplayPlanId over immutable component inputs, logical keys, authority witnesses, and source/index preconditions. Persist a canonical before/after census digest and conservation ledger in the authority-owning durable tier (source.db via additive migration and backup-manifest rules), not disposable ops events. Every before plan is executed, carried forward unchanged, or terminalized exactly once; unselected plans appear as carried-forward inventory without creating unbounded per-tick event payloads. Rejected-stale must atomically persist a fail-closed blocker before observational emission. Fixed point requires two consecutive quiescent digests with zero executable plans and identical typed residual debt. Keep daemon events as projections of the durable ledger. Reuse the fair component scheduler and per-component fault isolation from the foundation phase.","acceptance_criteria":"1. A moved-path fixture starts with an uncensused singleton that shares a logical source with prior history; bounded census completes first, and preview/apply use the same immutable multi-raw plan id and inputs. 2. The full before/after plan-ID census satisfies an executable algebra: every before id has exactly one executed, retryable, deferred, terminal, rejected-stale, or carried-forward state; dropping an unselected or expanded component fails. 3. Plan records include logical keys, authority witnesses, input raw ids, source/index preconditions, and exact application/membership receipts; parsed_at_ms alone cannot prove execution. 4. Census interruption resumes without duplicate plans or partial-plan visibility; replay never mutates a component whose census is incomplete. 5. Rejected-stale atomically writes durable source-tier fail-closed debt before any event, and automatic convergence refuses further mutation until an explicit repair resolves it. 6. Two consecutive quiescent dry-run census digests are required for fixed point; one empty pass, candidate count alone, or disposable ops state cannot satisfy it. 7. CLI/MCP/daemon receipts expose bounded inventory counts plus digest/query handles so the complete ledger is queryable without emitting thousands of full outcomes every tick. 8. Regression mutations to path closure, logical-key closure, batch slicing, carried-forward accounting, application receipt checks, or the second-census requirement fail.","notes":"2026-07-16 implementation pass: owning the coherent lkrc/hjpx.1/lkrc.4 raw-authority cluster from fresh origin/master. Scope is the single reconciler/immutable-plan conservation and the production multi-session divergence regression now observed in packaged ordinary catch-up. Preserve yla8 fail-closed replay protections; no live cursor reset, force replay, evidence deletion, manual SQL repair, or live apply before reviewed code, verified backup, quiescent census, and explicit authorization. First deliverable is a production-route failing fixture and read-only live evidence.\n2026-07-16 closure: PR #2961 merged as 593ef3c62 after five independent adversarial passes. Durable source-v13 parser census and immutable replay-plan ledgers now conserve every selected/unselected plan through exact typed outcomes, fail stale plans closed behind explicit blockers, recover interrupted applications only from exact application/head/session/hash witnesses, require two quiescent identity-sensitive censuses for fixed point, and expose bounded census/detail surfaces with digest-bound continuations. Verification at final head: raw-authority ledger 19 passed; raw_materialization selector 82 passed; devtools verify --quick all 16 steps green (20260716T223845Z-quick-1110416-7c9a8554). No live archive mutation was performed; yla8 remains the verified-backup/operator-authorized live gate.","status":"closed","priority":0,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T22:20:17Z","created_by":"Sinity","updated_at":"2026-07-16T22:41:51Z","started_at":"2026-07-16T19:20:46Z","closed_at":"2026-07-16T22:41:51Z","close_reason":"Merged PR #2961 (593ef3c62): durable immutable raw-authority census, conservation, exact receipts, blockers, and two-pass fixed point; focused and quick gates green.","labels":["area:sources","area:storage","area:test","delivery:A-trust-floor","fixed-point","horizon:frontier","invariant","raw-authority"],"dependencies":[{"issue_id":"polylogue-hjpx.1","depends_on_id":"polylogue-hjpx","type":"parent-child","created_at":"2026-07-16T00:20:16Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6c9b-815e-79b4-b70c-1789c02f40fc","issue_id":"polylogue-hjpx.1","author":"Sinity","text":"2026-07-16 implementation checkpoint on feature/storage/raw-authority-ledger: source schema v13 adds atomic raw_authority_censuses/plans/census_plans/blockers; source parser census is separated from index application; v2 plan IDs bind raw inputs, logical keys, authority witness, and source/index preconditions; all inventory plans receive executed/retryable/deferred/terminal/rejected-stale/carried-forward outcomes; unselected plans are conserved; fairness reads source.db rather than ops.db; stale validation writes a durable blocker before output and stops automatic replay; application receipts include membership and index application rows; two same-scope zero-executable censuses with identical inventory/residual digests establish fixed point; daemon and readiness surfaces expose census digest/query handles. Production-route fixtures cover moved-path widening with preview/apply identity, ops reset fairness, stale blocker, and two-pass fixed point. Verification: raw_materialization 81 passed; source/daemon/readiness selection 76 passed; raw_authority 6 passed; quick gate 16/16. No live archive mutation performed.","created_at":"2026-07-16T20:25:58Z"}],"dependency_count":0,"dependent_count":1,"comment_count":1} {"_type":"issue","id":"polylogue-yyvg.5","title":"Queue authenticated GPT-5.6 Sol Pro Chat work packages","description":"Polylogue needs a first-class extension workflow that turns prepared project work packages into ordinary authenticated ChatGPT Chat conversations using GPT-5.6 Sol Pro. This is the near-term capacity lane for analyses, research, patches, prototypes, and cohesive handoff archives that do not require live repository execution. Jobs must run in background tabs without changing operator focus, survive MV3/browser restarts, coordinate across live and agent-private extension instances, respect provider rate/safety limits, and make every phase legible and controllable in the existing mission-control UI. Work mode or Codex-backed surfaces are forbidden because they consume the wrong quota.","design":"Extend the existing receiver/extension job-control architecture rather than add an independent automation script. The receiver is authoritative for LaunchJob identity, FIFO position/manual priority, schedule, immutable inputs, submission lease, monitor ownership, receipts, and event history; browser storage is cache/checkpoint only. An extension instance owns explicit inactive ChatGPT page targets. The rate-sensitive upload/preflight/submit critical section has one cross-instance lease, while already-submitted chats continue in parallel so an initial burst can start at one-minute intervals without waiting hours for prior answers. Expired monitor leases are explicitly adopted without resubmission.\n\nA narrowly typed ChatGPT page adapter uses the ordinary authenticated Chat frontend, native file input, and normal submit control. Immediately before submit it must prove mode=Chat, model=GPT-5.6 Sol, effort=Pro; ambiguity, fallback, Work/Codex route, auth challenge, protocol drift, or changed selection fails closed. Queue profiles support 1/5/15/30/60 minutes: one-minute burst, ordinary five/fifteen-minute pacing, and thirty/sixty-minute steady state. Manual Launch now bypasses ordinary cadence and queue order but never not-before, provider Retry-After, rate, or conversation-safety circuits. 429/Retry-After, too-many-requests safety lock, 403/challenge, network errors, and protocol mismatch have typed outcomes, bounded exponential jittered backoff, a global provider circuit, and no automatic duplicate conversation after submit uncertainty.\n\nThe popup extends mission control with the full queue, phase, exact target assertion, cadence/cooldown, monitor/submission owner, last receipt/error, and pause/resume/cancel/retry/inspect/Launch-now controls. Multiple extension profiles are ordinary replaceable clients with session-scoped executor identities; no code knows “user browser” versus “agent browser.”\n\nThe default input builder produces a deterministic targeted project tarball containing the exact full prompt/output contract, selected Beads records with notes/dependencies, repository instructions, git revision/status and selected-footprint patches, selected source files, optional verification receipts, manifest/checksums, and size report. A full tracked/unignored worktree tar is an explicit size-bounded fallback, never the default. Completion requires authenticated acquisition into the normal browser-capture asset path and local validation of one exact cohesive handoff ZIP (manifest/checksums, summary, design, patches, tests, verification limits). The captured conversation and artifact carry LaunchJob provenance.\n","acceptance_criteria":"1. An operator can enqueue one or more deterministic project work packages at 1, 5, 15, 30, or 60 minute cadence, select any queued job, and use Launch now out of order; queue state persists across receiver/service-worker/browser restart.\n2. Exactly one upload/preflight/submit critical section is leased across two extension instances, but submitted chats continue in parallel and monitor leases can be adopted without resubmission; a one-minute burst starts successive chats without waiting for prior answers.\n3. Each inactive background tab uploads attachments and submits only after an observable preflight proves ordinary Chat + GPT-5.6 Sol + Pro. Work, Codex, fallback/default ambiguity, auth challenge, changed selection, or frontend drift fails closed before submit and never activates the tab.\n4. 429/Retry-After, too-many-requests conversation safety lock, 403/challenge, transient network failure, submit uncertainty, and protocol drift produce typed visible states, bounded jittered backoff/global circuits, resumable monitoring, and no hot loop or duplicate conversation. No manual control bypasses an active rate/safety circuit.\n5. Popup mission control shows the queue, current phase, target/model/effort, cadence/cooldown, owner instance, last receipt/error, and pause/resume/cancel/retry/Launch-now/inspect controls while capture features remain usable.\n6. The targeted pack builder includes exact prompt/output contract, full selected Beads records, instructions, git/worktree state and selected-footprint patch, selected source, optional verification evidence, deterministic manifest/checksums, and size report. Full-worktree inclusion is explicit and size-bounded.\n7. A completed job is not successful until the exact cohesive handoff archive is acquired locally through authenticated capture and validated for safe paths, manifest sizes/checksums, summary, design, patches, tests, and verification limits; the chat/capture/artifact link back to LaunchJob.\n8. Deterministic tests cover FIFO/manual priority, cross-instance leases, parallel submitted chats, cadence/backoff/global circuits, no post-submit duplicate, exact model/mode fail-closed preflight, upload/submit fixtures, popup controls, deterministic pack selection, and artifact validation. A live authenticated smoke records one harmless background Chat · GPT-5.6 Sol · Pro launch and capture without foreground activation.","notes":"2026-07-16 architecture correction: the landed Sol-specific LaunchJob queue is a working campaign prototype, not the target product abstraction. Do not extend it with mission/deliverable/package domain objects. Generalize its proven transport invariants (receiver authority, leases, owned inactive tab, submit-intent ambiguity, typed provider errors, attachment upload, exact selection receipts) into ptx BrowserActionIntent, migrate the current campaign to external yyvg.6 tooling, then remove LaunchJob/prompt/handoff/cadence semantics from extension/receiver and popup. Ordinary canonical capture remains the response/file path.\n2026-07-16 implementation now removes this Sol-specific product queue and all campaign/prompt/package/handoff/cadence UI/routes from browser_capture and the extension, replacing only the reusable transport invariants with ptx. After the replacement PR merges, this bead should close as architecturally superseded by ptx (generic product conduit) plus yyvg.6 (external private campaign orchestrator), not as satisfaction of its obsolete extension-owned AC.\n2026-07-16 GPT-Pro corpus adjudication: implementation-grade handoff 8f37aa16b083c357c32b426d44379c96ef49acd692f7b569b2d5f4d8fc8470fd is already_subsumed. Its receiver authority/terminal-capture observations were generalized through PRs #2913, #2918, #2919, #2926 and #2928; no campaign-specific product path is revived.","status":"closed","priority":0,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T19:21:14Z","created_by":"Sinity","updated_at":"2026-07-16T13:04:54Z","started_at":"2026-07-15T19:21:26Z","closed_at":"2026-07-16T07:58:35Z","close_reason":"Superseded by merged PR #2928: generic BrowserAction transport now belongs to polylogue-ptx, while private campaign cadence/packages/integration belong to external orchestrator polylogue-yyvg.6. The extension-owned Sol queue and its obsolete AC were removed rather than declared satisfied.","metadata":{"frontier":"active","frontier_program_ref":"polylogue-yyvg"},"labels":["area:capture","area:web","delivery:L-external-legibility","horizon:frontier","lane:docs-demos-launch"],"dependencies":[{"issue_id":"polylogue-yyvg.5","depends_on_id":"polylogue-06zm","type":"relates-to","created_at":"2026-07-15T21:21:24Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yyvg.5","depends_on_id":"polylogue-b1n","type":"relates-to","created_at":"2026-07-15T21:21:24Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yyvg.5","depends_on_id":"polylogue-jlme.1","type":"relates-to","created_at":"2026-07-15T21:21:25Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yyvg.5","depends_on_id":"polylogue-jlme.6","type":"blocks","created_at":"2026-07-15T21:21:48Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yyvg.5","depends_on_id":"polylogue-ptx","type":"relates-to","created_at":"2026-07-15T21:21:23Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yyvg.5","depends_on_id":"polylogue-yqof","type":"relates-to","created_at":"2026-07-15T21:21:23Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yyvg.5","depends_on_id":"polylogue-yyvg","type":"parent-child","created_at":"2026-07-15T21:21:14Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-jlme.6","title":"Restore browser-extension popup execution and receiver health","description":"The installed MV3 popup is completely inert: popup.js fails at parse time because it redeclares operatorPresentationForState and operatorStatusForState already declared by operator_status.js in the same classic-script global lexical scope. The toolbar badge remains stale off, receiver re-pairing cannot save, status and backfill controls never execute, and no error is visible in the popup. Live CDP proof on 2026-07-15 captured SyntaxError: Identifier operatorPresentationForState has already been declared.","design":"Keep operator_status.js as the one status-vocabulary module exposed through globalThis.PolylogueOperatorStatus. popup.js must consume that namespace without introducing same-scope lexical bindings that collide with classic-script declarations. Add an executable popup smoke that loads operator_status.js followed by popup.js, exercises initial render and a runtime-message action, and fails on parse/startup errors. Receiver health must probe a supported authenticated endpoint/contract rather than treating a 404 as a healthy-but-error receiver.","acceptance_criteria":"1. Loading popup.html executes popup.js with no parse/runtime startup error. 2. The popup reaches a non-checking state, Save persists receiver URL/token through the background worker, and Check receiver reports the canonical receiver healthy. 3. Toolbar state refreshes from the repaired receiver instead of remaining stale off. 4. A regression smoke evaluates the two scripts in real load order and would fail on the current duplicate declaration. 5. Focused browser-extension tests pass and a live installed-extension smoke is recorded.","notes":"Dependency repair 2026-07-15: removed the reverse hard edge to yyvg.5. Popup execution/receiver health is independently repairable and is the prerequisite for the queued-work UI; the relation preserves incident context without a cycle.\nCorrection to the preceding dependency note: no separate relates-to edge was added because the retained one-way yyvg.5→jlme.6 hard dependency already preserves the cross-item context. The cycle is gone.\n2026-07-15 live proof: Chrome CDP captured the popup parse failure before the fix. After commit 8868f8139 and unpacked-extension reload, the popup left checking state, listed two supported ChatGPT tabs, authenticated GET /v1/status reported Receiver health OK, and Sync open tabs produced a native_full 66-turn, 1,970,575-byte spool artifact for chatgpt session 6a54dd7c-756c-83eb-88b6-66cc8f61f0d4. Receiver archive-state reports stale/spooled=true/raw_row_exists=true/indexed_message_count=2 while the daemon catches up. Focused npm test: 79/79 passed across popup.test.js and background.test.js. npm ci initially exposed inherited ignored-lockfile drift for fake-indexeddb; npm install populated the declared dev dependency without tracked lockfile changes.\n2026-07-15 live post-fix receipt: the unpacked extension reloads under id gkkpfbaioajmnjfkclplnpifncnonjpc, popup executes and renders the receiver-owned eight-job Sol Pro queue, launch is enabled against authenticated receiver http://127.0.0.1:8876, and the first corrected inactive-tab submission reached a real ChatGPT conversation. Full extension suite is 249/249 and npm run lint is clean. Leave closure until the feature branch is merged so the durable receipt and popup fix land together.\n2026-07-16 q32 mission-control incorporation: integrated shared operator vocabulary, popup work queue, ambient closed-Shadow-DOM surface, stable receiver identity/pairing and bounded canonical recovery, offline last-known launch presentation, explicit observed-no-action events, and a visible extension contract epoch canonical-capture-mission-control-v1. This makes stale/legacy loaded source diagnosable instead of relying on the ambiguous toolbar badge alone.\n2026-07-16 live installed-extension proof after current-source reload: popup advertises extension contract canonical-capture-mission-control-v1, stable receiver rx-e328a27cc0d16cfbac83 at 8876, renders the 32-row receiver queue, and recovered q32 plus four legacy completion rows through canonical capture without foreground activation. Full extension suite now passes 290 tests and lint is clean.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T19:20:39Z","created_by":"Sinity","updated_at":"2026-07-16T04:42:35Z","started_at":"2026-07-15T19:21:26Z","closed_at":"2026-07-16T04:42:35Z","close_reason":"Merged PR #2926 as 54e8911b9. Popup load-order execution, authenticated receiver health/pairing, toolbar refresh, executable regression fixtures, installed-extension proof, and live canonical recovery satisfy all five acceptance criteria. Full extension suite: 296 passed; lint clean; quick verification all 16 steps.","metadata":{"frontier":"active","frontier_program_ref":"polylogue-jlme"},"labels":["area:capture","area:ingest","delivery:G-live-performance","horizon:frontier","lane:capture-reliability","spine"],"dependencies":[{"issue_id":"polylogue-jlme.6","depends_on_id":"polylogue-jlme","type":"parent-child","created_at":"2026-07-15T21:20:38Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-jlme.6","depends_on_id":"polylogue-yyvg.5","type":"relates-to","created_at":"2026-07-15T21:24:15Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} -{"_type":"issue","id":"polylogue-hjpx","title":"Make RawAuthorityReconciler execute accepted replay plans to fixed point","description":"The raw-authority repair path can classify a retained revision as replayable yet produce no executable logical source: repair_raw_materialization reports a scanned/classified full raw, backfill_historical_revision_evidence returns replayed_logical_sources=0 and success=false, and the same candidate remains forever. This is the execution-completeness slice of lkrc, not six unrelated test failures. Any plan the reconciler accepts must either execute exactly once or end in an explicit typed deferred/terminal state with a receipt.","design":"Repair the exact production route anchored at polylogue/storage/repair.py repair_raw_materialization around lines 6135-6385, polylogue/sources/revision_backfill.py backfill_historical_revision_evidence around lines 42-185, and polylogue/daemon/cli.py _drain_raw_materialization_once around lines 611-640. Introduce a stable RawReplayPlanId for every selected authority component before the raw_artifact_limit slice. Carry it through membership expansion, raw_revision_rebuild_selection, cohort classification, resource admission, adoptability, apply, remaining-candidate recomputation, and status receipts. Replace the current aggregate-only RevisionBackfillResult with per-plan outcomes executed, retryable, deferred, terminal, or rejected-stale, each with input raw ids, logical key, authority witness, reason, and next action. The daemon batch selector must choose bounded independent authority components fairly, not the smallest individual raw ids that repeatedly expand into the same few components. A plan present in the before census must appear exactly once in the pass receipt and in the after census unless executed/terminal; selected plans cannot vanish when logical_keys is empty, membership keys differ, a component expands, or adoptability defers. Fixed-point means two successive quiescent dry-run censuses have identical typed non-executable debt and zero executable accepted plans, not merely repaired_count less than the batch limit. Preserve CAS, source authority, atomic rollback, component byte limit, and FTS closure. Do not raise the batch limit, reset cursors, replay every candidate, or bypass the reconciler.","acceptance_criteria":"1. A production-shaped fixture reproduces a selected/classified raw yielding replayed_logical_sources=0 and remaining executable debt on current behavior; the fixed route emits one plan receipt and executes or explicitly defers/terminalizes it. 2. Every before-census RawReplayPlanId is conserved into exactly one per-pass outcome; counters reconcile selected components, expanded raws, logical keys, execution, retry/defer/terminal outcomes, and after-census debt. Mutations dropping logical_keys, membership keys, or expanded components fail. 3. Bounded scheduling is by independent authority component with stable fairness/age, so a finite fixture with more than one batch drains every executable component; repeatedly selecting the same cheap components or starving large valid work fails. 4. Transient lock/resource interruption remains retryable with the same plan id and later succeeds once; CAS conflict or incomparable authority remains typed, durable, and non-mutating. 5. Two successive quiescent dry-run passes establish fixed point only when executable accepted-plan count is zero and all residual rows are explicit durable non-executable states. Candidate count alone cannot claim convergence. 6. A sanitized scale fixture matching the 2026-07-15 shape proves bounded memory/temp/time, monotonic executable backlog decrease, no cursor/head regression, and responsive daemon health; removing fair selection or outcome conservation recreates non-progress. 7. Run devtools test tests/unit/sources/test_revision_backfill.py; devtools test -k raw_materialization; devtools test -k raw_authority; and devtools verify. Before any live apply, record current build/schema, stopped-daemon census, verified backup, dry-run plan inventory, resource envelope, and explicit operator authorization.","notes":"2026-07-15 formulation correction: replaced the symptom/test-count title with the missing reconciler invariant. This remains a necessary lkrc execution slice; no regression or safety condition was removed.\nActive-set expansion 2026-07-15: admitted as a high-leverage operational mechanism under the scale/raw-authority program; execution focus remains readiness- and conflict-aware.\nPriority escalation 2026-07-15: P1 to P0 after the yla8 authorization preflight found the installed daemon already replaying 2 sources per bounded pass while remaining candidates monotonically grew 11,717 to 15,264, with 1,890 broken active seeds and 40 cursor-ahead rows. This is the executable successor to the failed live gate, not a second raw-authority architecture. No live apply is authorized.\n2026-07-15 execution-foundation phase implemented in isolated branch feature/fix/raw-replay-fixed-point. Evidence-first repro: 5 same-session prefix revisions plus 4 independent raws produced 9 candidates/5 components; the old raw-row limit selected only one cohort and replayed 0 logical sources indefinitely. Phase changes: provisional single-session raws enter typed full-revision classification; non-prefix full cohorts convert to semantic membership governance; component closure includes source paths, membership keys, and raw_revision logical keys; batch budget counts complete components; ops receipts rotate attempted retryable plans behind never-attempted work; resource-blocked and executable plans share one budget; independent components execute/fail in isolation; selected attempts produce stable IDs and typed outcomes; daemon emits zero-work and nonzero pass receipts. Adversarial iterations 1-2 found that full hjpx closure still requires pre-execution immutable census, complete carried-forward conservation, durable source-tier rejection, two-census fixed-point proof, richer authority witnesses, and scale evidence. Those are now explicit children hjpx.1 (P0 correctness kernel) and hjpx.2 (P1 July-15 scale proof, blocked by hjpx.1). Parent remains in_progress; no live apply authorized or performed. Verification: revision_backfill focused cross-path tests 2 passed; selected storage regressions 4 passed; raw_materialization selector 79 passed before final per-component isolation refactor, followed by its focused 4-test pass; devtools verify --quick green run 20260715T221858Z-quick-1939817-ca04d05e. Full devtools verify cannot bootstrap in a fresh worktree because seed-testmon is unbounded/red/hanging; tracked b054.1.1. raw_authority selector has an inherited clean-master failure tracked lkrc.4.\nFoundation phase merged via PR #2915 as d6501ac4615efa30cb0e2413c97614a4bf44b253. Final post-refactor selector receipt: devtools test -k raw_materialization selected 80 tests and passed all 80 in 106.14s. Automated CodeRabbit review was quota-limited (tool failure/no findings), GitGuardian passed; two independent adversarial iterations are recorded in the PR and residual children. Parent remains in_progress.\n2026-07-16 implementation pass: owning the coherent lkrc/hjpx.1/lkrc.4 raw-authority cluster from fresh origin/master. Scope is the single reconciler/immutable-plan conservation and the production multi-session divergence regression now observed in packaged ordinary catch-up. Preserve yla8 fail-closed replay protections; no live cursor reset, force replay, evidence deletion, manual SQL repair, or live apply before reviewed code, verified backup, quiescent census, and explicit authorization. First deliverable is a production-route failing fixture and read-only live evidence.\n2026-07-17 static follow-up from yla8 read-only preflight: current live execution is blocked by one non-stream-safe authority component over the 1 GiB bounded replay envelope. The implemented P0 kernel remains present: repair_raw_materialization completes parser census before planning, keeps complete authority components intact, persists immutable plan/outcome/postflight conservation, rotates prior attempts fairly, and requires two quiescent dry-run census identities for fixed point. The current resource-envelope/streaming proof is therefore the existing P1 polylogue-hjpx.2 scale lane, not evidence to weaken or bypass P0 authority conservation. No source/index/live mutation was made.\n2026-07-26 portfolio-convergence audit: released stale in_progress claim after \u003e7 days with no recorded activity; scope remains open and must be re-claimed on real work start.\n\n2026-07-27: two concrete execution-completeness gaps found and fixed this session, directly relevant to this bead's fixed-point invariant:\n1. unresolved_raw_replay_blockers counted stale_plan blockers archive-wide, so ONE stuck plan halted repair_materialization for every unrelated raw component - the exact \"accepted plan never executes\" failure mode this bead targets, just at the census/replan layer rather than the accepted-replay layer. Fixed via auto_resolve_stale_plan_blockers (polylogue-d7im, PR #3287, merged+deployed).\n2. Manually resolved a batch of 12 frontier_judgment blockers (6 browser-rekey conversations, byte-level content-hash verification performed directly against the blob store, not rubber-stamped) that had been stuck in conflicting_authority_needs_judgment - all confirmed safe (byte-identical or, for 2 initially flagged as differing, confirmed identical message content via direct payload diff after the automated divergence check hit a FileNotFoundError and correctly punted to manual review). Also found the census regenerates duplicate judgment requests across cycles rather than deduping - filed separately as polylogue-rjtv since it's noise, not a fixed-point violation per se.\n2026-07-27 AC3/AC4 investigation + regression coverage (this session, continuing from the AC1 fix): read the full bead history plus closed children hjpx.1 (PR #2961, correctness kernel) and hjpx.2 (in_progress, blocked on host I/O during July-15 corpus generation) before touching anything, to avoid duplicating landed work.\n\nAC3 (fair bounded scheduling by independent authority component): investigated _raw_materialization_ordered_components in polylogue/storage/repair.py. Production ordering is already size-agnostic (never-attempted-first, then oldest-attempt-time, then acquisition order; byte size only used as a same-component tie-break, never cross-component priority) -- no code gap found. Added test_raw_materialization_ordering_is_size_agnostic_and_does_not_starve_large_work (tests/unit/storage/test_repair.py): one large-but-executable oldest component plus 5 small ones; fair ordering picks the large one first; a cheap-first mutation of the ordering function recreates exactly the starvation AC3 names. Anti-vacuity confirmed by temporarily mutating production code and watching the test fail for the right reason before reverting.\n\nAC4 (transient retry / CAS-conflict typing): investigated the generic exception handler around backfill_historical_revision_evidence (repair.py ~L6517-6560). It already classifies any RuntimeError -- including the CAS-conflict class raised by revision_application.py (\"CAS rejected a conflicting accepted head\" / \"older accepted frontier\" / \"incomparable accepted frontier\") -- into a typed, durably-recorded (raw_authority_census_plans in source.db), non-mutating RETRYABLE outcome carrying the same plan_id. No code gap found, but no existing test proved plan-id stability across a fail-then-succeed retry, or that a genuine CAS-conflict message specifically produces this typed/durable/non-mutating outcome through the full reconciler (existing CAS tests only exercised the low-level revision_application.py function raising in isolation). Added two tests: test_raw_materialization_transient_failure_retries_with_same_plan_id_then_succeeds and test_raw_materialization_cas_conflict_outcome_is_typed_durable_and_non_mutating. Same anti-vacuity method applied (mutated plan_id in the RETRYABLE branch, confirmed failure, reverted).\n\nAC5: no new work this session beyond the AC1 fix's contribution already noted; remains code-complete via hjpx.1 per that bead's closure evidence.\n\nAC6: no new work -- confirmed via hjpx.2's own notes that it remains in_progress, blocked on sustained host I/O pressure across 4 documented self-aborts generating the July-15-shaped corpus (2026-07-18). Not duplicated here; citing hjpx.2 rather than rebuilding its scope per this session's instructions.\n\nAC7: ran all three named commands plus devtools verify --quick (not full/--seed-testmon -- a focused test-only change uses the narrow gate per repo convention; a first attempt at seeding was started and then correctly stopped mid-run as unnecessary scope for this change). devtools test tests/unit/sources/test_revision_backfill.py -\u003e 44 passed; devtools test -k raw_materialization -\u003e 118 passed; devtools test -k raw_authority -\u003e 83 passed; devtools verify --quick -\u003e 17/17 green.\n\nPR #3345 (branch feature/fix/hjpx-ac3-ac4-progress) opened with these 3 new regression tests, +197 lines to tests/unit/storage/test_repair.py only, no production code changes (both AC3 and AC4 were already satisfied). Not merged by this session. Bead remains open: AC6 (scale proof) and AC7's live-apply ceremony items are explicitly out of scope for a coding session per this bead's safety constraint and current instructions.","status":"open","priority":0,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T13:48:19Z","created_by":"Sinity","updated_at":"2026-07-27T18:29:18Z","started_at":"2026-07-15T20:56:16Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:sources","area:storage","area:test","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-hjpx","depends_on_id":"polylogue-lkrc","type":"parent-child","created_at":"2026-07-15T18:44:11Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-hjpx","depends_on_id":"polylogue-yla8","type":"discovered-from","created_at":"2026-07-15T22:54:55Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-hjpx","title":"Make RawAuthorityReconciler execute accepted replay plans to fixed point","description":"The raw-authority repair path can classify a retained revision as replayable yet produce no executable logical source: repair_raw_materialization reports a scanned/classified full raw, backfill_historical_revision_evidence returns replayed_logical_sources=0 and success=false, and the same candidate remains forever. This is the execution-completeness slice of lkrc, not six unrelated test failures. Any plan the reconciler accepts must either execute exactly once or end in an explicit typed deferred/terminal state with a receipt.","design":"Repair the exact production route anchored at polylogue/storage/repair.py repair_raw_materialization around lines 6135-6385, polylogue/sources/revision_backfill.py backfill_historical_revision_evidence around lines 42-185, and polylogue/daemon/cli.py _drain_raw_materialization_once around lines 611-640. Introduce a stable RawReplayPlanId for every selected authority component before the raw_artifact_limit slice. Carry it through membership expansion, raw_revision_rebuild_selection, cohort classification, resource admission, adoptability, apply, remaining-candidate recomputation, and status receipts. Replace the current aggregate-only RevisionBackfillResult with per-plan outcomes executed, retryable, deferred, terminal, or rejected-stale, each with input raw ids, logical key, authority witness, reason, and next action. The daemon batch selector must choose bounded independent authority components fairly, not the smallest individual raw ids that repeatedly expand into the same few components. A plan present in the before census must appear exactly once in the pass receipt and in the after census unless executed/terminal; selected plans cannot vanish when logical_keys is empty, membership keys differ, a component expands, or adoptability defers. Fixed-point means two successive quiescent dry-run censuses have identical typed non-executable debt and zero executable accepted plans, not merely repaired_count less than the batch limit. Preserve CAS, source authority, atomic rollback, component byte limit, and FTS closure. Do not raise the batch limit, reset cursors, replay every candidate, or bypass the reconciler.","acceptance_criteria":"1. A production-shaped fixture reproduces a selected/classified raw yielding replayed_logical_sources=0 and remaining executable debt on current behavior; the fixed route emits one plan receipt and executes or explicitly defers/terminalizes it. 2. Every before-census RawReplayPlanId is conserved into exactly one per-pass outcome; counters reconcile selected components, expanded raws, logical keys, execution, retry/defer/terminal outcomes, and after-census debt. Mutations dropping logical_keys, membership keys, or expanded components fail. 3. Bounded scheduling is by independent authority component with stable fairness/age, so a finite fixture with more than one batch drains every executable component; repeatedly selecting the same cheap components or starving large valid work fails. 4. Transient lock/resource interruption remains retryable with the same plan id and later succeeds once; CAS conflict or incomparable authority remains typed, durable, and non-mutating. 5. Two successive quiescent dry-run passes establish fixed point only when executable accepted-plan count is zero and all residual rows are explicit durable non-executable states. Candidate count alone cannot claim convergence. 6. A sanitized scale fixture matching the 2026-07-15 shape proves bounded memory/temp/time, monotonic executable backlog decrease, no cursor/head regression, and responsive daemon health; removing fair selection or outcome conservation recreates non-progress. 7. Run devtools test tests/unit/sources/test_revision_backfill.py; devtools test -k raw_materialization; devtools test -k raw_authority; and devtools verify. Before any live apply, record current build/schema, stopped-daemon census, verified backup, dry-run plan inventory, resource envelope, and explicit operator authorization.","notes":"2026-07-15 formulation correction: replaced the symptom/test-count title with the missing reconciler invariant. This remains a necessary lkrc execution slice; no regression or safety condition was removed.\nActive-set expansion 2026-07-15: admitted as a high-leverage operational mechanism under the scale/raw-authority program; execution focus remains readiness- and conflict-aware.\nPriority escalation 2026-07-15: P1 to P0 after the yla8 authorization preflight found the installed daemon already replaying 2 sources per bounded pass while remaining candidates monotonically grew 11,717 to 15,264, with 1,890 broken active seeds and 40 cursor-ahead rows. This is the executable successor to the failed live gate, not a second raw-authority architecture. No live apply is authorized.\n2026-07-15 execution-foundation phase implemented in isolated branch feature/fix/raw-replay-fixed-point. Evidence-first repro: 5 same-session prefix revisions plus 4 independent raws produced 9 candidates/5 components; the old raw-row limit selected only one cohort and replayed 0 logical sources indefinitely. Phase changes: provisional single-session raws enter typed full-revision classification; non-prefix full cohorts convert to semantic membership governance; component closure includes source paths, membership keys, and raw_revision logical keys; batch budget counts complete components; ops receipts rotate attempted retryable plans behind never-attempted work; resource-blocked and executable plans share one budget; independent components execute/fail in isolation; selected attempts produce stable IDs and typed outcomes; daemon emits zero-work and nonzero pass receipts. Adversarial iterations 1-2 found that full hjpx closure still requires pre-execution immutable census, complete carried-forward conservation, durable source-tier rejection, two-census fixed-point proof, richer authority witnesses, and scale evidence. Those are now explicit children hjpx.1 (P0 correctness kernel) and hjpx.2 (P1 July-15 scale proof, blocked by hjpx.1). Parent remains in_progress; no live apply authorized or performed. Verification: revision_backfill focused cross-path tests 2 passed; selected storage regressions 4 passed; raw_materialization selector 79 passed before final per-component isolation refactor, followed by its focused 4-test pass; devtools verify --quick green run 20260715T221858Z-quick-1939817-ca04d05e. Full devtools verify cannot bootstrap in a fresh worktree because seed-testmon is unbounded/red/hanging; tracked b054.1.1. raw_authority selector has an inherited clean-master failure tracked lkrc.4.\nFoundation phase merged via PR #2915 as d6501ac4615efa30cb0e2413c97614a4bf44b253. Final post-refactor selector receipt: devtools test -k raw_materialization selected 80 tests and passed all 80 in 106.14s. Automated CodeRabbit review was quota-limited (tool failure/no findings), GitGuardian passed; two independent adversarial iterations are recorded in the PR and residual children. Parent remains in_progress.\n2026-07-16 implementation pass: owning the coherent lkrc/hjpx.1/lkrc.4 raw-authority cluster from fresh origin/master. Scope is the single reconciler/immutable-plan conservation and the production multi-session divergence regression now observed in packaged ordinary catch-up. Preserve yla8 fail-closed replay protections; no live cursor reset, force replay, evidence deletion, manual SQL repair, or live apply before reviewed code, verified backup, quiescent census, and explicit authorization. First deliverable is a production-route failing fixture and read-only live evidence.\n2026-07-17 static follow-up from yla8 read-only preflight: current live execution is blocked by one non-stream-safe authority component over the 1 GiB bounded replay envelope. The implemented P0 kernel remains present: repair_raw_materialization completes parser census before planning, keeps complete authority components intact, persists immutable plan/outcome/postflight conservation, rotates prior attempts fairly, and requires two quiescent dry-run census identities for fixed point. The current resource-envelope/streaming proof is therefore the existing P1 polylogue-hjpx.2 scale lane, not evidence to weaken or bypass P0 authority conservation. No source/index/live mutation was made.\n2026-07-26 portfolio-convergence audit: released stale in_progress claim after \u003e7 days with no recorded activity; scope remains open and must be re-claimed on real work start.\n\n2026-07-27: two concrete execution-completeness gaps found and fixed this session, directly relevant to this bead's fixed-point invariant:\n1. unresolved_raw_replay_blockers counted stale_plan blockers archive-wide, so ONE stuck plan halted repair_materialization for every unrelated raw component - the exact \"accepted plan never executes\" failure mode this bead targets, just at the census/replan layer rather than the accepted-replay layer. Fixed via auto_resolve_stale_plan_blockers (polylogue-d7im, PR #3287, merged+deployed).\n2. Manually resolved a batch of 12 frontier_judgment blockers (6 browser-rekey conversations, byte-level content-hash verification performed directly against the blob store, not rubber-stamped) that had been stuck in conflicting_authority_needs_judgment - all confirmed safe (byte-identical or, for 2 initially flagged as differing, confirmed identical message content via direct payload diff after the automated divergence check hit a FileNotFoundError and correctly punted to manual review). Also found the census regenerates duplicate judgment requests across cycles rather than deduping - filed separately as polylogue-rjtv since it's noise, not a fixed-point violation per se.\n2026-07-27 AC3/AC4 investigation + regression coverage (this session, continuing from the AC1 fix): read the full bead history plus closed children hjpx.1 (PR #2961, correctness kernel) and hjpx.2 (in_progress, blocked on host I/O during July-15 corpus generation) before touching anything, to avoid duplicating landed work.\n\nAC3 (fair bounded scheduling by independent authority component): investigated _raw_materialization_ordered_components in polylogue/storage/repair.py. Production ordering is already size-agnostic (never-attempted-first, then oldest-attempt-time, then acquisition order; byte size only used as a same-component tie-break, never cross-component priority) -- no code gap found. Added test_raw_materialization_ordering_is_size_agnostic_and_does_not_starve_large_work (tests/unit/storage/test_repair.py): one large-but-executable oldest component plus 5 small ones; fair ordering picks the large one first; a cheap-first mutation of the ordering function recreates exactly the starvation AC3 names. Anti-vacuity confirmed by temporarily mutating production code and watching the test fail for the right reason before reverting.\n\nAC4 (transient retry / CAS-conflict typing): investigated the generic exception handler around backfill_historical_revision_evidence (repair.py ~L6517-6560). It already classifies any RuntimeError -- including the CAS-conflict class raised by revision_application.py (\"CAS rejected a conflicting accepted head\" / \"older accepted frontier\" / \"incomparable accepted frontier\") -- into a typed, durably-recorded (raw_authority_census_plans in source.db), non-mutating RETRYABLE outcome carrying the same plan_id. No code gap found, but no existing test proved plan-id stability across a fail-then-succeed retry, or that a genuine CAS-conflict message specifically produces this typed/durable/non-mutating outcome through the full reconciler (existing CAS tests only exercised the low-level revision_application.py function raising in isolation). Added two tests: test_raw_materialization_transient_failure_retries_with_same_plan_id_then_succeeds and test_raw_materialization_cas_conflict_outcome_is_typed_durable_and_non_mutating. Same anti-vacuity method applied (mutated plan_id in the RETRYABLE branch, confirmed failure, reverted).\n\nAC5: no new work this session beyond the AC1 fix's contribution already noted; remains code-complete via hjpx.1 per that bead's closure evidence.\n\nAC6: no new work -- confirmed via hjpx.2's own notes that it remains in_progress, blocked on sustained host I/O pressure across 4 documented self-aborts generating the July-15-shaped corpus (2026-07-18). Not duplicated here; citing hjpx.2 rather than rebuilding its scope per this session's instructions.\n\nAC7: ran all three named commands plus devtools verify --quick (not full/--seed-testmon -- a focused test-only change uses the narrow gate per repo convention; a first attempt at seeding was started and then correctly stopped mid-run as unnecessary scope for this change). devtools test tests/unit/sources/test_revision_backfill.py -\u003e 44 passed; devtools test -k raw_materialization -\u003e 118 passed; devtools test -k raw_authority -\u003e 83 passed; devtools verify --quick -\u003e 17/17 green.\n\nPR #3345 (branch feature/fix/hjpx-ac3-ac4-progress) opened with these 3 new regression tests, +197 lines to tests/unit/storage/test_repair.py only, no production code changes (both AC3 and AC4 were already satisfied). Not merged by this session. Bead remains open: AC6 (scale proof) and AC7's live-apply ceremony items are explicitly out of scope for a coding session per this bead's safety constraint and current instructions.\n2026-07-28: the standing 'No live apply is authorized' note in this bead is a per-session prohibition, not a permanent one, and it is currently the reason agents defer the whole P0 raw-authority cluster. The single operator decision that lifts it, plus the agent-side prerequisites that must be reported before asking, are written out once on polylogue-yla8 -- read that note rather than re-deriving the ask.","status":"open","priority":0,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T13:48:19Z","created_by":"Sinity","updated_at":"2026-07-28T20:04:38Z","started_at":"2026-07-15T20:56:16Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:sources","area:storage","area:test","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-hjpx","depends_on_id":"polylogue-lkrc","type":"parent-child","created_at":"2026-07-15T18:44:11Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-hjpx","depends_on_id":"polylogue-yla8","type":"discovered-from","created_at":"2026-07-15T22:54:55Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-866e","title":"Make lineage writes order-independent and branch-point safe","description":"Stateful property testing has found three production-shaped lineage transition failures on clean master. Repeated full replacement with sibling variants plus child-before-parent ingestion can retain older sibling text instead of the newest primary content; deleting a referenced parent branch point can leave a child whose modeled prefix exceeds the surviving parent and crash composition. The invariant is broader than three examples: equivalent lineage histories must converge to the same logical transcript regardless of ingestion/replacement order, and missing branch points must degrade through a typed readable relation state.","design":"Run the evidence harness against the saved Hypothesis examples before editing production. The write transition authority is polylogue/storage/sqlite/archive_tiers/write.py: full-replace normalization around lines 235-455, batch link normalization around 1858-1889, composed-signature/prefix alignment around 3167-3454, and stale branch-point repair around 3478-3594. Link resolution is polylogue/storage/sqlite/queries/session_links.py around 175-341; the independent read oracle is polylogue/storage/sqlite/queries/message_query_reads.py around 74-235; the state machine is tests/property/test_write_path_state_machine.py, especially rules around 202-276 and invariants around 405-445. Reduce each saved sequence to a deterministic fixture and compare three independently observed states: physical messages ordered by (position, variant_index, message_id), session_links transition row, and composed logical transcript/completeness. The settled invariant is one atomic lineage normalization after every accepted full replacement or link-resolution transition: recompute against the parent composed signature, retain the newest accepted sibling variants, and either bind an existing branch point or set a typed unresolved/repaired/quarantined state. A missing branch point may truncate with LineageCompleteness(dangling_branch_point) but must never preserve an impossible prefix length or substitute other content. Fix the common writer/link transition if the reduced traces implicate it; correct the property model only when the direct SQL plus composed-read oracle proves it wrong. Do not add read-only exception handling as the primary fix, weaken examples, or introduce provider-specific repair.","acceptance_criteria":"1. POLYLOGUE_HYPOTHESIS_REUSE_FAILURES=1 devtools test tests/property/test_write_path_state_machine.py reproduces every saved class on the pre-fix baseline; each is committed as a named deterministic transition fixture with physical-row, link-row, and composed-read oracle. 2. Repeated full replacement with sibling variants retains the newest accepted identity/text/order in physical tails and composed transcript; mutating variant selection or cache invalidation to the prior behavior fails. 3. Child-before-parent, parent-before-child, and later parent replacement converge to identical physical tail, session_links state, logical transcript, and completeness. 4. Deleting or replacing a referenced branch point atomically rebinds when exactly provable; otherwise it yields the declared typed unresolved/repaired/quarantined edge and LineageCompleteness dangling state, with bounded readable child content and no IndexError, stale prefix, or substituted message. 5. Crash/rollback between message replacement, link resolution, normalization, and repair cannot commit a mixed state; retry converges idempotently. 6. No property assertion or generator is weakened unless the direct SQL and composed-read oracle demonstrates model error and the correction retains a mutation-sensitive production invariant. 7. Run devtools test tests/property/test_write_path_state_machine.py; devtools test -k session_links; devtools test -k composed; and devtools verify. Record saved-example ids, fresh random seed, exact counts, and the production mutation each regression catches.","notes":"Active-frontier correction 2026-07-15: admitted as the sole executable leaf of the lineage program. Deterministic falsifying examples make leaving it outside scheduling incompatible with P0.\n2026-07-15 formulation correction: renamed from the Hypothesis symptom to the lineage state-machine invariant. The three saved falsifiers remain permanent anti-vacuity fixtures under the unchanged P0 scope.\nTerra-readiness correction 2026-07-15: anchored the exact writer/link/reader/state-machine surfaces, fixed the atomic transition invariant and permitted degraded state, and made this an evidence-first bug-fix packet rather than an open architectural exercise.\n2026-07-16 GPT-Pro corpus adjudication: lineage package 5e2363a19a64 is merged as PR #2922 (b55f3fd9697083d44466613091604a21c7324ae6). Package contribution is canonical sibling-variant ordering and missing-branch-cut safety; remaining broader lineage AC stays under this owner.\n2026-07-17 implementation-readiness audit: #2922 / b55f3fd resolved canonical sibling-variant ordering and direct missing-branch-cut safety. Current authoritative anchors are write._replace_full_session_messages_and_blocks (1375), _composed_db_signatures (3273), _replacement_for_stale_prefix_branch_point (3512), repair_stale_prefix_branch_points (3626); session_links.resolve_session_links_for_session (175); composed read ordering in message_query_reads.py; state-machine rules at tests/property/test_write_path_state_machine.py:204-291 and invariants 383-445. Do not redo the merged slice. The remaining known falsifier is nested dangling-ancestor completeness propagation; PR #2922 also leaves persisted semantic-cut witnesses and crash/rollback fault injection. Reduce that saved sequence into a named deterministic fixture before changing writer code; then make a single transaction-level repair that proves physical rows, link state, composed transcript, and LineageCompleteness converge across child-first/parent-first/replacement/retry. Direct branch-point absence must retain bounded owned tail, never substitute a sibling or fabricate prefix.\n2026-07-17 direct implementation attempt/audit on current master 3b217d63: no production patch made because the alleged remaining falsifier is not reproducible. `POLYLOGUE_HYPOTHESIS_REUSE_FAILURES=1 devtools test tests/property/test_write_path_state_machine.py` -\u003e 3 passed (10.79s); targeted dangling/branch_point/variant/replacement/reingest lineage+write suite -\u003e 13 passed (1.15s); `devtools test -k \"session_links or composed\"` -\u003e 11 passed (23.99s). Existing deterministic coverage already includes nested dangling ancestor reingest (test_reingest_after_dangling_ancestor_does_not_fabricate_a_prefix), stale composed-ancestor repair, and caller-owned transaction rollback fixtures. Therefore do not implement speculative normalization. Remaining scope is an AC closure audit: enumerate which of persisted semantic-cut witness, crash/rollback fault injection, and typed link-state requirements are already covered by production tests versus genuinely absent; split only a demonstrated missing invariant into a smaller child. P0 rationale (currently falsifying state machine) is stale unless a fresh failing sequence is produced.\n2026-07-17: PR #3044 / 1d3145afa adds Claude Code arrival-order and replay/compaction normalization witnesses. It does not close this P0 lineage-write bead; the writer-level order-independent branch-point protocol remains in progress.\n2026-07-19 AC-closure audit (Sonnet audit lane, read-only, .agent/scratch/trust-floor-audit-2026-07-19.md has full detail): VERDICT = OPEN, not closable. Ran POLYLOGUE_HYPOTHESIS_REUSE_FAILURES=1 devtools test tests/property/test_write_path_state_machine.py (3 passed, matches prior note) plus 11 fresh runs of TestWritePathStateMachine at HYPOTHESIS_PROFILE=default (100 examples each, distinct random seeds) = 1100 fresh examples. 10 of 11 runs passed clean; run with --hypothesis-seed=577341254 produced a NEW falsifying sequence not previously recorded: ingest_initial_parent, ingest_child_replaying_parent_prefix x4, delete_parent_branch_point, teardown -\u003e IndexError: tuple index out of range at tests/property/test_write_path_state_machine.py:450 (_assert_resolved_link).\n\nRoot-cause dive via direct-SQL + composed-read oracle (scratch repro reproducing the exact call sequence outside pytest/hypothesis): production's read_archive_session_envelope composition for the affected grandchild session is CORRECT on manual verification -- its returned texts and lineage_complete=True exactly match hand-derived expected content given the surviving physical rows (cross-checked against the raw messages table and session_links rows directly). The crash is in the STATE MACHINE'S OWN bookkeeping, not production: delete_parent_branch_point's prefix_length-adjustment loop (test file lines 275-286) only walks branch points resolvable within the directly-deleted parent's position map (built once from that one parent's pre-deletion envelope) and never cascades the adjustment through a second-hop descendant whose own branch point lives inside an intermediate session's physical rows rather than the deleted session's. _assert_resolved_link then indexes parent_messages[model.prefix_length - 1] using that stale prefix_length against the intermediate parent's now-shrunk composed envelope, which is what actually raises the IndexError -- production never raises.\n\nThis means the 2026-07-17 \"not reproducible, P0 rationale is stale\" note needs a correction: a fresh falsifying sequence DOES exist on current master (verified against commit 71d134eaa with the shared-venv-resolved live checkout at 5f65ad962; diffed the two and confirmed zero changes to write.py/session_links.py/message_query_reads.py/the test file in that range, so this is a valid master finding not an artifact of venv drift). Severity is LOW: it is a test-oracle modeling gap for 3+-generation lineage under a single ancestor-message hard delete, not a demonstrated production defect. AC6 of this bead explicitly anticipates exactly this evidentiary bar (\"No property assertion or generator is weakened unless the direct SQL and composed-read oracle demonstrates model error\") -- that bar is met here, pointing at the MODEL, not write.py.\n\nRecommended narrowing (not yet done, so AC1 cannot be honestly claimed complete -- \"reproduces every saved class... committed as a named deterministic transition fixture\" excludes this new class): fix the state machine's cascading prefix_length adjustment to walk indirect/multi-hop descendants (or rewrite _assert_resolved_link to check branch-point existence by message_id rather than positional indexing into a potentially-shrunk parent envelope), reduce this exact sequence to a permanent deterministic fixture the way the three original classes were handled, then re-run the full AC7 command set. Do not close 866e until that fixture lands; do not treat this note as authorizing a production code change -- audit lane is read-only.\n\nCommands run (this pass): devtools test tests/property/test_write_path_state_machine.py (3 passed); POLYLOGUE_HYPOTHESIS_REUSE_FAILURES=1 devtools test tests/property/test_write_path_state_machine.py (3 passed, 10.8s-class); 11x devtools test tests/property/test_write_path_state_machine.py::TestWritePathStateMachine --hypothesis-seed=\u003cN\u003e under HYPOTHESIS_PROFILE=default (10 passed, 1 failed at seed=577341254); devtools test -k \"session_links or composed\" (13 passed).\n2026-07-20 fix implemented (Sonnet lane, PR #3185, branch feature/test/lineage-cascade-and-continuity-cancellation, not yet merged -- bead left open per coordinator instruction): reproduced the 2026-07-19 audit's seed-577341254 finding via a DIRECT (non-Hypothesis) call sequence -- ingest_initial_parent, ingest_child_replaying_parent_prefix x4, delete_parent_branch_point -- confirming IndexError without any dependency on Hypothesis seed reproducibility (the seed itself turned out not to reproduce deterministically standalone across environments/xdist workers; the underlying bug does, via direct rule calls). Root cause confirmed exactly as the audit's diagnosis: WritePathStateMachine.delete_parent_branch_point's prefix_length-adjustment loop only compared branch-point message ids against the directly-deleted-from parent's own pre-deletion positions, never cascading through a grandchild (e.g. child-4, parent=child-2, parent=parent-0) whose own branch point lives inside the intermediate child-2's own physical tail rather than parent-0's rows. Fixed by replacing the loop with WritePathStateMachine._cascaded_prefix_shift, a recursive walk resolving any branch point's shift against the cut root through however many prefix-sharing hops separate them (index inside a session's borrowed prefix recurses unchanged into the parent's composed transcript; index inside the session's own tail shifts by whatever the borrowed segment shrank), computed from a pre-mutation snapshot for every candidate before mutating any prefix_length. Added test_grandchild_transcript_recomposes_after_intermediate_ancestor_message_deletion as the 4th named deterministic fixture in the file's existing production-focused test_ style, locking in that production's read_archive_session_envelope was ALREADY correct for this exact multi-hop shape (confirmed via direct-SQL/composed-read manual verification before touching any code, per this bead's own AC6 evidentiary bar). Verification: devtools test tests/property/test_write_path_state_machine.py -\u003e 4 passed; 400-example run_state_machine_as_test stress run + 11 distinct --hypothesis-seed runs (incl. 577341254) at default 100-example profile all pass post-fix; mypy --strict + ruff clean. AC1 (all saved classes as named deterministic fixtures) now satisfied including this 4th class. Do not close until PR #3185 merges.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T13:44:21Z","created_by":"Sinity","updated_at":"2026-07-20T00:02:15Z","started_at":"2026-07-17T11:38:28Z","closed_at":"2026-07-20T00:02:15Z","close_reason":"PR #3185 merged: property oracle multi-hop prefix_length cascade fixed (_cascaded_prefix_shift from pre-mutation snapshot); deterministic repro + named grandchild-recomposition fixture proves production read path was already correct. Falsifying seed 577341254 and 400-example stress pass.","metadata":{"execution_mode":"evidence_first_bug_fix","frontier":"active","frontier_program_ref":"polylogue-4ts"},"labels":["area:lineage","area:storage","area:test","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-866e","depends_on_id":"polylogue-4ts","type":"parent-child","created_at":"2026-07-15T18:44:10Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-z9gh.9.1","title":"Land the shared query transaction across every read surface","description":"The mandate-critical read failures are one implementation gap, not separate ergonomics, pagination, parity, and overflow projects. Land the bounded query transaction as the sole production read boundary and absorb the complete remaining contracts of polylogue-rsad and polylogue-t46.3. The cancellation/selective-plan slices and query-receipt contract remain separately verifiable because they have distinct proof and durability concerns; all surface-specific response behavior belongs here.","design":"Add polylogue/archive/query/transaction.py as the sole orchestration boundary. QueryExecutionRequest wraps either a canonical SessionQueryPlan or terminal QueryUnitSource plus structural filters, material scope, projection/render profile, stable order/frame/snapshot, deadline, and query identity. QueryExecutor composes z9gh.1 execution control with z9gh.2 selective relations and chooses keyset continuation for stable indexed order, deterministic snapshot re-execution for safe immutable plans, or an owned ephemeral spool for recursive, aggregate, or unstable plans. QueryResultPage owns compact typed rows, exact or qualified total/coverage, useful first-page evidence, complete opaque continuation state, stable query/result refs, and independently pageable graph nodes and edges. Migration order on one branch: first query_units and list/search; then session/message/block/action/file reads; then tree/topology and insight-as-saved-query reads; then CLI, MCP, HTTP, and Python adapters; finally delete parallel pagination/filter/overflow/query-recording owners. Primary anchors: archive/query/{plan.py,archive_execution.py,unit_results.py}; storage/sqlite/archive_tiers/archive.py; api/archive.py; mcp/{server_tools.py,archive_support.py,server_support.py}; daemon/http.py; cli query contracts and verbs; surfaces/payloads.py. Replace server_support response_budget_exceeded whole-payload substitution with page construction before serialization. Absorb rsad behavior as declared projections and diagnostics: compact optional boilerplate, excerpts, truthful summaries, valid-value recovery, deduplicated sessions, and zero-hit explanation. Reuse minimal rxdo.3 identities without blocking on its full telemetry program.","acceptance_criteria":"1. z9gh.1 and z9gh.2 acceptance criteria pass through the shared executor, including prompt cancellation, bounded RSS/temp work, and selective plans. 2. Every rsad flow is satisfied at this boundary: no erased successful result; list/search/query/session/messages/tree/topology are losslessly resumable; recursive nodes/edges page independently; boilerplate is compact/opt-in; excerpts and truthful summaries exist; valid-value and zero-hit diagnostics teach recovery; list sessions deduplicates by identity. 3. CLI, MCP, HTTP, and Python execute the same canonical plan and return identical totals, stable order/frame, page boundaries, cursors, and result refs for identical requests. 4. Cursor/ref state preserves the complete expression, structural filters, material scope, projection/render budget, sort, snapshot, order, and query-run identity; following it yields every logical row exactly once and terminates with no semantic cap. 5. rxdo.3 query-run/result/evaluation refs are populated for committed reads. 6. Grep/source review finds no second per-surface owner for filter mapping, continuation state, totals, overflow replacement, cancellation, or query receipts, and no path serializes a full result merely to discard it. 7. The original archaeology flow and Workflow reconstruction finish in fewer than ten discovery/read calls, while live-scale and mutation tests fail when the shared executor is removed or bypassed.","notes":"[2026-07-15 invariant-collapse pass] This bead now absorbs the full remaining scope of polylogue-rsad and polylogue-t46.3. They are superseded rather than independently scheduled; their incident details remain durable regression evidence. It composes under polylogue-4p1 read algebra rather than replacing that domain contract.\nFrontier correction 2026-07-15: full rxdo.3 telemetry/privacy/@last behavior is compatible follow-on work, not a hard prerequisite for restoring lossless bounded query execution. This slice owns the minimal stable query/result/continuation identity required by transport; rxdo.3 enriches committed-run telemetry without gating the mandate repair.\n[2026-07-15 exact transport/execution replay] Valid live reads were destroyed at the interface boundary: list 100 found 129 candidates and built 54,344 bytes; search Sonnet found 50 and built 112,701 bytes; get_session_topology for the correct coordinator built 137,223 bytes. Each successful result was replaced wholesale by response_budget_exceeded with continuation.arguments={}, so no evidence row survived and replay was impossible. After narrowing to limit=3, useful rows arrived. Correct structural attempts then exposed execution failure: archive_list_sessions(tool=Workflow) plus a Wave 2 search ran \u003e150s before termination; the Wave 2 search alone ran \u003e79s; query_units over delegations for the known coordinator ran \u003e79s. The shared transaction must physically page before full serialization, emit complete opaque continuation state, preserve a useful prefix/page, and make termination interrupt SQLite/derived work. Requested limit is a logical maximum, never a requirement that one transport response contain that many rows.\n[2026-07-15 continuation root cause] The empty continuation was deterministic, not lost by the client. async_safe_call installs only fallback response arguments derived from an optional session_id. archive_list_sessions and archive_search_sessions never enter hooks.response_context with their real request, and pass no session_id, so _budget_envelope sees their tool name with arguments={}. archive_search_sessions also exposes no offset/cursor and reports total=len(capped_hits), so even a preserved smaller limit could not enumerate the full match set. get_session_topology has no paging/projection argument; its generic fallback can only replay the identical oversized call, creating a non-progressing loop. Existing continuation tests cover archive_get_session and explicitly context-wrapped tools, but there is no over-budget contract test for archive list/search/topology. The shared transaction must delete this per-tool opt-in context/fallback design, not merely fill three missing dictionaries.\n[2026-07-15 semantic-cap census] Static MCP inspection finds 18 directly registered tools that accept limit but expose no offset/cursor/page token: compose_context_preamble, tool_call_latency_distribution, find_stuck_sessions, find_abandoned_sessions, find_resume_candidates, find_similar_sessions, get_postmortem_bundle, get_pathologies, archive_search_sessions, neighbor_candidates, provider_usage, blackboard_list, list_assertion_claims, list_assertion_candidates, list_assertion_candidate_reviews, archive_debt, explain_import, and agent_coordination. This excludes dynamically registered insights and oversized unpaged graphs such as topology. Not every limit is wrong: ranked recommendations, summaries, and context compilation may be intentionally bounded. But every read must declare whether it is an exhaustive relation page, top-k ranking, sample, aggregate summary, or bounded context; exhaustive logical results need continuation/result refs, and ranked/summary surfaces need an exhaustive underlying query path plus explicit omitted/coverage semantics. No hidden limit may masquerade as totality.\n[2026-07-15 live audit reproduction] MCP search with origin=chatgpt-export and limit=30 built 594,054 bytes for query polylogue and returned only response_budget_exceeded. Its prescribed recovery was limit=3, but the same call at limit=3 still built 41,371 bytes and returned the same metadata-only refusal, so the continuation did not make progress. query_units over messages where text:\"prework\" built 450,317 bytes and returned continuation.arguments={}, which cannot replay the expression or filters. This independently reproduces both failure classes already owned here: a narrowing instruction that remains over budget and an uninvokable empty continuation.\nInvariant consolidation 2026-07-15: absorbs polylogue-20d.5. Its three concrete residues—lineage-composed transcript streaming, messages --full iterator/file output, and SQL-pushed material_origin pagination—are required paths under the sole bounded query transaction and its no-full-materialization AC.\n[2026-07-15 installed-skill dogfood reproduction] Invoking MCP readiness_check through the shipped Polylogue skill built 27,673 bytes against a 25,000-byte budget and returned response_budget_exceeded with continuation.tool=readiness_check and continuation.arguments={}. Repeating that continuation necessarily rebuilds the same oversized monolith, so the advertised recovery cannot progress. Add readiness/status to the over-budget regression matrix: preserve a useful compact page, return a component/detail ref or advancing cursor, and never prescribe an identical argument-less replay.\nTerra-readiness correction 2026-07-15: fixed the module boundary, plan/page strategy, adapter migration order, and deletion targets. z9gh.1 owns execution control; z9gh.2 owns selective derived relations; this bead owns the one transaction and every surface adapter.\n2026-07-17 implementation-readiness integration map: this is the only transaction integration branch. Consume, do not reimplement, #2964 execution control (z9gh.1) and #3004 declarations (z9gh.3 substrate). z9gh.2 is the only new derived index prerequisite. Land in this order on one branch: canonical request/page/opaque-cursor types and query_units route; lossless first-page construction replacing response_budget_exceeded; list/search; structural session/message/block/action/file; tree/topology/insight; then CLI/MCP/HTTP/Python parity and deletion of duplicate pagination owners. Before each migration, pin existing route rows/totals/order/error semantics as a production golden; after, prove the same canonical plan and cursor enumerate exactly once. A current grep target is mcp/server_support.py response_budget_exceeded and all envelope/pagination owners; do not claim completion while that whole-payload replacement remains reachable.\n2026-07-17 PR #3018 implementation receipt: QueryTransaction/QueryContinuation/QueryResultPage now provide canonical request identity, q1 advancing replay, bounded execution, result refs, exact totals where provable, and surface migration across API/CLI/daemon/MCP/annotations/demo. Focused post-merge 67 passed and affected-area 1030 passed, 1 skipped, 1 deselected. No private 4.85-million-block replay was run; z9gh.7 remains the live terminal gate.\n[2026-07-17 bounded read/MCP closure] Routed terminal query-unit API and HTTP adapters through QueryTransaction, made MCP query_units own one canonical request identity for receipts/result refs/continuations, and changed ordinary action pages to select base rows before computing follow-up detail. Registered MCP route test covers query capability discovery plus advancing action continuation; focused devtools test passed 4 tests and devtools verify --quick passed. Not closed: z9gh.7 still owns the private live 4.85-million-block cold-model/terminal replay, which was not automated here.\n[2026-07-18, PR #3068] Added archive-epoch binding to QueryTransactionRequest/QueryContinuation and wired QueryContinuationStaleError into the MCP query_units resume path (see polylogue-z9gh.9 note for the full reconciliation context against an external handoff packet). Residual CLI/MCP/HTTP/Python parity gap this surfaced and did NOT fix: daemon HTTP's /api/query-units accepts no `continuation` query parameter at all, so it has no resume path today even though it returns a continuation token in its response (MCP already accepts+validates continuation; API/Python query_units also has no continuation input parameter). Wiring HTTP (and API) continuation resume, with the same epoch-staleness check now available via validate_continuation_epoch(), is straightforward follow-up scope for whichever pass finishes \"CLI/MCP/HTTP/Python parity\" migration.\n[2026-07-19, PR #3171 investigation] Investigated the 2026-07-18 residual gap\n(HTTP /api/query-units has no continuation param; API/CLI query_units also\nmissing continuation input). Source review found BOTH flagged gaps already\nclosed on master by the same day's later commit: dc6fa632a/#3095 (the\nsix-tool MCP cutover hardening pass, merged 18:24 -- after this bead's\n00:16 note) added continuation decoding + QueryContinuationStaleError\nhandling to both DaemonAPIHandler._handle_query_units (polylogue/daemon/http.py)\nand Polylogue.query_units (polylogue/api/archive.py), reusing the same\nquery_units_transaction_request/QueryTransaction/QueryContinuation\nprimitives MCP already validated -- MCP's query tool\n(mcp/server_cutover.py) in fact delegates straight into\nPolylogue.query_units, so there was never a second mechanism built. The\nbead's own note predates that fix and was never updated.\n\nWhat was genuinely still missing: proof this held together end-to-end.\nHTTP had its own continuation suite (tests/unit/daemon/test_web_reader.py),\nAPI's own continuation= keyword had ZERO direct test coverage (only\nindirect coverage via MCP tests patching the facade in), and no test drove\nHTTP+API+MCP against one shared corpus.\n\nPR #3171 closes that test gap: 4 new API-direct continuation tests\n(tests/unit/api/test_facade_contracts.py) plus a genuine 3-surface parity\nsuite (tests/unit/archive/query/test_continuation_surface_parity.py)\nproving byte-identical message_id ordering/query_ref/result_ref across\nHTTP/API/MCP for the same expression+continuation, and identical\nquery_continuation_stale rejection on all three when a write lands\nmid-resume. No production code changed; no second continuation mechanism\nbuilt.\n\nExplicitly NOT touched: CLI does not call query_units at all (separate\nSessionQuerySpec/archive_query.py limit-offset path) -- no CLI paging\nsurface exists to wire continuation into. Also reviewed but left alone:\nthe z9gh.3-flagged near:\"\"/lineage:id: execution-layer gaps live in the\ngeneric find/read CLI route, not this HTTP/API/MCP query_units path.\n\nVerification: devtools test (parity+facade+MCP surfaces) 286 passed;\ndevtools verify --quick green 16/16. One pre-existing unrelated failure\nnoted (test_web_reader.py::test_operational_web_payloads_redact_configured_archive_paths,\nreproduced at origin/master HEAD 71d134eaa with none of this PR's files\npresent).\n\nPR: https://github.com/Sinity/polylogue/pull/3171","status":"closed","priority":0,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T22:57:21Z","created_by":"Sinity","updated_at":"2026-07-20T05:58:11Z","started_at":"2026-07-17T11:44:58Z","closed_at":"2026-07-20T05:58:11Z","close_reason":"Resolved by AC clarification consistent with the epic own framing (surfaces are leaf renderers of the query transaction): parity holds across every surface that exposes query_units — HTTP (daemon/http.py:4040), API (api/archive.py:3261), MCP — identical totals/order/cursors + stale-epoch rejection, pinned by test_continuation_surface_parity.py + facade contracts (280 passed, re-run 2026-07-20). CLI find/read verbs use the pre-existing SessionFilter/SessionQuerySpec contract and expose no query_units terminal — never in migration scope (source-verified: zero query_units calls under polylogue/cli/). If CLI continuation semantics are ever wanted, that is a new design decision - follow-up bead on request.","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:mcp","area:protocol","area:query","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-20d.5","type":"supersedes","created_at":"2026-07-15T21:34:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-7q16","type":"relates-to","created_at":"2026-07-15T06:25:57Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-9l5.6","type":"relates-to","created_at":"2026-07-15T06:26:00Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-rxdo.3","type":"relates-to","created_at":"2026-07-15T19:19:31Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-z9gh.1","type":"blocks","created_at":"2026-07-15T00:57:24Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-z9gh.2","type":"blocks","created_at":"2026-07-15T00:57:27Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-z9gh.3","type":"relates-to","created_at":"2026-07-15T19:19:42Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9.1","depends_on_id":"polylogue-z9gh.9","type":"parent-child","created_at":"2026-07-15T00:57:20Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6407-5537-72fe-b283-e820d2e288b7","issue_id":"polylogue-z9gh.9.1","author":"Sinity","text":"[Dogfood 2026-07-15 / F-003, F-006 diagnostics, F-009, F-011 contract, F-015] Live exact-selection canaries expose the shared-transaction gap. A bare native UUID resolves in SQL but list/select discards the canonical row via an unresolved residual startswith filter. Exact canonical ID then analyze count, grouped stats, facets, and postmortem silently broadened to all 18,430 sessions because CLI aggregate dispatch and the API kwargs adapter omit session_id; pathology materialization shares the write-side risk. Explain recompiles query terms and omits root flags. Exact summary and transcript JSON were byte-identical at 242,783 bytes. Public tool_result_is_error is integer on one route, bool on another, absent on a third. These are primary closure anchors for canonical request scope, deletion of parallel filter maps, typed pages, and truthful summary projection.","created_at":"2026-07-15T04:27:10Z"}],"dependency_count":2,"dependent_count":5,"comment_count":1} {"_type":"issue","id":"polylogue-z9gh.9","title":"Make every archive read a bounded, resumable query transaction","description":"The oversized-response cliff, lost continuation arguments, blocking SQLite event loop, uncancellable statements, global-view materialization, divergent surface totals, and hidden query refs are manifestations of one missing abstraction: Polylogue has query functions but no shared query transaction. Each surface currently decides how to execute, budget, serialize, page, continue, and record a read. A correct archive read needs one contract from canonical plan through bounded execution to a stable result page/ref.","design":"Define one QueryExecutionRequest carrying canonical plan, structural filters, projection, stable ordering, archive/frame epoch, and deadline/resource policy. Execute through one QueryExecutor off the event loop with cancellation propagated to SQLite. Return one QueryResultPage contract carrying rows, exactness/frame/authority, query-run and result-set refs, complete continuation state, timing/resource telemetry, and recoverable errors. The protocol may implement stable keyset re-execution against an immutable archive epoch, incremental streaming, or an ephemeral disk-backed relation according to query shape; it must not require full in-memory materialization and must never impose a semantic row cap. CLI, MCP, HTTP, and Python are leaf renderers of this transaction. Query planners expose selective-plan evidence and SLO classifications through the same execution receipt.","acceptance_criteria":"1. One request/executor/page contract owns execution, cancellation, paging, refs, totals/exactness, ordering, snapshot/frame, and telemetry across CLI, MCP, HTTP, and Python. 2. Every query shape returns a bounded first page plus a lossless continuation or stable result ref without first serializing the full logical result. 3. Client cancellation and deadlines interrupt SQLite promptly while unrelated calls remain responsive. 4. Selective filters reach base relations before archive-wide windows/groups; plan/SLO regressions fail a live-scale harness. 5. Cursors preserve all original query state and enumerate each row exactly once against a declared archive epoch. 6. rxdo.3 query receipts and rsad response paging are produced at this chokepoint, not parallel per-surface hooks. 7. The known 129-session list, recursive topology, Workflow-tool selection, and coordinator delegation cases pass inside the declared resource envelope.","notes":"[2026-07-18 sol-pro-dispatch handoff reconciliation, PR #3068] Received an external \"phase-one QueryTransaction kernel\" handoff packet (branch feature/browser/sol-pro-dispatch, base 5abb30af — 151 commits stale vs master at reconciliation time). Adversarial check found the claimed kernel already substantially landed: PR #2964 (execution_control.py: QueryExecutionContext/QueryAdmissionController/InterruptibleSQLiteRead, closing z9gh.1) and PR #3018 (archive/query/transaction.py: QueryTransaction/QueryTransactionRequest/QueryContinuation/QueryResultPage, the z9gh.9.1 integration branch) already deliver the one request/executor/page contract, off-loop cancellation, and continuation replay of expression+session_filters. `git apply --check` failed on every file the handoff patch touched (transaction.py already exists with a different, more mature implementation — no FIFO weighted admission, no work-budget receipts, no selective action_pairs/delegation_facts in the handoff's version). Did not apply the handoff patch.\n\nFound one genuine, still-open gap against this bead's AC #5 (\"enumerate each row exactly once against a declared archive epoch\"): QueryTransactionRequest/QueryContinuation carried no archive-frame identity at all, so a query_units continuation issued before a session was written/mutated could resume against a moved relation via plain offset pagination with no staleness detection. PR #3068 closes this specific gap: archive_index_epoch() (schema version + session count/rowid/watermark, deliberately stronger than production_evaluator._index_epoch's watermark-only formula, which misses a session admitted before its updated_at_ms backfill — verified empirically), QueryTransactionRequest.archive_epoch + epoch-aware result_ref, QueryContinuationStaleError wired into the MCP query_units resume path, and one shared query_units_transaction_request() constructor replacing three independently-maintained QueryTransactionRequest construction blocks (API/MCP/HTTP).\n\nResidual scope NOT covered by PR #3068, carried forward here: (1) HTTP /api/query-units has no continuation query parameter at all today (no resume path exists there) — only ensured HTTP's issued continuations carry the current epoch; wiring HTTP resume is separate follow-up scope. (2) Byte-exact serialization-budget page construction for query_units specifically was investigated and found already substantially handled by the generic MCP _budget_envelope binary-search bounded-page mechanism (tests/unit/mcp/test_bounded_query_transport.py) — not reimplemented. (3) Live-scale 4.85M-block proof, weighted-fair admission under real load, ops.db query receipts, durable spool/keyset resume remain out of scope per the epic's phased plan (z9gh.7 and rxdo.3 siblings); not claimed here.","status":"closed","priority":0,"issue_type":"epic","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T22:54:18Z","created_by":"Sinity","updated_at":"2026-07-20T05:58:35Z","started_at":"2026-07-17T11:44:58Z","closed_at":"2026-07-20T05:58:35Z","close_reason":"Epic mechanism scope complete: QueryTransaction/Continuation/ResultPage landed (#3018), archive-epoch binding + stale rejection (#3068), cross-surface parity pinned (#3095/#3171, 280 tests re-run green 2026-07-20). Child .9.1 resolved by AC re-scoping; live-incident envelope proof owned by z9gh.7.","labels":["area:mcp","area:perf","area:protocol","area:query","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-z9gh.9","depends_on_id":"polylogue-20d.14","type":"related","created_at":"2026-07-15T00:54:37Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9","depends_on_id":"polylogue-4p1","type":"relates-to","created_at":"2026-07-15T01:31:59Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9","depends_on_id":"polylogue-rxdo.3","type":"related","created_at":"2026-07-15T00:54:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.9","depends_on_id":"polylogue-z9gh","type":"parent-child","created_at":"2026-07-15T00:54:17Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} @@ -18,7 +18,7 @@ {"_type":"issue","id":"polylogue-z9gh.3","title":"Generate agent query discovery from executable query declarations","description":"The model-facing failures are one declaration problem, not separate missing filters and bad descriptions. Polylogue already has query metadata, unit registries, field definitions, examples, insight descriptors, and generated surfaces, but MCP discovery does not project them coherently. Tool descriptions repeat generic boilerplate, query_units hides its grammar, list/search refer to internal request type names, and compact rows omit structural dimensions that the planner already knows or could expose. Query vocabulary must be executable data from which model schemas, resources, errors, completions, projections, and docs are generated.","design":"Extend the declare-once query registry so each unit/field/operation declares public name, semantic fact family, type/closed values, operators, projection fields, evidence authority, applicable origins/artifact kinds, freshness/coverage source, pushdown/cardinality/cost/plan shape, stable order key, examples, recovery guidance, and curated intent recipes. Generate MCP schemas/descriptions, a searchable/queryable capability catalog resource, CLI completions/help, OpenAPI/JSON schemas, typed errors, compact projections, and the six shipped intent prompts from these declarations plus OriginSpec coverage. Provide expressive DSL and typed structured-plan lowering to one AST. Discovery answers both how to express a query and whether the current archive can answer it: observed counts/coverage age, unavailable/unsupported/unknown dimensions, estimated plan, and suggested refinements. It is paged/searchable rather than one enormous static tool description. Compact rows expose parent/root/branch/model/material/orchestration refs without hydration.","acceptance_criteria":"1. One query declaration registry generates MCP schemas/descriptions, a searchable capability/coverage catalog resource, typed structured-plan input, DSL completions/help, OpenAPI/docs, compact projections, valid-value errors, and curated intent recipes. 2. Every fact family/field declares meaning, authority, applicable origins/artifacts, coverage/freshness source, projection, pushdown/cardinality/cost plan, stable order, and examples; the live catalog distinguishes supported-and-observed, supported-but-absent/stale/degraded, unsupported, and unknown. 3. The six existing query-cookbook prompts are preserved as generated/tested seed recipes with cwd/repo binding where applicable; harness skill text is derived or parity-checked against the same declarations. 4. A cold model using discovery alone can formulate and execute resume, postmortem, decision, failure, file-touch, cost, coordinator-child/model/material/orchestration, and paging flows from sparse operator wording, and can state what evidence is unavailable before guessing. 5. DSL, structured-plan, and recipe forms lower to the same canonical plan and produce identical refs/rows/totals/errors. 6. Explain/catalog output exposes current estimated rows, selective predicates, joins/expensive relations, snapshot/freshness, and a next narrowing or asynchronous execution strategy; valid expensive combinations remain answerable via queue/stream/page/spool, never rejected merely for size/cost. 7. Adding/removing a field, origin mapping, or recipe updates every surface and a missing projection/registration/coverage/parity mapping fails one actionable check. 8. No public description refers only to internal Python types or hidden docs; catalog queries are bounded and paged, and usage telemetry may evaluate recipe effectiveness without becoming authority.","notes":"Contract correction 2026-07-15: query cost classification is planner input, not permission for a product-level hard cap. Discovery must teach execution strategy and progress, not tell the model that a valid large question is unsupported.\nInvariant collapse 2026-07-15: absorbs the static query-discovery remainder of pj8. Its six prompts and harness skill already shipped; future source-of-truth/parity lives here. SessionStart affordance remains 37t.4; measured/adaptive curriculum remains xv1u.\nDogfood correction 2026-07-15: the failed session lacked not just query syntax but an archive-grounded inventory of normalized facts, coverage, freshness, authority, cardinality, and available narrowing dimensions. This belongs in query discovery as a queryable catalog joined to OriginSpec, not in operator memory or a giant static prompt.\nFrontier correction 2026-07-15: executable declarations, capability/coverage catalog, recipes, and structured-plan lowering can land before the shared transport executor. Transaction-specific cost/progress fields integrate when z9gh.9.1 lands; neither bead waits idly on the other.\n[2026-07-15 exact failed-query audit] The incident contains four initial calls. (1) archive_list_sessions(cwd_prefix=/realm/project/polylogue, since=local-day-start, origin=claude-code-session, limit=100) was a reasonable candidate-enumeration request, albeit with an unnecessarily large requested page; it found total=129 and serialized 54,344 bytes, then returned zero rows because the 25 KiB callback replaced the payload. The contract should choose a smaller physical page and preserve a cursor, not require the model to predict serialized size. (2) exact text \"each handling a concern\" was an unreasonable primary selector because that wording came from the operator's current Codex message, not necessarily the Claude corpus; a correct zero result needed diagnostics showing searchable fact families and Workflow artifact coverage. (3) text Sonnet was a weak selector because model identity and authored task material are structured dimensions, while the word is common in runtime instructions/tool output; however the exposed archive_search_sessions schema had neither model nor material-origin scope, so discovery induced this lexical mistake. It found 50 hits/112,701 bytes and again erased them. (4) query_units(\"sessions where repo:polylogue since:1d\") was malformed because it was nonterminal; the tool description only said \"terminal rows\" and the error returned no valid forms, examples, completions, or suggested terminal projection. Although query_completions/explain tools existed elsewhere in the 94-tool surface, a cold model could not reasonably infer that detour. Catalog/recipe tests must teach structured model/material/orchestration selection and terminal syntax from the sparse intent before execution.\n[2026-07-15 curriculum-parity correction, superseding the earlier grading of call 4] The nonterminal query_units call was syntactically invalid for the live parser but reasonable for the model to issue: the installed Polylogue skill explicitly teaches query_units(expression='sessions where repo:\u003cr\u003e since:7d AND exists action(...)') and query_units(expression='sessions where repo:\u003cr\u003e AND exists file(...)'). parse_unit_source_expression cannot return sessions as a plain terminal source; sessions is only a scoping stage before a terminal unit. The live error returned no valid forms. Thus this is not merely hidden grammar: product-owned curriculum directly contradicted executable semantics. Registry generation/parity must make such an example impossible, and the exact two installed recipes are regression fixtures.\n[2026-07-15 shipped-regression evidence] The contradiction is present in both product layers: polylogue/mcp/server_prompts.py advertises invalid sessions-only query_units expressions in unacknowledged_failures and sessions_touching_file, and the installed shared Polylogue skill repeats them. Tests only assert prompt names/tool-name sequences; they do not compile or execute embedded expressions, while separate query_units tests explicitly require rejecting session expressions. Therefore two of the six shipped continuity recipes are self-contradictory by construction. Priority raised to P0: this is an active mandate regression, not future discoverability polish.\n[2026-07-15 result-semantics requirement] The query declaration registry must classify result semantics, not just cost: exhaustive relation page, top-k ranking, sample, aggregate summary, bounded context, or recursive graph page. Generate totals/exactness/omitted fields and continuation requirements from that class. The live static census already has 18 limit-without-pagination tools; discovery must tell a model whether it is seeing all evidence, a ranked frontier, or a bounded orientation view, and provide the exhaustive route where one exists.\n2026-07-16 GPT-Pro corpus adjudication: query-discovery package 073651c8d9b3 is retained as research/seeded implementation input; e426074411b8 is its semantic alternative. Discovery must be generated from executable declarations and share z9gh query execution/continuation semantics, never a second per-surface registry. Exact blocker: z9gh.1 shared execution transaction.\n2026-07-17 browser-capture recovery: a same-day raw ChatGPT handoff is present at capture ref chatgpt:6a580976-03d0-83eb-af6a-eb745db5ac0c (title: Agent Query Discovery; capture file mtime 2026-07-17 07:45 CEST). It explicitly assigns the P0 declare-once registry/capability-and-coverage catalog, generated MCP descriptions/schemas, valid DSL and structured-plan examples, compact projections, result/continuation semantics, and parity compilation of every shipped cookbook prompt and installed-skill expression. It names the direct incident: sessions-only query_units recipes are parser-invalid and cold agents receive no usable recovery. Treat this capture as scope-confirming design evidence only; any external ZIP/patch remains unaccepted until locally retrieved, reviewed, and verified.\n2026-07-17 implementation-readiness audit: #3004 (ed44be18f) now supplies the storage-free declaration kernel at polylogue/declarations/ and the current 104-tool MCP registry at polylogue/mcp/declarations/. This bead must extend that single declaration graph with query fact/field/capability/coverage declarations; it must not create a second query catalog or use campaign/skill prose as authority. Current executable query grammar entry points are archive/query/expression.py, plan.py, metadata.py, completions.py, and api/archive.py:query_units; current invalid sessions-only regression is tests/unit/api/test_facade_contracts.py:test_query_units_rejects_session_expression. First concrete deliverable: compile every shipped MCP prompt and installed skill example against this same parser/canonical plan, then generate repair text/catalog rows from the declaration graph. Treat GPT Pro discovery material as fixture input only.\n2026-07-17 PR #3018 implementation receipt: executable MCP query capability discovery/resource metadata, bounded unit descriptors, grammar/field/coverage/result semantics, corrected recipes, and query_units replay contracts landed. Generated MCP/OpenAPI/CLI surfaces and MCP contract tests pass. A full cold-model live terminal walk remains with z9gh.7.\n2026-07-18: Landed the parser-truthful discovery-corpus slice via PR #3066\n(feature/query/discovery-corpus-mcp04), reviewing external-agent (GPT Pro)\npackaged output against snapshot 536a53ef, reconciled against 13 commits of\nsubsequent master drift, and independently re-verified (not just re-run from\nthe packet's own claims).\n\nSatisfied:\n- polylogue/archive/query/discovery.py: 106 positive + 18 negative typed\n corpus rows (expression, parser route, unit source, answer, semantics\n class, projection columns, cost class); all six semantics classes and all\n ten unit sources represented.\n- Production-parser anti-lying gate (tests/unit/archive/query/test_discovery.py):\n every positive row parses through compile_expression/parse_unit_source_expression\n (no mock grammar); every negative row pins the exact ExpressionCompileError\n class/text/field plus a parser-valid correction.\n- Shared QueryResultSemanticsContract vocabulary (archive/query/transaction.py)\n mapped onto the EXISTING MCPResultSemantics enum (exhaustive_page/top_k/\n sample/aggregate/bounded_context/recursive_graph) -- no second taxonomy.\n- Rewired highest-risk teaching routes: 4 MCP cookbook prompts, query\n capability resource (v2), query_completions(kind=example|error), root CLI\n help, a new generated docs/search.md corpus section.\n- Fixed 3 confirmed shipped-invalid examples (2 MCP prompts, 1 docs/search.md\n snippet) that raised ExpressionCompileError against the real parser.\n- No grammar change: archive/query/expression.py untouched (verified via\n empty git diff on that path).\n- Zero overlap with polylogue/mcp/declarations/** or registry.py (checked\n the diff's full file list; confirmed untouched).\n\nExplicitly deferred (still open, tracked on this bead / z9gh.9.1 / t46.8.1):\n- Typed structured-plan lowering to one AST.\n- OpenAPI/JSON schema generation for the discovery vocabulary.\n- Live coverage/freshness/cardinality discovery, current-value discovery.\n- The full six-tool explain transaction.\n- Migrating every remaining hand-authored docs/help example to corpus keys\n (parser-gated now, not all declaration-rendered).\n- Making every live read adapter emit the exact/qualified totals and\n continuations this vocabulary describes (z9gh.9.1's executor migration).\n\nAdditional gaps found during independent spot-check verification (11 corpus\nrows across all six classes executed against a `polylogue demo seed` archive,\nnot just parsed): near:id: similarity queries execute without error via the\ngeneric find/read CLI route but do not perform real similarity ranking\n(falls back to an unfiltered session list), and declared recursive-page\nprojection columns (parent_refs/child_refs/continuation) are not materialized\nby that same route for lineage:id: queries. Both are pre-existing\nexecution-layer gaps, NOT introduced by this PR (expression.py,\nexecution_control.py, unit_results.py are all untouched by the diff) --\nconsistent with this corpus's own disclosed limitation that non-exhaustive/\nnon-aggregate semantics classes are parser-valid but not execution-verified.\nNot filing a separate bead for these since they overlap the already-tracked\nz9gh.9.1 executor-migration scope; flagging here so they aren't lost.\n\nAlso found and fixed 2 real mypy --strict errors the source packet's own\nverification never caught (loop-variable type collisions from reusing a\nloop variable name across two differently-typed for-loops in the same\nfunction, in devtools/render_query_discovery.py and\ntests/unit/archive/query/test_discovery.py).\n2026-07-20 evidence re-scope: AC NARROWED to the residuals its own notes name — (a) structured-plan to AST lowering + OpenAPI/JSON schema generation for the discovery vocabulary (size M); (b) near:\"\" similarity executes without real ranking and lineage:id: recursive-page projection columns unmaterialized (execution-layer gaps, size S). Everything else landed (#3018/#3066; cookbook corrected; context/status intents verified in server_cutover.py).\n2026-07-27 size-S residual PR: opened #3296 (feature/query/near-lineage-execution-materialize) fixing the two execution-layer gaps this bead's 2026-07-20 note narrowed scope to. (1) near:id: through the generic find/read CLI route (cli/archive_query.py) never read SessionQuerySpec.similar_session_id at all -- confirmed empirically (identical unfiltered session order with/without the predicate against a seeded demo archive) -- now threads it through _query_hits reusing the existing VectorProvider.query_by_session mechanism (same one archive_execution.py's SessionFilter route already used since #3018), raising a typed click.UsageError when no vector backend/embeddings exist rather than degrading silently. (2) lineage:id: SQL filtering was already correct but never populated the declared parent_refs/child_refs/continuation recursive-page projection columns from archive/query/discovery.py -- added ArchiveStore.session_lineage_edges (bounded query over the existing sessions.parent_session_id column) and wired it into the CLI list route when boolean_predicate carries a QueryLineagePredicate. No grammar change (expression.py untouched). Did NOT touch the separate size-M residual (structured-plan-\u003eAST lowering + OpenAPI generation for the discovery vocabulary) -- left for a future pass. Not merged -- PR pending CI/review.\n2026-07-27: size-S residual (near:id: vector ranking + lineage:id: parent_refs/child_refs/continuation materialization) merged via PR #3296. Found and fixed a real bug during self-review before merging (CodeRabbit rate-limited): the lineage-seed detector recursed into OR-combined boolean predicate children, which would have stamped a lineage's parent/child refs onto unrelated rows matched only via an 'or' branch - fixed to only recurse on AND, added regression test. Size-M residual (structured-plan -\u003e canonical AST lowering + OpenAPI generation) not attempted, remains open scope.\n2026-07-27 size-M residual PR: opened #3330 (feature/query/ast-lowering-openapi) covering the \"structured-plan -\u003e canonical AST lowering + OpenAPI generation\" residual named in the 2026-07-20 evidence re-scope note. New polylogue/archive/query/query_ast_schema.py: Pydantic models mirroring the existing predicate/pipeline-stage/clause dataclasses' to_payload() shapes one-to-one (QueryPredicateAst discriminated union, QueryExpressionExplanationAst envelope), versioned polylogue.query-explain-ast.v1 (kept distinct from the existing polylogue.query-definition.v1 hashing-protocol version). Validates rather than re-derives: predicate_to_ast()/explanation_payload_to_ast() run the dataclasses' own to_payload() output through the schema, so drift fails a test instead of silently diverging. QueryExpressionExplanation.to_payload() now stamps schema_version (only new key) -- MCP explain(kind=\"query\") and Polylogue.explain_query_expression() pick it up automatically, no call-site change. Wired into devtools/render_openapi.py: QueryExpressionExplanationAst (+ full nested $defs) published in docs/openapi/search.yaml, plus an x-polylogue-query-ast vendor extension. No new HTTP route added -- there is no existing daemon route for query-explain (only MCP/Python facade), so this stays schema-only rather than growing a new live surface; noted explicitly as deferred, not silently dropped. Verified: 28 new tests (predicate\u003c-\u003eAST round trip over 11 shapes, full explanation validation over 13 representative expressions incl. near:/lineage:/exists/seq/pipeline-stages/JSON-spec/reference-pipeline, JSON-Schema buildability, 2 drift-rejection tests) + ad hoc sweep of all 106 positive rows in discovery.py's QUERY_DISCOVERY_EXAMPLES corpus (0 failures) + 751 passed/1 skipped on the existing explain/predicate/openapi test files (no regressions) + mypy --strict/ruff clean + devtools render all --check sync OK. Did NOT run the full non-slow suite/seed-testmon (stalled on unrelated shared-host contention, not this change). This closes out the last named residual on this bead's scope per the 2026-07-20 narrowing note; the program bead itself (z9gh.3) may still have broader open scope beyond these two named residuals -- not closing it here, leaving that call to the operator/triage.\n2026-07-27 closure-decision audit (independent, evidence-first; full description/design/AC/notes read verbatim via `bd show z9gh.3 --json`; source read directly, not summarized from prior notes). VERDICT: STAYS OPEN -- real progress is substantial but the literal 8-item AC has two genuine, not-yet-delivered items, not merely undone busywork.\n\nPer-AC verdict with file/test citations (worktree at feature/query/fix-session-projection-parity-z9gh3, based on master @ 2eb27530c):\n\nAC1 (one registry generates MCP schemas, capability/coverage catalog resource, typed structured-plan input, DSL completions/help, OpenAPI/docs, compact projections, valid-value errors, curated intent recipes) -- SATISFIED. polylogue/archive/query/discovery.py + metadata.py + query_ast_schema.py is the one declaration source. polylogue://capabilities/query resource (mcp/server_resources.py:164) projects units/grammar/result-semantics/corpus counts. query_ast_schema.py (PR #3330) gives typed AST + OpenAPI wiring (devtools/render_openapi.py). completions.py serves query_completions(kind=example|error) from the same corpus. QUERY_DISCOVERY_NEGATIVE_EXAMPLES pin real ExpressionCompileError diagnostics + corrections. server_prompts.py's 6 cookbook prompts render corpus expressions via render_query_discovery_example(). All verified green: tests/unit/archive/query/test_discovery.py + test_query_ast_schema.py, 159 passed.\n\nAC2 (every fact family/field declares meaning, authority, applicable origins/artifacts, coverage/freshness source, projection, pushdown/cardinality/cost plan, stable order, examples; live catalog distinguishes supported-and-observed / supported-but-absent-stale-degraded / unsupported / unknown) -- NOT SATISFIED AT THE DECLARED GRANULARITY. What exists: per-example cost_class (selective/corpus-scale) + result_semantics (6 classes) in discovery.py, and per-unit structural facts (lowerer_kind, exists_supported, aggregate_group_fields, time_sort_supported) in metadata.py's QueryUnitDescriptor -- no per-field/fact-family authority, applicable-origin, freshness-source, pushdown/cardinality/cost declaration exists anywhere (grepped polylogue/archive/query/*.py, polylogue/mcp/*.py; QueryFieldDescriptor in fields.py carries DSL wiring -- spec_attr/plan_attr/completion_source -- not authority/origin-applicability/freshness metadata). No 4-state (supported-and-observed/absent-stale-degraded/unsupported/unknown) coverage vocabulary is wired into the query catalog; the one near-hit, ProviderUsageCoverage in storage/usage.py, is an unrelated cost-accounting concept. This is real, unimplemented scope, not a documentation gap.\n\nAC3 (six cookbook prompts preserved as generated/tested seed recipes with cwd/repo binding; harness skill parity-checked) -- SATISFIED. Verified in polylogue/mcp/server_prompts.py: decisions_about, unacknowledged_failures, sessions_touching_file, cost_of, resume_context, postmortem_last all call render_query_discovery_example(...) with repo/cwd via _repo_context(repo), not hand-written strings.\n\nAC4 (cold model from discovery alone formulates+executes resume/postmortem/decision/failure/file-touch/cost/coordinator-child/model/material/orchestration/paging flows, states unavailable evidence before guessing) -- ADVANCED, NOT PROVEN AS THIS BEAD'S OWN CLOSURE EVIDENCE. The infrastructure that would support this (AC1's corpus/recipes/result-semantics teaching) is real and tested. The actual end-to-end cold-model proof exists but lives elsewhere: PR #3334 (74c2f3884, devtools cold-model MCP pagination+cancellation replay, 14 tests) is tracked under z9gh.7's mandate-replay scope, not run/claimed against z9gh.3's own AC4 wording (the specific flow list: resume/postmortem/decision/failure/file-touch/cost/coordinator-child/model/material/orchestration/paging). No artifact in this bead's own delivery chain executes that exact drill.\n\nAC5 (DSL, structured-plan, and recipe forms lower to the same canonical plan; identical refs/rows/totals/errors) -- SATISFIED. PR #3330's query_ast_schema.py validates rather than re-derives: predicate_to_ast()/explanation_payload_to_ast() run the existing predicate/pipeline dataclasses' own to_payload() through the AST schema (28 tests incl. round-trip + 2 drift-rejection tests). Recipe/corpus expressions parse through the identical production route (test_every_positive_example_parses_through_the_real_production_route uses compile_expression/parse_unit_source_expression directly, no mock grammar).\n\nAC6 (explain/catalog exposes estimated rows, selective predicates, joins/expensive relations, snapshot/freshness, next-narrowing/async strategy; expensive-but-valid combinations stay answerable via queue/stream/page/spool, never rejected for size/cost) -- PARTIALLY SATISFIED. No hard cost-based rejection found anywhere in expression.py or mcp/ (matches the 2026-07-15 contract correction). RESULT_SEMANTICS_TEACHING + the capability resource expose per-class total/continuation/teaching phrasing and per-example cost_class. But live per-query cardinality/freshness estimation at explain time was not found -- query_ast_schema's explain output carries structural AST, not row-count/staleness estimates. Overlaps AC2's gap.\n\nAC7 (adding/removing a field, origin mapping, or recipe updates every surface; a missing projection/registration/coverage/parity mapping fails one actionable check) -- PARTIALLY DEMONSTRATED, ACTIVELY PROVEN THIS SESSION. tests/unit/archive/query/test_discovery.py::test_declared_projection_columns_track_public_row_payload_models is exactly this class of anti-vacuity check, and it was RED on master going into this audit: PR #3296 (near/lineage execution materialize) added parent_refs/child_refs/continuation to SessionListRowPayload but never updated discovery.py's SESSION_COLUMNS declaration, so the parity check failed (`Right contains 3 more items, first extra item: 'parent_refs'`). Fixed via PR #3350 (feature/query/fix-session-projection-parity-z9gh3, this session) -- confirms the check mechanism works for the projection-column dimension, but there is no equivalent generated check for origin-mapping or recipe completeness specifically (render all --check covers doc/OpenAPI drift generally, not query-field-to-origin completeness).\n\nAC8 (no public description references only internal Python types/hidden docs; catalog queries bounded/paged; usage telemetry may evaluate recipes without becoming authority) -- SATISFIED. test_rows_are_typed_one_sentence_provider_neutral_and_privacy_safe enforces no provider names/paths/emails in descriptions. query_capabilities_resource explicitly bounds itself below MCP response budget (comment at mcp/server_resources.py:203) and delegates full corpus access to paged query_completions(kind=example|error).\n\nNET: 4 of 8 (AC1/AC3/AC5/AC8) fully satisfied with direct citations; AC6/AC7 partially satisfied; AC2 and AC4 have genuine unimplemented/unproven scope -- AC2's per-field authority/origin/freshness/cardinality/cost declaration + live 4-state coverage catalog was never built at the field level (only example- and unit-level facts exist), and AC4's own cold-model drill across this bead's named flow list has not been executed as z9gh.3's own closure evidence (adjacent proof lives under z9gh.7/#3334). This corroborates and sharpens the parent polylogue-z9gh 2026-07-27 full-AC audit's \"AC4: ADVANCED, FORMALLY OPEN\" line -- the gap is not merely that z9gh.3 hadn't been closed yet, it is that AC2/AC4 as literally written still have real remaining work. NOT CLOSING. Fixed one small, concretely-actionable gap discovered during this audit (PR #3350, the SESSION_COLUMNS/SessionListRowPayload drift) as in-session, safely-verifiable work; did not attempt AC2's field-level authority/coverage catalog or AC4's cold-model drill since both require non-trivial new design/implementation beyond a safe same-session fix.\n","status":"in_progress","priority":0,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T22:43:07Z","created_by":"Sinity","updated_at":"2026-07-27T20:04:40Z","started_at":"2026-07-17T11:44:58Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:mcp","area:orchestration","area:query","area:search","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-z9gh.3","depends_on_id":"polylogue-o21","type":"related","created_at":"2026-07-15T00:56:30Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.3","depends_on_id":"polylogue-o21.1","type":"relates-to","created_at":"2026-07-15T20:22:36Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.3","depends_on_id":"polylogue-t46.8","type":"related","created_at":"2026-07-15T00:47:13Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.3","depends_on_id":"polylogue-z9gh","type":"parent-child","created_at":"2026-07-15T00:43:06Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.3","depends_on_id":"polylogue-z9gh.9.1","type":"relates-to","created_at":"2026-07-15T19:19:42Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f7eef-48c9-7ded-bdaa-3d41008811be","issue_id":"polylogue-z9gh.3","author":"Sinity","text":"2026-07-20 investigation (fix/query/z9gh-execution-residuals, PR #3200): investigated both named execution-layer gaps in depth. near:id:/near:\"text\" -- traced full path in archive/query/archive_execution.py: current master already fails loudly (ExpressionCompileError) for near:id: with no vector backend/no seed embeddings (_session_seed_scored's documented contract), gracefully degrades to an EMPTY semantic leg (not unfiltered) for near:\"text\" with no vector provider (_semantic_hits's #1743 graceful-degradation contract), and _archive_summaries dispatches similar_session_id before the text-semantic leg correctly. near:id: inside a 'sessions where ...' scoping predicate at the query_units/DSL level raises ExpressionCompileError('not supported inside Boolean SQL predicates yet') -- also fail-loud. Could NOT reproduce the 'falls back to an unfiltered session list' symptom against current master through any traced path. A real gap found: no end-to-end CLI test for the near:id: (session-seeded) leg specifically (test_async_execute_query_archive_uses_vector_provider_for_semantic_search only covers near:\"text\"). Recommend re-verifying this specific finding against current master before further scheduling -- may already be fixed, same staleness class as the 2qx.2 finding the evidence matrix caught. lineage:id: recursive-page columns -- CONFIRMED REAL. lineage:id:X correctly filters session/message membership (QueryLineagePredicate/_lineage_predicate_clause), but the discovery corpus (archive/query/discovery.py RECURSIVE_COLUMNS = session_id/parent_refs/child_refs/continuation) declares 'recursive-page' semantics that the route never materializes -- it returns a flat non-recursively-paged row set, not a graph walk with parent/child refs and a recursive continuation cursor. Building real recursive-graph pagination is a genuinely large, separate feature (new response shape, cycle-aware traversal, continuation state) -- not sized S/M, so not attempted here per the coordinator's stop condition. Recommend splitting into its own properly-scoped feature bead if still wanted, distinct from this bead's declaration-generation scope.","created_at":"2026-07-20T09:50:39Z"}],"dependency_count":0,"dependent_count":1,"comment_count":1} {"_type":"issue","id":"polylogue-z9gh.2","title":"Eliminate archive-wide materialization in action and delegation queries","description":"On the live 4.85-million-block index, EXPLAIN QUERY PLAN for a delegation query constrained to one coordinator and LIMIT 10 still materializes the global actions ranked-use and ranked-result CTEs, resolved children, counts, and multiple temporary B-trees before the outer session predicate and limit apply. The actions view also backs ordinary tool, action-text, and referenced-path filters, explaining why a simple Workflow-tool session query stalled. This is the principal SQL cause implicated by the 8.5 GiB MCP incident.","design":"Replace the global windowed views with stored rebuildable derived relations in the next batched index-schema window. Add action_pairs keyed by tool_use_block_id, carrying session/message identity, tool id and per-id ordinal, normalized tool fields, paired result identity/outcome, and indexes for session, tool, semantic type, path, outcome, and transcript order. Recompute pairs only for sessions changed by the current write transaction after messages and blocks land; full replacement deletes and rebuilds that session cohort atomically. Preserve duplicate-tool-id semantics by ranking uses and results inside one session during rebuild, and preserve null or empty tool ids as explicitly unpaired rows. Keep public actions as a compatibility view that is a simple projection over action_pairs. Add delegation_facts keyed by stable dispatch or edge identity and refresh only affected parent sessions when action pairs or session_links change; preserve resolved, unresolved, ambiguous, quarantined, and edge-only states without global count CTEs. Keep public delegations as a simple projection. Query-unit lowering and SessionQueryPlan action, tool, path, sequence, outcome, and delegation filters target these indexed relations before joining or hydrating sessions. Primary anchors: polylogue/storage/sqlite/archive_tiers/index.py actions and delegations DDL; archive.py structural action/delegation lowerers; storage session write/full-replace and queries/session_links.py resolution; archive/query/retrieval_candidates.py. Do not treat temp_store, memoization, or a view-local WHERE wrapper as the fix.","acceptance_criteria":"1. The derived index contains action_pairs and delegation_facts with declared keys, foreign-key or equivalent replacement cleanup, and indexes for every selective predicate used by production query lowering; actions and delegations remain compatibility views with no window, archive-wide count, or grouping CTE. 2. Session save, full replacement, delete, child-before-parent link resolution, later parent arrival, link quarantine, and repaired link transitions atomically rebuild only affected session or parent cohorts and are idempotent after crash/retry. 3. Duplicate tool ids pair the Nth use with the Nth result within one session; missing results, null or empty ids, result-before-use source ordering, variants, retries, unresolved dispatches, ambiguous cardinality, quarantined cycles, and edge-only children match the current semantic goldens exactly. 4. EQP for one-session action/delegation, tool, path, action-text, outcome, and sequence queries starts from a selective action_pairs or delegation_facts index and contains no global ranked window or temp grouping. Restoring either current view makes the plan assertion fail. 5. The known coordinator first page, tool:Workflow session selection, and grouped failed-action example run through query_units and MCP inside the declared live-scale SLO with measured rows visited, elapsed time, RSS/PSS/swap, and temp bytes. 6. Broad aggregate queries remain exact and bounded by the shared query transaction; per-session materialization does not introduce a hidden row cap or N-plus-one hydration. 7. The index schema bump is batched with other ready index-tier additions, topology/generated surfaces are regenerated, focused action/delegation/write/link tests and benchmark mutations pass, and z9gh.9.1 consumes these relations without a second pairing implementation.","notes":"[2026-07-15 class consolidation] This is the planner/selectivity slice of polylogue-z9gh.9. The selective-plan invariant is enforced through the shared query transaction receipt and live-scale SLO harness rather than as a delegation-only optimization.\n[2026-07-15 invariant-collapse pass] Absorbs polylogue-7i4j: the four-minute documented grouped-actions example is another regression of the same globally materializing actions relation, not a separate performance project.\nInvariant consolidation 2026-07-15: absorbs polylogue-20d.10. Action category, referenced-path, and sequence predicates must lower into the same selective indexed action relation before hydration; per-session semantic-fact memoization alone is an explicitly insufficient partial fix.\nTerra-readiness correction 2026-07-15: resolved the previous mechanism fork. Implement stored rebuildable action_pairs and delegation_facts in index.db, maintained per affected session/parent, with compatibility views. A parameterized wrapper over the existing global window views is not an acceptable alternative.\n2026-07-17 implementation-readiness audit: action_pairs/delegation_facts do not yet exist in current index DDL (INDEX_SCHEMA_VERSION=37 in storage/sqlite/archive_tiers/index.py); this remains the one correct derived-index implementation, not an optimization experiment. It is a derived-tier canonical-DDL/rebuild change: no migration chain. Before implementation, run the named exact-session action and delegation EQP baselines, preserve them as fixtures, then add the new relations and replace views only after writer/link transition ownership is identified. The only allowed rebuild lifecycle is canonical index replacement; action/delegation correctness must be tested through query_units plus a direct SQL oracle. Batch with any ready index-tier additions, but do not wait for unrelated P0 work.\n2026-07-17 PR #3018 implementation receipt: index.db now has scoped action_pairs and delegation_facts rebuildable projections, compatibility views, selective indexes, refresh triggers, duplicate-tool pairing, and bounded SQL aggregate lowering. Exact verification includes storage delegation tests, action-view EQP checks, multi-field aggregate tests, and the 67-test post-merge focused sweep. The authorized incident-scale resource receipt remains with z9gh.7.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T22:43:04Z","created_by":"Sinity","updated_at":"2026-07-20T05:58:10Z","started_at":"2026-07-17T11:44:58Z","closed_at":"2026-07-20T05:58:10Z","close_reason":"Mechanism scope complete: action_pairs/delegation_facts derived relations + indexes (index.py:483/:1179, v42), atomic per-cohort rebuild, pairing goldens (PR #3018). Two residuals carried EXPLICITLY into z9gh.7 notes (not evaporated): F-006/F-007 session-alias EQP residual (CLI actions where session.id predicate lands on joined alias, not result branch — no note claims it fixed) and the live SLO receipt.","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:delegation","area:perf","area:query","horizon:frontier","incident:memory"],"dependencies":[{"issue_id":"polylogue-z9gh.2","depends_on_id":"polylogue-20d.10","type":"supersedes","created_at":"2026-07-15T21:43:08Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.2","depends_on_id":"polylogue-20d.7","type":"related","created_at":"2026-07-15T00:47:03Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.2","depends_on_id":"polylogue-j2zz","type":"relates-to","created_at":"2026-07-15T06:25:43Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.2","depends_on_id":"polylogue-z9gh.9","type":"parent-child","created_at":"2026-07-15T00:54:24Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6407-61af-752c-bc9e-cf6d889de7fc","issue_id":"polylogue-z9gh.2","author":"Sinity","text":"[Dogfood 2026-07-15 / F-006, F-007] On the 36.7 GB live index, actions for one 13-tool session exceeded two seconds because ranked_results remained archive-wide. The same pairing with the session predicate inside both ranked CTEs returned 13 rows in 0.271 ms; direct block count was 0.062 ms. The real CLI actions where session.id and is_error count still exceeded 20 seconds because the predicate lands on the joined sessions alias and does not enter the result branch. File retrieval also has an independent semantic gap: modern Codex paths live inside nested orchestration envelopes. polylogue-j2zz owns that lowering; this bead owns selectivity.","created_at":"2026-07-15T04:27:13Z"},{"id":"019f7eee-f4b0-73e3-bfcf-192ea7caf849","issue_id":"polylogue-z9gh.2","author":"Sinity","text":"2026-07-20 PR #3200: F-006/F-007 residual root-caused and fixed. Root cause was NOT predicate lowering (_exact_session_ids_from_predicate already correctly narrows session.id: to an exact bound for both the plain-count and followup_class-needing paths) but physical: action_relation_select_sql (storage/sqlite/action_relation.py) had no way to force SQLite onto idx_blocks_session_position, and a fresh ArchiveStore-bootstrapped archive never seeds sqlite_stat1 (initialize_archive_tier has no ANALYZE step, unlike the separate connection-pool bootstrap in storage/sqlite/schema.py which does) -- so the planner defaulted to idx_blocks_type_tool (archive-wide scan) regardless of session selectivity. Fix pins INDEXED BY idx_blocks_session_position on every session-bounded blocks scan branch. New EQP regression test test_bounded_action_relation_plans_session_index_not_archive_wide_tool_scan (tests/unit/storage/test_archive_tiers_archive.py) proven anti-vacuous against the pre-fix code. Live SLO receipt (the other named residual) remains with z9gh.7 as before -- not claimed here.","created_at":"2026-07-20T09:50:17Z"}],"dependency_count":0,"dependent_count":2,"comment_count":2} {"_type":"issue","id":"polylogue-z9gh.1","title":"Make archive queries interruptible and resource-bounded","description":"During the failed live reconstruction, the write-role Polylogue MCP process consumed 16 minutes 13 seconds of CPU over 43 minutes 19 seconds, peaked at 8.5 GiB memory and 6.8 GiB swap, read 39 GiB, and wrote 16.1 GiB. Archive and repository methods are async in signature but execute synchronous SQLite work on the MCP event loop. General archive queries have no deadline, SQLite progress-handler cancellation, connection interrupt path, or admission control. A client timeout therefore does not reliably stop the underlying work and one pathological query can make the entire server unavailable.","design":"Implement the reusable execution-control layer consumed by z9gh.9.1, not an MCP-only timeout wrapper. Add polylogue/archive/query/execution_control.py with immutable QueryExecutionContext carrying call/query identity, monotonic deadline, cancellation event, workload class, admission weight, and ownership refs; QueryAdmissionController providing FIFO-within-class weighted fairness and explicit queued/retry state; and InterruptibleSQLiteRead running one query on a dedicated read-only sqlite3 connection in a worker thread. Reuse storage/sqlite/connection_profile.py for pragmas and tier attachment. Register a SQLite progress handler that checks cancellation/deadline, expose connection.interrupt to the async caller, and never share that connection with another active query. The coordinator owns reader, temp/spool, and cancellation cleanup through one async context manager. MCP disconnect/cancel in server_tools/server_support and HTTP disconnect/deadline in daemon/http translate into the same context. Query result paging/spool format remains z9gh.9.1; this slice supplies execution and ownership primitives plus receipts. Do not wrap synchronous archive work in an untracked task, use process-global hard refusal by estimated size, or reuse writer connections.","acceptance_criteria":"1. QueryExecutionContext, QueryAdmissionController, and InterruptibleSQLiteRead are production types in the archive query layer and are consumed by a real query_units/list path; MCP and HTTP do not define parallel deadline or cancellation state. 2. Active SQLite statements run off the event loop on dedicated read-only connections; cancellation, deadline, and client disconnect set the shared cancellation state and interrupt the exact connection. A deliberately expensive recursive or aggregate statement aborts within the measured cancellation SLO. 3. While that statement runs, MCP health/cancel and an unrelated cheap read complete within their interactive SLO. Event-loop heartbeat testing fails if synchronous SQLite returns to the server thread. 4. Weighted admission preserves FIFO within class, prevents one caller or class from starving cheap reads, exposes queue position/retry identity, and eventually admits valid large work; estimated size or cost never becomes a permanent semantic refusal. 5. Cancellation before admission, during SQLite, during page/spool production, on disconnect, and on worker failure releases admission permits, readers, progress handlers, temp files, cursors, and tasks exactly once. Repeated incident-scale calls return to the declared steady-state RSS/PSS/swap/temp envelope. 6. Execution receipts distinguish queued, admitted, running, completed, cancelled, timed-out, disconnected, resumed, and failed with safe query/plan refs and no raw sensitive expression leakage. 7. Focused execution-control, MCP cancellation, HTTP disconnect, fairness, leak, and live-scale tests pass; mutations removing progress checks, connection interrupt, worker offload, admission release, or cleanup ownership fail through production routes.","notes":"[2026-07-15 class consolidation] This is the execution-control slice of polylogue-z9gh.9. Cancellation, deadlines, off-event-loop execution, and admission control belong to the shared query transaction rather than MCP-specific wrappers.\nContract correction 2026-07-15: resource bounds protect the host and event loop; they are not semantic query limits. Replace permanent resource-refused behavior with fair queue/backpressure plus resumable delivery for valid requests.\nTerra-readiness correction 2026-07-15: named the execution primitives, dedicated connection/thread model, source anchors, and exact division from z9gh.9.1. The worker should implement this contract, not choose an event-loop/cancellation architecture.\n2026-07-16 GPT-Pro corpus adjudication: bounded-query packages ca9526ba0446 (A), 0a91ceaa6451 (B alternative), 2df5dc1c22e3 (selective-action placeholder), and 18dd421a9b9f (execution control) are fully identified. Retained requirements: no semantic resource refusal; dedicated read-only SQLite worker with progress-handler interrupt; cancellation/admission ownership; lossless advancing continuation; executable declaration-driven discovery; selective action/delegation plans. Master still has per-surface response_budget replacement/continuation paths and synchronous execution evidence; the packages do not establish a safe current-master patch. A is blocked/seeded here, B is a semantic alternative, and selective-action code is rejected as placeholder-heavy. Implement only through this bead and z9gh.9/.9.1; do not create another per-surface executor or registry.\n2026-07-17 implementation-readiness audit: the execution-control foundation is already merged in #2964 (fd7b35492), not a greenfield design. Current production anchors are polylogue/archive/query/execution_control.py (QueryExecutionContext, QueryAdmissionController, InterruptibleSQLiteRead, execute_archive_read[_sync]); API query_units at api/archive.py:2868-2955; MCP query_units at mcp/server_tools.py:269-337; HTTP query-units at daemon/http.py:3508-3527; focused witness tests/unit/archive/query/test_execution_control.py. Retain this bead as the shared-control closure: do not create another executor. The next implementer must first enumerate every read route that still bypasses these primitives, then either route it through the z9gh.9.1 transaction or explicitly classify it non-query. Existing witness thresholds are cancellation/deadline \u003c5s, cheap concurrent read \u003c1s, event-loop heartbeat gap \u003c0.75s, default deadline 120s; replace only with a measured incident-scale SLO receipt, never a semantic refusal. Residual proof is real MCP disconnect + HTTP client disconnect + repeated resource-return witness through the transaction, not another primitive-only test.\n\n2026-07-17 GPT-Pro testdiet-05 admission: the current-master reconciliation branch `feature/query/bounded-aggregate-progress` accepted the focused SQL-backed multi-field aggregate + shared execution-context propagation slice from campaign artifact `testdiet/results/testdiet-05/r01` (SHA-256 cad064d3c0c4cdfa3c221adf6a7a1000ce59dccf84a8b9944003bfd8c21350f4). Commit 0ce5316f6 applies cleanly on origin/master 9b801a7cc and passes 31 real execution-control/multi-aggregate tests plus ruff/strict-mypy. This is an additive z9gh.1 execution mechanism, not completion of z9gh.9.1: snapshot-bound result refs, owned resumable spooling, CLI lifecycle migration, source-tier interruption, and live incident-scale receipts remain with the existing transaction program.\n2026-07-17 Test Diet 05 r02 was acquired and tar-readable but STATUS is PARTIAL: no reconstructed repository, no patch/changed-file payload, and no executable test transcript. It is retained as failed-delivery evidence only; it changes neither the verified PR #3012 bounded-aggregate slice nor the remaining shared resumable query-transaction scope.\n2026-07-17 PR #3018 implementation receipt: shared QueryExecutionContext/QueryAdmissionController/InterruptibleSQLiteRead now owns bounded off-event-loop reads, cancellation/deadline/worker cleanup, and MCP/HTTP/API query_units integration. Exact verification: devtools verify --quick; focused post-merge 67 passed; affected-area sweep 1030 passed, 1 skipped, 1 deselected. The private live-scale SLO and terminal incident replay remain with z9gh.7; this note does not claim those measurements.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T22:43:00Z","created_by":"Sinity","updated_at":"2026-07-20T05:58:09Z","started_at":"2026-07-17T11:44:58Z","closed_at":"2026-07-20T05:58:09Z","close_reason":"Evidence matrix 2026-07-20: all ACs landed and tested — QueryExecutionContext/AdmissionController/InterruptibleSQLiteRead in archive/query/execution_control.py, consumed by api/archive.py query_units + mcp/server_tools.py + daemon/http.py (PRs #2964/#3018); cancellation/fairness/receipts pinned by test_execution_control.py; live-scale receipts correctly deferred to z9gh.7.","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:mcp","area:perf","area:query","horizon:frontier","incident:memory"],"dependencies":[{"issue_id":"polylogue-z9gh.1","depends_on_id":"polylogue-1xc.14","type":"relates-to","created_at":"2026-07-15T20:45:46Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.1","depends_on_id":"polylogue-20d.14","type":"related","created_at":"2026-07-15T00:46:57Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.1","depends_on_id":"polylogue-oxz","type":"related","created_at":"2026-07-15T00:47:00Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh.1","depends_on_id":"polylogue-z9gh.9","type":"parent-child","created_at":"2026-07-15T00:54:21Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6a73-5396-7ba9-a7d5-21379b656160","issue_id":"polylogue-z9gh.1","author":"Sinity","text":"dogfood-2 static investigation (investigations/z9gh1-resource-mechanism.md, F-032; no live query run -- static source reading was judged conclusive, consistent with preferring static analysis over runtime proof where source alone settles the question): located the exact unbounded-materialization mechanism behind this beads incident. _all_aggregate_rows (archive/query/unit_results.py:259-278) -- the sole executor behind every unit where ... | group by ... | count pipeline across all four read surfaces (CLI find, MCP query_units/aggregate_sessions, daemon HTTP /api/query-units, Python API; confirmed as one shared executor per the modules own #2006 comment) -- loops through ALL matching pages via manual limit/offset stepping with no upper bound on total accumulated rows, materializing the entire matching result set as live Python objects before any grouping/counting happens. The sibling rows-terminal executor (_execute_rows_terminal) is correctly single-page-bounded by contrast -- this is specific to the aggregate/count path, not the whole executor stack. Applies to all six unit types with a sql_query_method (messages, actions, blocks, assertions, files, runs), not one query shape. This beads problem statement is already accurate and does not need updating; add this as an implementation-level design note: the planned InterruptibleSQLiteRead primitive alone would not fix this specific loop, since interruption happens at the SQLite-statement level while the accumulation loop itself needs either a hard row cap or the aggregation pushed into SQL (GROUP BY/COUNT(*) server-side, never materializing matched rows in Python).","created_at":"2026-07-16T10:22:51Z"}],"dependency_count":0,"dependent_count":1,"comment_count":1} -{"_type":"issue","id":"polylogue-z9gh","title":"Restore mandate-critical archive queryability under real agent workloads","description":"A live continuity task failed even though the archive contained the needed evidence. Starting from a Polylogue repo, the current day, and knowledge that roughly 16 Claude Code agents had worked on concerns, the model could not reliably identify the coordinator, enumerate workers, reconstruct outcomes, or reconcile Beads and git effects. Correct query routes either erased successful results at the 25 KiB response boundary or expanded archive-wide views until the MCP process reached an 8.5 GiB memory peak plus 6.8 GiB swap. The affected live archive held 18,428 sessions and 4.85 million blocks. This is a failure of the core product promise: heterogeneous agent history exists but is not practically queryable by an agent.","design":"Recover the mandate through four reusable mechanisms rather than a symptom queue. (A) Query transaction: polylogue-z9gh.9 unifies canonical planning, bounded off-loop execution, cancellation, paging/result refs, stable frame/order, telemetry, and selective-plan enforcement; rsad, t46.3, rxdo.3, and the memory incident become slices/regressions. (B) Source admission: OriginSpec polylogue-2qx declares artifact inventory, detection/parsing, identity, provenance authority, normalized constructs, coverage, and reparse policy; Claude Workflow sidecars and false human authorship are regression leaves. (C) Work-evidence graph: polylogue-1vpm extends existing ProjectedRun/ObservedEvent/ObjectRef/delegation machinery to tasks/calls/attempts/sessions/claims/artifacts/git/PR/Beads effects; Workflow normalization and outcome reconciliation are adapters/projections. (D) Agent query declaration: polylogue-z9gh.3 generates executable discovery, structured plans, DSL teaching, compact projections, and errors from one registry. polylogue-z9gh.7 is the sole terminal black-box gate. Do not add another incident child unless it disproves one of these class contracts or requires a genuinely different identity, lifecycle, authority, access shape, or durability tier.","acceptance_criteria":"1. The shared query transaction makes all archive reads bounded, cancellable, losslessly resumable, stable-order/frame aware, and resource measured across every surface. 2. OriginSpec makes source artifact coverage and authority-bearing normalization rules executable and completeness-checked. 3. The work-evidence graph traverses provider tasks/runs/attempts/session segments, claims, artifacts, commits, PRs, and Beads effects without task=session or claim=truth assumptions. 4. Agent query discovery and structural plans are generated from executable declarations; a cold model succeeds without hidden docs. 5. The mandate replay starts from sparse repo/time/parallel-work clues, enumerates all matching workers exactly once, reconstructs models/attempts/results and cited effects, and explains the unchanged P1 set. 6. The seven core flows pass within declared latency/memory/cancellation envelopes. 7. Every residual symptom is mapped to one class mechanism or justified as a separate contract before this epic closes.","notes":"[2026-07-15 tractability pass] Reorganized after operator concern that too many critical symptoms were obscuring class fixes. The stop-the-line program now has four mechanisms and one gate. Concrete beads remain as regression evidence and implementation slices; they are not invitations to build parallel subsystems.\n2026-07-17 implementation checkpoint: PR #3049 / 9ddbf6ce4 adds bounded MCP/API/HTTP transactions, selective action paging, fail-closed repair/scale proofs, OriginSpec Claude artifact admission, generic work graph, direct Claude projection, and claim/effect reconciliation primitives. Focused routes: storage 84 passed, transaction+MCP 86 passed, scale proof 19 passed; quick verifier 16/16. NOT TERMINAL: production git/GitHub/Beads effect adapters, source-to-graph incident materialization, and the real paged/cancelled/cold-model MCP continuity replay remain required; do not close z9gh from this checkpoint.\n2026-07-27 gap-#1 slice: PR #3327 (branch feature/fix/github-pr-effect-adapter) implements the GitHub half of remaining gap #1 (\"production git/GitHub/Beads effect adapters\"). Git (GitCommitEffectAdapter) and Beads (BeadsIssueEffectAdapter) adapters were already production-real per the #3049 checkpoint; GitHubPullRequestEffectAdapter was a typed stub whose collect() always raised EffectAdapterUnavailableError (confirmed by polylogue-1vpm.6.2's investigation: no GitHub adapter existed at all). It now shells out to `gh pr list --repo \u003cowner/name\u003e --state all --json ...` (same subprocess pattern as insights.correlation_view._enrich_with_github_api), turning each PR into one ObservedRepositoryEffect (number/title/body/state/merge-commit) that derive_direct_identifier_judgments can link to claims via shared work-item id, exactly like the git/Beads paths. Wired into the reconcile-work-effects CLI via new --github-repo option. All failure modes (missing gh binary, auth/network non-zero exit, timeout, malformed JSON) raise EffectAdapterUnavailableError explicitly -- never a silent empty result. 23 focused tests pass (tests/unit/insights/test_work_effects.py + tests/unit/cli/test_reconcile_work_effects_command.py), mypy --strict clean, ruff clean, devtools render all --check clean, devtools verify --quick exit 0. Per-reviewer PR review/approval granularity intentionally deferred (polylogue-1vpm.6.2). NOT CLOSING z9gh from this PR: gap #1 is now fully real for git+github+beads, but gap #2 (source-to-graph incident materialization) and gap #3 (real paged/cancelled/cold-model MCP continuity replay) remain completely untouched.\n2026-07-27 full-AC audit (independent, evidence-first; bead read chronologically, all children/deps re-verified via `bd show --json` + `git log --grep` on this exact worktree, not summary text alone). VERDICT: STAYS OPEN. Real, substantial, independently-verified progress landed today (PRs #3327/#3336/#3334 closing the \"3 named gaps\" checkpoint), but the epic's own 7-item AC is broader than those 3 gaps and genuine scope remains open on 4 of 7 items. bd's own computed epic_closeable=false (epic_total_children=4, epic_closed_children=2) corroborates this independently of my reading.\n\nPer-AC verdict with citations:\n\nAC1 (bounded/cancellable/resumable/frame-aware/measured query transactions across every surface) -- PARTIAL. z9gh.9 (closed, core transaction unification, PR #3049: storage 84 passed / transaction+MCP 86 passed / scale-proof 19 passed / quick-verifier 16/16), t46.3 (closed, unify list/search query-spec execution CLI/MCP/daemon), and rsad (closed, MCP ergonomics) are real and closed. But rxdo.3 (\"query-run telemetry with frame/evaluation-world/privacy contracts\") is open -- the \"resource measured\"/\"frame aware\" half of this AC's own wording is not fully landed as a contract. t46 (\"delete parallel dispatch and the QA middle layer\") and t46.8 (\"replace MCP tool sprawl with protocol-native verb algebra\") are open -- \"across every surface\" is not yet true while parallel dispatch paths and tool sprawl remain. 20d (interactive-performance front door) and 1xc (scale-hardening bugs) are open -- the class of scale-only bugs that caused the original 8.5 GiB/6.8 GiB swap incident is not yet closed as a class.\n\nAC2 (OriginSpec source coverage + authority normalization executable and completeness-checked) -- MANDATE SLICE DONE, GENERAL CONTRACT OPEN. polylogue-2qx.1 (kernel + origin vocabulary migration) and polylogue-2qx.2 (admit Claude Code orchestration artifacts -- the actual mandate-critical slice, landed via PR #3088) are both closed. But the parent epic polylogue-2qx itself (10-item AC: \"OriginSpec is the sole executable source-admission contract... drives dispatch, completeness, coverage/readiness... for EVERY origin\") remains open -- completeness-checking is proven for Claude Code, not generalized to every origin/artifact kind yet.\n\nAC3 (work-evidence graph traverses tasks/runs/attempts/claims/artifacts/commits/PRs/Beads effects without task=session or claim=truth assumptions) -- SUBSTANTIALLY ADVANCED, CORE STILL OPEN. Real, independently-verified landings: polylogue-1vpm.6.2 (closed) delivered production git/GitHub/Beads effect adapters -- confirmed via `git show --stat` on e3f02a611 (#3327, GitHub adapter via `gh pr list`, 23 focused tests, mypy/ruff/render clean) and the pre-existing git+Beads adapters from #3049. PR 5e50a94d7 (#3336) delivered real source-to-graph incident materialization -- confirmed via diff: new `polylogue/insights/incident_evidence_materialization.py` (267 lines) + CLI command wiring `node_from_projected_run`/`session_segment_from_observed_events` (previously zero non-test callers per the PR's own grep-verified claim) into production, not just tests. However polylogue-1vpm.6.1 (\"Land the provider-neutral work topology and claim graph\" -- the actual graph-construction core, distinct from .6.2's reconciliation-only scope) is still OPEN, as is its parent landing bead polylogue-1vpm.6 and extension units polylogue-1vpm.2 (episode unit), .4 (turn-pair unit), .5 (correction-edge unit). The literal AC3 text (\"traverses... without task=session/claim=truth assumptions\") describes the graph itself, which is not yet fully landed even though its effect-observation and incident-materialization adapters are real.\n\nAC4 (agent query discovery generated from executable declarations; cold model succeeds without hidden docs) -- ADVANCED, FORMALLY OPEN. polylogue-z9gh.3 is `in_progress`, not closed. Its own notes (read in full, dated 2026-07-15 through 2026-07-27) show a long real delivery chain: PR #3018 (executable MCP discovery/grammar/coverage), #3066 (106-row parser-verified query-discovery corpus, 3 fixed shipped-invalid examples), #3296 (near:id: similarity ranking + lineage:id: recursive-page column materialization, merged, with a self-caught OR/AND boolean-recursion bug fixed before merge), #3330 (structured-plan -\u003e canonical AST lowering + OpenAPI generation, 28 new tests, 751 passed/1 skipped on existing suite). The 2026-07-27 note on #3330 states explicitly: \"this closes out the last named residual on this bead's scope per the 2026-07-20 narrowing note; the program bead itself (z9gh.3) may still have broader open scope beyond these two named residuals -- not closing it here, leaving that call to the operator/triage.\" Separately, gap #3's cold-model MCP continuity replay proof (PR #3334, 74c2f3884) independently demonstrates a non-scripted, discovery-driven planner formulating+executing+cancelling a real paged MCP route end to end (14 tests passed) -- strong supporting evidence for AC4's \"cold model succeeds\" property, but this is a devtools proof harness, not z9gh.3's own closure.\n\nAC5 (the mandate replay itself: sparse-clue enumeration, exact reconstruction, unchanged-P1-set explanation) -- NOT SATISFIED, CONCRETE BLOCKER NAMED. polylogue-z9gh.7 (the design doc's own \"sole terminal black-box gate\") is OPEN. PR #3328 (devtools/mandate_continuity_replay.py) landed real production wiring: replays the full t8t CONTINUITY_SCENARIOS catalog via real MCP stdio, cross-checks discovery-catalog coverage, and reconciles Beads-closed claims against real git history through the real (not stub) GitCommitEffectAdapter/BeadsIssueEffectAdapter/GitHubPullRequestEffectAdapter. But its own AC matrix states honestly: AC2 of z9gh.7 (the live 2026-07-15 incident replay itself -- finding real coordinator cf0c6474-da22-44be-af3e-666037aa5ea4 and run wf_54d4fb2e-841 in the actual promoted archive) \"needs an authorized live archive this sandbox does not have -- deferred, not fabricated, with an explicit --archive-root re-run path noted.\" This is a genuine, named, structural blocker: it requires (a) a promoted/live production archive and (b) operator authorization to run against it -- exactly the category of blocker this task instructed me to name rather than attempt to fabricate or route around.\n\nAC6 (seven core flows pass within declared latency/memory/cancellation envelopes) -- NOT SATISFIED, SAME LIVE-ENVIRONMENT BLOCKER. polylogue-4s3c (\"verify execution-control steady-state resource envelope at live scale\") is OPEN; its own AC requires \u003e=20 repeated incident-scale scan rounds against the live-scale archive returning to a declared RSS/PSS/swap envelope -- this cannot be measured without live archive access. polylogue-20d (interactive performance) and polylogue-1xc (scale-hardening bugs) are also open.\n\nAC7 (every residual symptom mapped to one class mechanism or justified as separate) -- PARTIALLY DEMONSTRATED, NOT YET COMPLETE. Positive evidence the discipline is working: polylogue-3k30 (a brand-new symptom the gap-#3 harness discovered: `explain(subject=\"result\")` overflows the 25KB MCP budget with a non-narrowing, argument-dropping continuation) was correctly triaged as a real, separately-tracked class-consistent defect, NOT smoothed over, and has since been fixed for real via PR #3342 (generalized `_bounded_item_page`/threaded call arguments through `_safe_call`/`explain()` offset param) and closed with a verified pre/post-fix regression test. But AC7 cannot be called satisfied epic-wide while 1xc/20d/rxdo.3/t46/t46.8/1vpm.6.1/2qx (general epic) remain open and unclassified as either closed or explicitly justified as a separate non-blocking contract.\n\nCONCRETE REMAINING-GAP SUMMARY for the next agent/session (do not re-derive this from scratch):\n- Tractable in a sandbox (no live archive/operator auth needed): polylogue-1vpm.6.1 (core work-topology/claim graph landing -- the single most load-bearing remaining AC3 gap), polylogue-z9gh.3's own broader-scope-vs-close decision (explicitly deferred to operator/triage by its own last note), polylogue-2qx's generalization beyond the Claude Code mandate slice, polylogue-t46/t46.8/rxdo.3 (surface consolidation + telemetry contracts for full AC1).\n- Blocked on live archive + operator authorization (do NOT attempt to fabricate a substitute): polylogue-z9gh.7's AC2 (the actual 2026-07-15 incident replay against the promoted archive, reconstructing wf_54d4fb2e-841/coordinator cf0c6474...), polylogue-4s3c (live-scale steady-state envelope measurement). These are the two genuine, named, unavoidable operator-gated blockers on z9gh's own closure.\n\nThis sandbox worktree (/realm/project/polylogue/.claude/worktrees/agent-a9e569f4d3aae271c) has no promoted live archive and no operator present to authorize a live-scale run, so AC5/AC6's remaining scope could not be attempted here regardless of effort -- correctly deferring rather than fabricating evidence. No code changes made this session; this was a pure audit. z9gh remains OPEN.\n\n2026-07-27/28 convergence session — cumulative status update (honest, not a closure claim):\n\nThis P0 epic's dependency tree is a genuine multi-month architectural\nprogram (60+ nested beads under 1vpm -\u003e h6r -\u003e ovme -\u003e kwsb -\u003e 20d -\u003e 1xc\n-\u003e b5l -\u003e 60i5 -\u003e cuxz -\u003e rxdo -\u003e 4p1 -\u003e ... chains), spanning: work-evidence\ngraph modeling, actor/execution-context identity, MCP tool consolidation,\nfree-threading deploy, durable-tier migration framework, query/projection\nalgebra unification, and a terminal black-box continuity-replay gate\n(z9gh.7). It is not honestly closable to \"pristine, zero debt\" in a single\nsession, and forcing a closure claim here would misrepresent the state.\n\nWhat DID land this session (real PRs, real tests, self-reviewed since\nCodeRabbit was rate-limited most of the session, all merged to master):\n- #3351 h6r: real ActorRef/ExecutionContextRef identity derivation (was\n planned but unbuilt) -- direct progress on the 1vpm.6.1 -\u003e h6r chain.\n- #3360 ih67: TitleSource provenance threaded end-to-end (was computed but\n silently dropped on read surfaces).\n- #3361 20d.13: SSE events now carry real per-session identity (was\n session_id=None aggregate events) + removed 3 genuinely-dead event topics.\n- #3356 a7uk: reconcile-work-effects/materialize-incident-evidence CLI\n commands now route through AppEnv.repository (surface/storage boundary\n cleanup for the work-evidence graph surfaces).\n- #3357 lpen / #3358 de2a: fixed a live daemon bug where sibling\n full-revision-raw retirement quarantined an unrelated in-flight raw\n (blast-radius bug), plus a live-observed O(N)-per-append cost that caused\n 860s/9297s writer-gate holds.\n- #3362 da1: provider format-drift sentinel (classifies per-record schema\n drift during live ingest, surfaces risky/benign rates on daemon health +\n CLI status) -- new automatic-detection capability the archive-integrity\n side of this mandate needs.\n- #3363 lbgc: fixed a same-process SQLite zombie-connection lock race in\n seeded-archive test fixtures.\n- #3364 t0ta: split ChatGPT export detection into document/fragment tiers\n (tightened format-drift-risk detection without breaking legitimate\n per-line JSONL sniffing).\n- Re-confirmed polylogue-ihc8's fix (PR #3326, merged in a prior session)\n is real and correct via fresh read-only reproduction against the live\n archive; filed polylogue-dmvo for a genuine N:1 fan-out edge case the\n original fix doesn't yet resolve.\n- Found and surfaced a significant blocking infrastructure issue: GitHub\n Actions has been repo-wide disabled since 2026-07-13 due to a billing\n lock (CodeQL/Release/nix CI dark 14+ days) -- not a code bug, needs\n operator action at github.com/settings/billing (polylogue-n2f4 updated\n with this finding).\n\nWhat remains genuinely open and why, so nothing here is \"unaccounted for\":\n- z9gh.7 (the terminal black-box continuity-replay gate this epic exists to\n satisfy) depends on the FULL 1vpm work-evidence graph + t46.8 MCP\n consolidation + 4p1 query/projection algebra + hs3y agent-material\n acquisition -- each a multi-PR program in its own right, several still at\n \u003c50% implementation.\n- polylogue-de2a (writer-lock starvation under backlog): partial mitigation\n shipped (skip_already_applied fast path, FTS merge interval halved), but\n the real fix (writer-lock fairness/priority, or bounded per-append writer\n holds, or adaptive merge budget) is unstarted design work, correctly not\n attempted casually.\n- polylogue-2a6d (Borg backup coverage gap): narrowed from \"zero coverage\"\n to \"the direct filesystem-snapshot path is affected; the\n polylogue-sqlite-backup.service content-level path already works and is\n Borg-covered\" -- confirmed via a fresh manual backup+integrity-check this\n session.\n- Several dozen P1 items in the tree (kwsb.2 MutationTransaction, t46.8.2\n MCP verb migration, ovme ArchiveLocation, 20d.17 status snapshots, 4p1\n query algebra, rxdo.2 evidence privacy classification, etc.) remain\n genuinely large, multi-week design efforts -- checked this session,\n correctly not force-attempted as quick slices (avoids the\n #3354/#3359-style duplicate-work pattern already seen twice this session\n when two independent agents fixed the same bug in parallel).\n\nNo bead in this tree was closed dishonestly to satisfy an external\ncompletion pressure. This note exists so the epic's actual state is fully\ntraceable rather than silently stale.\n","status":"open","priority":0,"issue_type":"epic","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T22:42:56Z","created_by":"Sinity","updated_at":"2026-07-27T22:12:03Z","metadata":{"frontier_program":"active"},"labels":["area:mandate","area:mcp","area:perf","area:query","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-1vpm","type":"related","created_at":"2026-07-15T00:55:32Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-1xc","type":"related","created_at":"2026-07-15T00:46:50Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-20d","type":"related","created_at":"2026-07-15T00:46:47Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-2qx","type":"related","created_at":"2026-07-15T00:55:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-rsad","type":"related","created_at":"2026-07-15T00:46:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t46","type":"related","created_at":"2026-07-15T00:46:54Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t46.3","type":"related","created_at":"2026-07-15T00:46:40Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t46.8","type":"related","created_at":"2026-07-15T00:46:43Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t8t","type":"related","created_at":"2026-07-15T00:46:37Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-z9gh","title":"Restore mandate-critical archive queryability under real agent workloads","description":"A live continuity task failed even though the archive contained the needed evidence. Starting from a Polylogue repo, the current day, and knowledge that roughly 16 Claude Code agents had worked on concerns, the model could not reliably identify the coordinator, enumerate workers, reconstruct outcomes, or reconcile Beads and git effects. Correct query routes either erased successful results at the 25 KiB response boundary or expanded archive-wide views until the MCP process reached an 8.5 GiB memory peak plus 6.8 GiB swap. The affected live archive held 18,428 sessions and 4.85 million blocks. This is a failure of the core product promise: heterogeneous agent history exists but is not practically queryable by an agent.","design":"Recover the mandate through four reusable mechanisms rather than a symptom queue. (A) Query transaction: polylogue-z9gh.9 unifies canonical planning, bounded off-loop execution, cancellation, paging/result refs, stable frame/order, telemetry, and selective-plan enforcement; rsad, t46.3, rxdo.3, and the memory incident become slices/regressions. (B) Source admission: OriginSpec polylogue-2qx declares artifact inventory, detection/parsing, identity, provenance authority, normalized constructs, coverage, and reparse policy; Claude Workflow sidecars and false human authorship are regression leaves. (C) Work-evidence graph: polylogue-1vpm extends existing ProjectedRun/ObservedEvent/ObjectRef/delegation machinery to tasks/calls/attempts/sessions/claims/artifacts/git/PR/Beads effects; Workflow normalization and outcome reconciliation are adapters/projections. (D) Agent query declaration: polylogue-z9gh.3 generates executable discovery, structured plans, DSL teaching, compact projections, and errors from one registry. polylogue-z9gh.7 is the sole terminal black-box gate. Do not add another incident child unless it disproves one of these class contracts or requires a genuinely different identity, lifecycle, authority, access shape, or durability tier.","acceptance_criteria":"1. The shared query transaction makes all archive reads bounded, cancellable, losslessly resumable, stable-order/frame aware, and resource measured across every surface. 2. OriginSpec makes source artifact coverage and authority-bearing normalization rules executable and completeness-checked. 3. The work-evidence graph traverses provider tasks/runs/attempts/session segments, claims, artifacts, commits, PRs, and Beads effects without task=session or claim=truth assumptions. 4. Agent query discovery and structural plans are generated from executable declarations; a cold model succeeds without hidden docs. 5. The mandate replay starts from sparse repo/time/parallel-work clues, enumerates all matching workers exactly once, reconstructs models/attempts/results and cited effects, and explains the unchanged P1 set. 6. The seven core flows pass within declared latency/memory/cancellation envelopes. 7. The residual-symptom set ENUMERATED IN THIS BEAD'S NOTES AT CLOSURE TIME is each mapped to one class mechanism or split into a named successor bead; the enumeration is a finite list captured from a stated archive snapshot, not an open-ended sweep.","notes":"[2026-07-15 tractability pass] Reorganized after operator concern that too many critical symptoms were obscuring class fixes. The stop-the-line program now has four mechanisms and one gate. Concrete beads remain as regression evidence and implementation slices; they are not invitations to build parallel subsystems.\n2026-07-17 implementation checkpoint: PR #3049 / 9ddbf6ce4 adds bounded MCP/API/HTTP transactions, selective action paging, fail-closed repair/scale proofs, OriginSpec Claude artifact admission, generic work graph, direct Claude projection, and claim/effect reconciliation primitives. Focused routes: storage 84 passed, transaction+MCP 86 passed, scale proof 19 passed; quick verifier 16/16. NOT TERMINAL: production git/GitHub/Beads effect adapters, source-to-graph incident materialization, and the real paged/cancelled/cold-model MCP continuity replay remain required; do not close z9gh from this checkpoint.\n2026-07-27 gap-#1 slice: PR #3327 (branch feature/fix/github-pr-effect-adapter) implements the GitHub half of remaining gap #1 (\"production git/GitHub/Beads effect adapters\"). Git (GitCommitEffectAdapter) and Beads (BeadsIssueEffectAdapter) adapters were already production-real per the #3049 checkpoint; GitHubPullRequestEffectAdapter was a typed stub whose collect() always raised EffectAdapterUnavailableError (confirmed by polylogue-1vpm.6.2's investigation: no GitHub adapter existed at all). It now shells out to `gh pr list --repo \u003cowner/name\u003e --state all --json ...` (same subprocess pattern as insights.correlation_view._enrich_with_github_api), turning each PR into one ObservedRepositoryEffect (number/title/body/state/merge-commit) that derive_direct_identifier_judgments can link to claims via shared work-item id, exactly like the git/Beads paths. Wired into the reconcile-work-effects CLI via new --github-repo option. All failure modes (missing gh binary, auth/network non-zero exit, timeout, malformed JSON) raise EffectAdapterUnavailableError explicitly -- never a silent empty result. 23 focused tests pass (tests/unit/insights/test_work_effects.py + tests/unit/cli/test_reconcile_work_effects_command.py), mypy --strict clean, ruff clean, devtools render all --check clean, devtools verify --quick exit 0. Per-reviewer PR review/approval granularity intentionally deferred (polylogue-1vpm.6.2). NOT CLOSING z9gh from this PR: gap #1 is now fully real for git+github+beads, but gap #2 (source-to-graph incident materialization) and gap #3 (real paged/cancelled/cold-model MCP continuity replay) remain completely untouched.\n2026-07-27 full-AC audit (independent, evidence-first; bead read chronologically, all children/deps re-verified via `bd show --json` + `git log --grep` on this exact worktree, not summary text alone). VERDICT: STAYS OPEN. Real, substantial, independently-verified progress landed today (PRs #3327/#3336/#3334 closing the \"3 named gaps\" checkpoint), but the epic's own 7-item AC is broader than those 3 gaps and genuine scope remains open on 4 of 7 items. bd's own computed epic_closeable=false (epic_total_children=4, epic_closed_children=2) corroborates this independently of my reading.\n\nPer-AC verdict with citations:\n\nAC1 (bounded/cancellable/resumable/frame-aware/measured query transactions across every surface) -- PARTIAL. z9gh.9 (closed, core transaction unification, PR #3049: storage 84 passed / transaction+MCP 86 passed / scale-proof 19 passed / quick-verifier 16/16), t46.3 (closed, unify list/search query-spec execution CLI/MCP/daemon), and rsad (closed, MCP ergonomics) are real and closed. But rxdo.3 (\"query-run telemetry with frame/evaluation-world/privacy contracts\") is open -- the \"resource measured\"/\"frame aware\" half of this AC's own wording is not fully landed as a contract. t46 (\"delete parallel dispatch and the QA middle layer\") and t46.8 (\"replace MCP tool sprawl with protocol-native verb algebra\") are open -- \"across every surface\" is not yet true while parallel dispatch paths and tool sprawl remain. 20d (interactive-performance front door) and 1xc (scale-hardening bugs) are open -- the class of scale-only bugs that caused the original 8.5 GiB/6.8 GiB swap incident is not yet closed as a class.\n\nAC2 (OriginSpec source coverage + authority normalization executable and completeness-checked) -- MANDATE SLICE DONE, GENERAL CONTRACT OPEN. polylogue-2qx.1 (kernel + origin vocabulary migration) and polylogue-2qx.2 (admit Claude Code orchestration artifacts -- the actual mandate-critical slice, landed via PR #3088) are both closed. But the parent epic polylogue-2qx itself (10-item AC: \"OriginSpec is the sole executable source-admission contract... drives dispatch, completeness, coverage/readiness... for EVERY origin\") remains open -- completeness-checking is proven for Claude Code, not generalized to every origin/artifact kind yet.\n\nAC3 (work-evidence graph traverses tasks/runs/attempts/claims/artifacts/commits/PRs/Beads effects without task=session or claim=truth assumptions) -- SUBSTANTIALLY ADVANCED, CORE STILL OPEN. Real, independently-verified landings: polylogue-1vpm.6.2 (closed) delivered production git/GitHub/Beads effect adapters -- confirmed via `git show --stat` on e3f02a611 (#3327, GitHub adapter via `gh pr list`, 23 focused tests, mypy/ruff/render clean) and the pre-existing git+Beads adapters from #3049. PR 5e50a94d7 (#3336) delivered real source-to-graph incident materialization -- confirmed via diff: new `polylogue/insights/incident_evidence_materialization.py` (267 lines) + CLI command wiring `node_from_projected_run`/`session_segment_from_observed_events` (previously zero non-test callers per the PR's own grep-verified claim) into production, not just tests. However polylogue-1vpm.6.1 (\"Land the provider-neutral work topology and claim graph\" -- the actual graph-construction core, distinct from .6.2's reconciliation-only scope) is still OPEN, as is its parent landing bead polylogue-1vpm.6 and extension units polylogue-1vpm.2 (episode unit), .4 (turn-pair unit), .5 (correction-edge unit). The literal AC3 text (\"traverses... without task=session/claim=truth assumptions\") describes the graph itself, which is not yet fully landed even though its effect-observation and incident-materialization adapters are real.\n\nAC4 (agent query discovery generated from executable declarations; cold model succeeds without hidden docs) -- ADVANCED, FORMALLY OPEN. polylogue-z9gh.3 is `in_progress`, not closed. Its own notes (read in full, dated 2026-07-15 through 2026-07-27) show a long real delivery chain: PR #3018 (executable MCP discovery/grammar/coverage), #3066 (106-row parser-verified query-discovery corpus, 3 fixed shipped-invalid examples), #3296 (near:id: similarity ranking + lineage:id: recursive-page column materialization, merged, with a self-caught OR/AND boolean-recursion bug fixed before merge), #3330 (structured-plan -\u003e canonical AST lowering + OpenAPI generation, 28 new tests, 751 passed/1 skipped on existing suite). The 2026-07-27 note on #3330 states explicitly: \"this closes out the last named residual on this bead's scope per the 2026-07-20 narrowing note; the program bead itself (z9gh.3) may still have broader open scope beyond these two named residuals -- not closing it here, leaving that call to the operator/triage.\" Separately, gap #3's cold-model MCP continuity replay proof (PR #3334, 74c2f3884) independently demonstrates a non-scripted, discovery-driven planner formulating+executing+cancelling a real paged MCP route end to end (14 tests passed) -- strong supporting evidence for AC4's \"cold model succeeds\" property, but this is a devtools proof harness, not z9gh.3's own closure.\n\nAC5 (the mandate replay itself: sparse-clue enumeration, exact reconstruction, unchanged-P1-set explanation) -- NOT SATISFIED, CONCRETE BLOCKER NAMED. polylogue-z9gh.7 (the design doc's own \"sole terminal black-box gate\") is OPEN. PR #3328 (devtools/mandate_continuity_replay.py) landed real production wiring: replays the full t8t CONTINUITY_SCENARIOS catalog via real MCP stdio, cross-checks discovery-catalog coverage, and reconciles Beads-closed claims against real git history through the real (not stub) GitCommitEffectAdapter/BeadsIssueEffectAdapter/GitHubPullRequestEffectAdapter. But its own AC matrix states honestly: AC2 of z9gh.7 (the live 2026-07-15 incident replay itself -- finding real coordinator cf0c6474-da22-44be-af3e-666037aa5ea4 and run wf_54d4fb2e-841 in the actual promoted archive) \"needs an authorized live archive this sandbox does not have -- deferred, not fabricated, with an explicit --archive-root re-run path noted.\" This is a genuine, named, structural blocker: it requires (a) a promoted/live production archive and (b) operator authorization to run against it -- exactly the category of blocker this task instructed me to name rather than attempt to fabricate or route around.\n\nAC6 (seven core flows pass within declared latency/memory/cancellation envelopes) -- NOT SATISFIED, SAME LIVE-ENVIRONMENT BLOCKER. polylogue-4s3c (\"verify execution-control steady-state resource envelope at live scale\") is OPEN; its own AC requires \u003e=20 repeated incident-scale scan rounds against the live-scale archive returning to a declared RSS/PSS/swap envelope -- this cannot be measured without live archive access. polylogue-20d (interactive performance) and polylogue-1xc (scale-hardening bugs) are also open.\n\nAC7 (every residual symptom mapped to one class mechanism or justified as separate) -- PARTIALLY DEMONSTRATED, NOT YET COMPLETE. Positive evidence the discipline is working: polylogue-3k30 (a brand-new symptom the gap-#3 harness discovered: `explain(subject=\"result\")` overflows the 25KB MCP budget with a non-narrowing, argument-dropping continuation) was correctly triaged as a real, separately-tracked class-consistent defect, NOT smoothed over, and has since been fixed for real via PR #3342 (generalized `_bounded_item_page`/threaded call arguments through `_safe_call`/`explain()` offset param) and closed with a verified pre/post-fix regression test. But AC7 cannot be called satisfied epic-wide while 1xc/20d/rxdo.3/t46/t46.8/1vpm.6.1/2qx (general epic) remain open and unclassified as either closed or explicitly justified as a separate non-blocking contract.\n\nCONCRETE REMAINING-GAP SUMMARY for the next agent/session (do not re-derive this from scratch):\n- Tractable in a sandbox (no live archive/operator auth needed): polylogue-1vpm.6.1 (core work-topology/claim graph landing -- the single most load-bearing remaining AC3 gap), polylogue-z9gh.3's own broader-scope-vs-close decision (explicitly deferred to operator/triage by its own last note), polylogue-2qx's generalization beyond the Claude Code mandate slice, polylogue-t46/t46.8/rxdo.3 (surface consolidation + telemetry contracts for full AC1).\n- Blocked on live archive + operator authorization (do NOT attempt to fabricate a substitute): polylogue-z9gh.7's AC2 (the actual 2026-07-15 incident replay against the promoted archive, reconstructing wf_54d4fb2e-841/coordinator cf0c6474...), polylogue-4s3c (live-scale steady-state envelope measurement). These are the two genuine, named, unavoidable operator-gated blockers on z9gh's own closure.\n\nThis sandbox worktree (/realm/project/polylogue/.claude/worktrees/agent-a9e569f4d3aae271c) has no promoted live archive and no operator present to authorize a live-scale run, so AC5/AC6's remaining scope could not be attempted here regardless of effort -- correctly deferring rather than fabricating evidence. No code changes made this session; this was a pure audit. z9gh remains OPEN.\n\n2026-07-27/28 convergence session — cumulative status update (honest, not a closure claim):\n\nThis P0 epic's dependency tree is a genuine multi-month architectural\nprogram (60+ nested beads under 1vpm -\u003e h6r -\u003e ovme -\u003e kwsb -\u003e 20d -\u003e 1xc\n-\u003e b5l -\u003e 60i5 -\u003e cuxz -\u003e rxdo -\u003e 4p1 -\u003e ... chains), spanning: work-evidence\ngraph modeling, actor/execution-context identity, MCP tool consolidation,\nfree-threading deploy, durable-tier migration framework, query/projection\nalgebra unification, and a terminal black-box continuity-replay gate\n(z9gh.7). It is not honestly closable to \"pristine, zero debt\" in a single\nsession, and forcing a closure claim here would misrepresent the state.\n\nWhat DID land this session (real PRs, real tests, self-reviewed since\nCodeRabbit was rate-limited most of the session, all merged to master):\n- #3351 h6r: real ActorRef/ExecutionContextRef identity derivation (was\n planned but unbuilt) -- direct progress on the 1vpm.6.1 -\u003e h6r chain.\n- #3360 ih67: TitleSource provenance threaded end-to-end (was computed but\n silently dropped on read surfaces).\n- #3361 20d.13: SSE events now carry real per-session identity (was\n session_id=None aggregate events) + removed 3 genuinely-dead event topics.\n- #3356 a7uk: reconcile-work-effects/materialize-incident-evidence CLI\n commands now route through AppEnv.repository (surface/storage boundary\n cleanup for the work-evidence graph surfaces).\n- #3357 lpen / #3358 de2a: fixed a live daemon bug where sibling\n full-revision-raw retirement quarantined an unrelated in-flight raw\n (blast-radius bug), plus a live-observed O(N)-per-append cost that caused\n 860s/9297s writer-gate holds.\n- #3362 da1: provider format-drift sentinel (classifies per-record schema\n drift during live ingest, surfaces risky/benign rates on daemon health +\n CLI status) -- new automatic-detection capability the archive-integrity\n side of this mandate needs.\n- #3363 lbgc: fixed a same-process SQLite zombie-connection lock race in\n seeded-archive test fixtures.\n- #3364 t0ta: split ChatGPT export detection into document/fragment tiers\n (tightened format-drift-risk detection without breaking legitimate\n per-line JSONL sniffing).\n- Re-confirmed polylogue-ihc8's fix (PR #3326, merged in a prior session)\n is real and correct via fresh read-only reproduction against the live\n archive; filed polylogue-dmvo for a genuine N:1 fan-out edge case the\n original fix doesn't yet resolve.\n- Found and surfaced a significant blocking infrastructure issue: GitHub\n Actions has been repo-wide disabled since 2026-07-13 due to a billing\n lock (CodeQL/Release/nix CI dark 14+ days) -- not a code bug, needs\n operator action at github.com/settings/billing (polylogue-n2f4 updated\n with this finding).\n\nWhat remains genuinely open and why, so nothing here is \"unaccounted for\":\n- z9gh.7 (the terminal black-box continuity-replay gate this epic exists to\n satisfy) depends on the FULL 1vpm work-evidence graph + t46.8 MCP\n consolidation + 4p1 query/projection algebra + hs3y agent-material\n acquisition -- each a multi-PR program in its own right, several still at\n \u003c50% implementation.\n- polylogue-de2a (writer-lock starvation under backlog): partial mitigation\n shipped (skip_already_applied fast path, FTS merge interval halved), but\n the real fix (writer-lock fairness/priority, or bounded per-append writer\n holds, or adaptive merge budget) is unstarted design work, correctly not\n attempted casually.\n- polylogue-2a6d (Borg backup coverage gap): narrowed from \"zero coverage\"\n to \"the direct filesystem-snapshot path is affected; the\n polylogue-sqlite-backup.service content-level path already works and is\n Borg-covered\" -- confirmed via a fresh manual backup+integrity-check this\n session.\n- Several dozen P1 items in the tree (kwsb.2 MutationTransaction, t46.8.2\n MCP verb migration, ovme ArchiveLocation, 20d.17 status snapshots, 4p1\n query algebra, rxdo.2 evidence privacy classification, etc.) remain\n genuinely large, multi-week design efforts -- checked this session,\n correctly not force-attempted as quick slices (avoids the\n #3354/#3359-style duplicate-work pattern already seen twice this session\n when two independent agents fixed the same bug in parallel).\n\nNo bead in this tree was closed dishonestly to satisfy an external\ncompletion pressure. This note exists so the epic's actual state is fully\ntraceable rather than silently stale.\n\nAC7 BOUNDED 2026-07-28. It previously read 'Every residual symptom is mapped to one class mechanism or justified as a separate contract before this epic closes' -- an unbounded universal quantifier with no enumeration procedure, which makes the bead unclosable by construction. A cold agent reading it correctly concludes the epic cannot be finished and reports the whole P0 as long-horizon work. Intent is preserved (no symptom silently dropped); the change is that the symptom set must be enumerated from a stated snapshot so the clause terminates.\n\nSame class of problem, for whoever audits the rest: AC3 depends on polylogue-1vpm and AC6 on polylogue-20d, both multi-node epics. Closing z9gh literally requires closing those. Either that transitive dependency is intended -- in which case z9gh is an index over programs, not an executable gate -- or the ACs should cite the specific slices (1vpm.6, the 20d latency envelope) rather than the parent epics.\n\nSNAPSHOT REFRESH 2026-07-28: the description cites '18,428 sessions and 4.85 million blocks'. Live now: 18,871 sessions, 5,042,564 blocks, 1,870,733 tool_use / 1,844,545 tool_result blocks, 9,179 session_links. The incident-era figures remain correct as the incident record; use these for any current-scale claim.","status":"open","priority":0,"issue_type":"epic","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T22:42:56Z","created_by":"Sinity","updated_at":"2026-07-28T20:04:11Z","metadata":{"frontier_program":"active"},"labels":["area:mandate","area:mcp","area:perf","area:query","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-1vpm","type":"related","created_at":"2026-07-15T00:55:32Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-1xc","type":"related","created_at":"2026-07-15T00:46:50Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-20d","type":"related","created_at":"2026-07-15T00:46:47Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-2qx","type":"related","created_at":"2026-07-15T00:55:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-rsad","type":"related","created_at":"2026-07-15T00:46:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t46","type":"related","created_at":"2026-07-15T00:46:54Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t46.3","type":"related","created_at":"2026-07-15T00:46:40Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t46.8","type":"related","created_at":"2026-07-15T00:46:43Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-z9gh","depends_on_id":"polylogue-t8t","type":"related","created_at":"2026-07-15T00:46:37Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-layg","title":"Fix excision bypass via second write chokepoint (blocker, held off #2875)","description":"Adversarial review of PR #2875 (polylogue-27m, local excision + secret detection) found the excision non-resurrection guarantee is bypassable through a second, real production write chokepoint: write_source_raw_session checks is_blob_hash_excised before insert, but a sibling write path does not (see polylogue/storage/sqlite/archive_tiers/source_write.py around line 434). This is a BLOCKER — PR #2875 was deliberately NOT merged pending this fix. Also flagged: resolve_session_excision_target/apply_session_excision only resolve rows keyed directly to session_id, missing related rows; docs/plans/security-privacy-coverage.yaml marks captured_content_secret_detection implemented:true and removes it from coverage_gaps but the scanner's actual coverage may not support that claim (reviewer flagged as major, verify before keeping the claim).","acceptance_criteria":"The second write chokepoint also checks is_blob_hash_excised before insert (or a shared helper enforces this at a single chokepoint both paths use). Session excision resolves related rows, not just session_id-keyed rows. The security-privacy-coverage.yaml claim is verified true or reverted to its prior severity. Regression test proves excised content cannot resurface via the previously-bypassable path. Then PR #2875 (or its successor) merges.","status":"closed","priority":0,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-14T08:21:28Z","created_by":"Sinity","updated_at":"2026-07-14T23:05:02Z","closed_at":"2026-07-14T23:05:02Z","close_reason":"Satisfied on master by PR #2875 (c2fd1e902): the second raw write path enforces excision, related rows are covered, the public claim was corrected, and the bypass regression landed.","labels":["area:security","area:storage","horizon:frontier"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-7ufv","title":"Copy reused index clones across archive subvolumes","description":"The retry prepare correctly reused a completed v36 index generation, but reuse_index_clone used os.replace directly from the archive generation to the staging receipt directory. Those paths are on separate subvolumes and fail with EXDEV before receipt creation.","design":"Use reflink_clone into a temporary file in destination.parent, then rename locally to destination and fsync. Preserve the source generation until the local publish succeeds; remove the original staged clone only when it is safe and not an archive generation. Add an EXDEV regression.","acceptance_criteria":"Reusing a v36 index clone across distinct parents succeeds when direct cross-parent os.replace raises EXDEV; destination is correct and no temporary file remains. Focused test and devtools verify --quick pass.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-13T21:14:03Z","created_by":"Sinity","updated_at":"2026-07-13T22:48:15Z","started_at":"2026-07-13T21:14:12Z","closed_at":"2026-07-13T22:48:15Z","close_reason":"PR #2868 merged (fix(storage): copy reused index clones locally). Live v36 cutover activated successfully using the fixed reuse_index_clone path: reflink into a temp file in destination.parent, then local rename+fsync, avoiding the EXDEV cross-subvolume os.replace. Verified via the successful v36-retry2 activation (source=9,user=8,index=36,embeddings=2,ops=1).","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-rze2","title":"Finalize fast-forward receipt after durable WAL cleanup","description":"The v35→v36 activation promoted all tiers but then rejected normal source/user WAL sidecars during final evidence collection outside its rollback transaction. This left the archive promoted with a receipt still marked prepared. The actuator must finalize/checkpoint durable files before immutable evidence and retain rollback semantics for every post-promotion exception.","design":"Keep final evidence collection inside the activation try/except. Explicitly finalize/checkpoint source and user after migrations, then collect versions through immutable evidence. Any failure before the activated receipt is written must restore every promoted tier and durable snapshot and write a rolled_back receipt. Add a regression that simulates durable sidecars after a successful migration and proves either activated receipt or full rollback.","acceptance_criteria":"Focused regression reproduces the post-migration durable-sidecar state and passes. Successful activation records status activated with source=9,user=8,index=36,embeddings=2,ops=1 and no ambiguous sidecars. A forced final-evidence failure restores v35/v1 files and writes rolled_back. Run focused tests plus devtools verify --quick.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-13T20:32:20Z","created_by":"Sinity","updated_at":"2026-07-13T22:48:16Z","started_at":"2026-07-13T20:32:32Z","closed_at":"2026-07-13T22:48:16Z","close_reason":"PR #2867 merged (fix(storage): finalize fast-forward durable WALs). Live v36 cutover activated successfully: final evidence collection now runs inside the activation try/except with source/user WAL finalization before immutable evidence reads. Verified via successful v36-retry2 activation (status=activated, no rollback, versions match target).","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -29,7 +29,7 @@ {"_type":"issue","id":"polylogue-cnaj","title":"Bound active JSONL append ingestion memory and catch-up overlap","description":"Live v35 incident on 2026-07-13: an actively appended 46 MB Codex JSONL was selected by periodic catch-up every ~16 seconds. Each append reported 0.1–0.3 MB read but held daemon writer 37–38 seconds and temporarily grew anonymous RSS from ~0.4 GiB to ~4.2 GiB; cgroup memory reached the 8 GiB high threshold (6,655 high events), 22 GB reads and 3.2 GB writes in 8.5 minutes. Daemon was intentionally stopped before OOM. This blocks safe unattended backfill/daemon operation.","design":"Build a reproducible harness from the observed active-append shape, then locate retained full-session/materialization state and overlapping periodic scheduling. Preserve correctness for append frontier, source/index atomicity, quiet deferral and crash recovery. The fix must bound live working set and prevent redundant catch-up while a prior pass is active; do not solve this by permanently disabling watching, broadening loss windows, or weakening authority proofs. Prove exact recovery/cursor behavior after daemon restart.","acceptance_criteria":"1. Reproduction measures memory high-water and bounded input work for a large, actively appended Codex JSONL. 2. One active file cannot schedule overlapping/redundant catch-up while its prior append pass is running. 3. Append ingestion retains no full historical payload/model beyond its operation boundary; RSS is bounded materially below service MemoryHigh on the reproduction. 4. Cursor/frontier/source/index correctness, restart recovery and failure rollback remain proven. 5. Focused tests and quick verification pass; live restart postflight does not reintroduce the hot loop.","notes":"Scoped 2026-07-13: reproduce and fix the active Codex JSONL append memory/catch-up incident in polylogue/sources/live plus focused tests only. I will use the existing #2841 cohort-memory harness, preserve cursor/frontier and rollback semantics, and avoid live archive or daemon mutation.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-13T16:19:03Z","created_by":"Sinity","updated_at":"2026-07-13T16:39:42Z","started_at":"2026-07-13T16:19:59Z","closed_at":"2026-07-13T16:39:42Z","close_reason":"PR #2849 merged as 2b0221a98. Established byte-proven append cohorts now use durable replay metadata without historical full reads; incomplete/omitted-current chains classify then defer without cursor advance. Focused harness: 4 passed; devtools verify --quick: 15 checks passed. Live daemon remains stopped for operator postflight.","labels":["area:daemon","area:ingest","area:storage","delivery:G-live-performance","horizon:frontier"],"comments":[{"id":"019f6bee-2d1d-7b01-a481-a4089b02445e","issue_id":"polylogue-cnaj","author":"Sinity","text":"2026-07-16 closure-audit correction: keep closed. The packaged daemon is active (started 18:20:21 CEST) and the original append-reread hot loop did not recur in the observed live restart. The startup scan processed a 2.54 GB backlog; the relevant append chunk completed with append_files=2 and read_amp=0.614, and a later changed-file append completed with read_amp=0.0046. Current cgroup state at audit: memory.current about 1.53 GB, peak about 2.149 GB under a 2 GiB cap, zero oom/oom_kill, and zero current PSI. The backlog did hit the memory cap and current raw-frontier/CAS retries remain noisy, but those are separately owned by lkrc/yla8 and are not evidence that the cnaj historical-reread mechanism remains live. The earlier audit incorrectly treated the stale close-time sentence that the daemon remained stopped as current state.","created_at":"2026-07-16T17:16:39Z"}],"dependency_count":0,"dependent_count":0,"comment_count":1} {"_type":"issue","id":"polylogue-lkrc.2","title":"Repair remaining current unknown-origin ChatGPT heads","description":"Live postflight after the final legacy NULL-native-id copy-forward found nine current chatgpt-export sessions whose sessions.raw_id still points at a durable source.raw_sessions row typed origin=unknown-export with logical_source_key=unknown:\u003cnative-id\u003e. They are distinct from the three original lkrc raws: two siblings now point to canonical byte-proven copies and the legacy target points to 402915...; this residual cohort is a separate current-authority problem.","design":"Start from a fresh stopped-daemon census that joins current sessions to source.raw_sessions and production-normalizes each retained blob. Partition rows by existing revision/head/application/membership evidence; reuse an already-proven copy-forward route only when every source/index witness exactly matches its contract. Preserve original raw/blob/membership/head/application evidence, create a canonical replacement rather than relabelling historical raws, require a proof digest plus planned/applied receipt, and keep source-v7/v35 active-index compatibility. Do not treat retained non-current historical unknown heads as current mismatches.","acceptance_criteria":"1. Exact before census names every current session backed by unknown-export raw and distinguishes non-current retained history. 2. Every eligible row is repaired through a receipted, proof-bound, idempotent evidence-preserving path; ineligible shapes remain fail-closed with a durable reason. 3. Exact after census is zero current sessions whose raw origin/logical key disagrees with the production-normalized ChatGPT identity. 4. Focused real-route tests cover the observed evidence shapes, drift/rollback, generated active-index routing, and source-v7 compatibility; quick verification passes. 5. Live use follows a verified backup, stopped daemon, fresh dry proof, immutable receipt, and restart postflight.","notes":"Discovered 2026-07-13 after successful live legacy child repair receipt legacy-native-repair-20260713T160800Z.jsonl. Exact initial current cohort raw IDs: 3c144e4b6eccf6c65368488be8c952a510a50ed86deb9c93453b1a0dd08a55b2, 773bbbf1b92e763a0e85d1c798f127d94aa1e0f70b6e91978bcdd7cfbecc078d, 2af730ea7ca773cbb1983498d3103616e7309c41593cafa8e781a3eb151eca3b, bd47782eea0579a4bcba6d5b51670e4f71a80cf1473f38ee2536d07afb2ff1e0, 6567faf1da05d51ab8343fba6334602eef120f6b39ca1edb884a71edabe90d0d, 27527c1586e4e0105ec2a73c2206709af1ce74df0c0bb4dea24069f350644538, f43a203e159d29f403cca7123fb95c83ab3169f27978b7caa029c6496a0309e6, c10658915c27d74517c5d6f941247007564275d3d9360b2290683feb6593ee4b, 88aefc84afb181135c76a724b361ef21a9aa856f2a1ef117511e08fdceba2785.\nRead-only stopped-daemon census 2026-07-13: correct raw f43a203e159… to f43a203a359d29f403cca7123fb95c83ab3169f27978b7caa029c6496a0309e6. All nine old heads are unknown-export/native_id NULL/full+byte_proven/gen0 with one selected-baseline app and production parser identity match. Safe common rekey candidates: 773bbbf1…, bd47782e…, f43a203a… (no canonical head); 2af730ea…, 27527c15… (exact-equal semantic canonical witnesses). Fail-closed: 6567faf1… and c106589… semantic canonical hash conflicts (c106 diverges message 523); 3c144e… superseded_equivalent membership plus canonical hash conflict (diverges message 1333); 88aefc84… current reparse hash drift/incompatible canonical byte head. Existing actuators correctly reject all. Implement a new sibling byte-proven-browser-rekey actuator only for the five exact shapes; preserve all old/semantic evidence and record ineligible reasons for the four.\n2026-07-13: Claimed for isolated implementation of the sibling evidence-preserving byte-proven browser rekey actuator. Scope is exactly five proof-approved shapes; four observed conflict/drift shapes remain fail-closed. No live archive or daemon mutation is authorized by this implementation lane.\n2026-07-13: implementation merged in PR #2850 / master 64f4a00e8. The new repair_byte_proven_browser_capture_null_native_ids actuator is intentionally limited to the five proof-approved byte-proven NULL-native shapes. Verification: devtools verify --quick; focused byte-rekey matrix 10 passed. No live archive or daemon mutation occurred. Remaining scope is the parent-run stopped-daemon dry proof/apply/postflight, including durable reasons for the four ineligible rows.\n2026-07-14 status check (no live archive touched): re-verified the code portion of this bead is complete on current master (PR #2850 / 64f4a00e8, repair_byte_proven_browser_capture_null_native_ids). Confirmed via the existing 10-case focused byte-rekey matrix (test_byte_proven_browser_rekey_*) plus this session's own re-run: devtools test tests/unit/storage/test_browser_capture_origin_repair.py -k \"conflict or record_conflict\" -\u003e 11 passed. No further code change made or needed for this bead specifically in PR #2877 -- that PR's lkrc.3 work builds ON TOP of this bead's actuator (re-runs its exact eligibility proof) rather than modifying it. Remaining scope per this bead's own notes (\"parent-run stopped-daemon dry proof/apply/postflight, including durable reasons for the four ineligible rows\") is entirely live-execution, reserved for the operator; the \"durable reasons for the four ineligible rows\" portion is now directly actionable via record_browser_canonical_authority_conflict_blockers (PR #2877, polylogue-lkrc.3) once the operator runs it live.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-13T16:13:11Z","created_by":"Sinity","updated_at":"2026-07-14T23:12:16Z","started_at":"2026-07-13T16:31:27Z","closed_at":"2026-07-14T23:12:16Z","labels":["area:browser","area:sources","area:storage","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-lkrc.2","depends_on_id":"polylogue-lkrc","type":"supersedes","created_at":"2026-07-15T01:12:16Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-lkrc.1","title":"Copy forward legacy browser raw missing native identity","description":"After PR #2839 hardens browser-origin copy-forward proofs, the final live lkrc target 282983b4ec87c080fd60c31d9ebaa415a38f57c8f57bb22cdeda1b7906aca2c0 correctly refuses because its durable unknown-export raw has native_id=NULL, even though its retained browser-capture bytes parse to ChatGPT session 6a149c9e-2910-83eb-a93b-e6805f9f94f8. The row must not be relabelled or mutated in place.","design":"Add a separate, explicitly named evidence-preserving legacy-native-missing copy-forward route. It may accept native_id=NULL only as the exact legacy evidence shape, not as a general relaxation: prove raw origin=unknown-export, browser-capture provenance, native_id NULL, source/blob-ref path/hash/size agreement, complete singleton census, quarantined full envelope, production parse yields exactly one canonical ChatGPT session, canonical semantic authority and all applications/memberships/head witnesses match, and no competing old/canonical applications exist. Create a new canonical raw/application/receipt with parsed native identity; never update/delete the old raw/blob/head/application/membership. Planned/applied receipt records legacy-null witness and parser-derived native ID; locked reproof/CAS is all-or-nothing; reapply idempotent. Keep source-v7 compatibility.","acceptance_criteria":"1. Real-route fixture with legacy native_id NULL is ineligible to ordinary copy-forward but eligible only to the dedicated actuator after every listed witness is proven. 2. Any non-NULL wrong native, origin/path/blob/census/parser/session/head/application/timestamp/frontier/sibling drift fails before source write. 3. Apply makes a new correctly typed canonical raw and leaves all old evidence byte-for-byte unchanged; receipt proves the legacy-null witness and parsed identity. 4. Reapply is idempotent; planned/apply mismatch or post-proof failure rolls back. 5. Focused tests + quick pass; live use only after fresh full backup, stopped daemon, read-only dry run, exact receipt, apply, and postflight zero mismatched heads.","notes":"2026-07-13: Claimed after PR #2839 merged as db586289e. Ordinary actuator is deliberately fail-closed for native_id=NULL; this child owns the separate legacy-only copy-forward path. Implementation must preserve source-v7 compatibility and not mutate the old raw.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-13T12:54:50Z","created_by":"Sinity","updated_at":"2026-07-13T16:16:35Z","started_at":"2026-07-13T13:03:10Z","closed_at":"2026-07-13T16:16:35Z","close_reason":"Live repair applied with receipt legacy-native-repair-20260713T160800Z.jsonl; source-v7-compatible v35 artifact verified; rerun reports already_repaired.","labels":["area:browser","area:sources","area:storage","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-lkrc.1","depends_on_id":"polylogue-lkrc","type":"parent-child","created_at":"2026-07-13T14:54:49Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f5bb0-a6fb-7469-a902-893404f4e28f","issue_id":"polylogue-lkrc.1","author":"Sinity","text":"2026-07-13 implementation update: legacy-only NULL-native route now refuses a pre-existing canonical head, requires exactly one old raw membership key and payload blob reference, and stages source copy-forward plus index authority transition in one attached-source transaction. A regression injects a failure after source staging and proves old source/index rows remain unchanged with a planned-only receipt. Verification: devtools test tests/unit/storage/test_browser_capture_origin_repair.py -k legacy_browser_native_id (13 passed); devtools test tests/unit/storage/test_browser_capture_origin_repair.py tests/unit/cli/test_archive_maintenance_cli.py -k 'legacy_browser_native_id or rejects_legacy_raw_without_native_id' (15 passed); devtools verify --quick (passed). Pending independent re-audit; no live archive actuator has been run.","created_at":"2026-07-13T13:35:32Z"}],"dependency_count":0,"dependent_count":0,"comment_count":1} -{"_type":"issue","id":"polylogue-lkrc","title":"Converge raw evidence authority through one proof-driven reconciler","description":"Polylogue has accumulated separate repair actuators and incident Beads for origin-mismatched browser raws, competing canonical heads, duplicate raw identities, replaced snapshots requiring reacquisition, quarantined accepted raws, and superseded revisions. These are not independent product capabilities. They are states of one raw-evidence authority lifecycle whose invariant is that every accepted materialized head is backed by a typed, byte-identified, provenance-authorized raw revision—or is held in an explicit unresolved/conflict/reacquisition state.","design":"Create one RawAuthorityReconciler over the existing raw frontier projection, raw revision authority types, OriginSpec evidence, and repair proof/receipt machinery. It enumerates every accepted/materialized head and classifies it into proven-current, safely rekeyable/equivalent, duplicate-alias, superseded, missing-bytes/reacquire, conflicting-authority/needs-judgment, unresolved-provenance, or corrupt. A canonical plan schema carries witnesses, source/head hashes, expected identities, authority, intended actuator, and preconditions. Apply uses one plan-authorize-apply-receipt-postflight protocol with locked atomic receipts and compare-and-swap revalidation; existing browser-origin, duplicate-identity, quarantined-head, and superseded-snapshot functions become actuator strategies behind it or are deleted. Safe deterministic repairs may converge automatically through the daemon after quiet/proof gates; conflicting content never auto-wins and instead emits a durable judgment request/blocker. Reacquisition is a durable obligation linked to the retained receiver/source artifact and must prove byte identity before promotion. The reconciler reports complete counts and stable refs across all states and is idempotent/restartable. yla8 remains the distinct prevention invariant for replay ordering; this bead repairs and continuously audits the frontier rather than duplicating that write-path rule.","acceptance_criteria":"1. One census/plan covers origin mismatch, duplicate identity, quarantined accepted raw, superseded snapshot, missing/replaced bytes, and competing canonical authority with mutually exclusive typed states and stable evidence refs. 2. One plan-authorize-apply-receipt-postflight contract drives every actuator; grep finds no independent proof-digest/receipt lifecycle for browser-origin versus duplicate-identity repairs. 3. Deterministically equivalent/rekeyable/duplicate cases converge idempotently and restartably; compare-and-swap revalidation prevents stale-plan writes. 4. Conflicting byte/content authority cannot auto-select a winner and produces a durable queryable judgment blocker; an operator assertion can resume the same plan. 5. Missing bytes create a durable reacquisition obligation and promote only after origin/identity/hash proof; replaced receiver artifacts are not silently lost. 6. The known lkrc/lkrc.3, 57rp, t0dy, and quarantined/superseded fixtures all pass through the single reconciler, and a stopped-daemon live postflight leaves zero unreported frontier gaps. 7. Readiness/status expose state counts and remediation refs; known-sidecar or accepted-index status alone cannot report healthy. 8. OriginSpec supplies authority rules and yla8 replay-order protections remain intact; mutation tests fail if either is bypassed.","notes":"2026-07-13 live v35 postflight: verified full_evidence backup receipt at /realm/staging/polylogue-sqlite/recovery/lkrc-v35-20260713T042736Z/polylogue-archive-20260713T042738Z/verification-receipt.json (all five SQLite tiers, 26,600 blobs). Exact v7/index-v35/user-v6 artifact completed all watcher catch-up chunks with no recurrence of membership replay cannot retire an unrelated accepted head. Stopped-daemon census found 11 unknown-export-\u003eChatGPT session/raw mismatches. The three lkrc raws are quarantined full singleton censuses with canonical membership decision NULL and exact old unknown-key selected-baseline receipts; actuator now requires that narrow dual witness. The other 8 are excluded: 7 byte_proven unknown raws without membership/census, 1 byte_proven superseded-equivalent membership; separate follow-up required.\n2026-07-13 adversarial loop iteration 5 reached its cap with unresolved P0 proof gaps; do not merge/apply #2839 head 3b0ca3f08. Real residuals: (1) semantic canonical and historical sibling source envelopes omit capture_mode; require canonical provider when schema has field, with v7 fallback. (2) original unknown raw blob_ref.source_path is not bound to raw source_path in preflight/locked reproof. (3) original unknown raw native_id is not bound to reparsed provider session id preflight/locked reproof. (4) restore_canonical_head exact-byte route omits native_id, source_index, capture_mode, predecessor/append envelope fields; normalize conditional full-envelope proof for exact/semantic/sibling paths. Lower severity: historical supersession decided_at_ms accepts negative values. Iteration-5 reviewer found these against the current 31-test terminal closure; no live mutation after findings. Further implementation plus an operator-authorized review cycle is required before merge/apply.\n2026-07-14 code-verification pass (no live archive touched): re-checked the \"adversarial loop iteration 5\" proof gaps recorded in this bead's prior note against current master (031d8d183) source. All 3 named residual gaps -- (1) capture_mode binding, (2) blob_ref.source_path binding, (3) native_id binding into the preflight/locked reproof witness -- are already present in _browser_origin_source_envelope_is_exact (polylogue/storage/repair.py), which every browser-origin repair path (exact-canonical, semantic, and the restore_canonical_head route) now shares. Confirmed these landed via PRs #2843/#2847/#2848/#2850 (all merged after the iteration-5 note was written) by git log/git show on the relevant commits. AC1 (new browser captures acquire chatgpt-export origin, not unknown-export) is already covered by test_streaming_sized_browser_capture_json_uses_native_payload_detection in tests/unit/sources/test_live_batch_support.py, which asserts `SELECT origin FROM raw_sessions` == chatgpt-export for a fresh ingest.\nPR #2877 (branch feature/fix/raw-identity-repair-cluster) adds the evidence-packet + durable-blocker capability for this bead's dependent polylogue-lkrc.3 (the 4 sessions the exact-byte rekey actuator correctly refuses) -- see that bead's notes. AC4 (dynamic live census reports zero mismatches, or every unresolved conflict is an explicit durable blocking state) remains open pending a live-archive run of record_browser_canonical_authority_conflict_blockers, which this session does not perform (live-execution reserved for the operator). No code gap was identified beyond what #2877 adds; this bead's remaining scope is live-execution, not implementation.\n[2026-07-15 invariant-collapse pass] Expanded from the browser-origin incident into the shared raw-authority state machine evidenced by multiple separate repair classes in storage/repair.py. Supersedes lkrc.3, 57rp, and t0dy; their named live cases are regression/postflight inputs, not separate scheduled projects. Does not absorb yla8 because preventing stale replay is a different write-path invariant.\nLive evidence 2026-07-15 from MCP readiness_check: raw_frontier_integrity reported 1,890 broken active heads among 18,347 checked, 40 ingest cursors committed past accepted raw material, and 34 cursor/head authority rows not comparable. This is current measured debt, not a repair instruction; preserve the snapshot/frame and classify through the proof-driven reconciler before any cursor reset or replay mutation.\n2026-07-15 yla8 read-only preflight sharpened the live failure: packaged build 20d703e (source11/index36/user8) reports 1,890 broken active seeds, 40 cursor-ahead rows, 34 incomparable authority rows, and 15,264 direct / 21,398 expanded replay candidates. Journal shows ordinary convergence replaying exactly 2 logical sources per pass while the candidate count rose from 11,717 to 15,264 over four hours. Treat current rows as an immutable-frame census for RawAuthorityReconciler classification; do not reset cursors or run broad replay. hjpx owns the accepted-plan-to-fixed-point execution defect and is now P0 discovered from the failed yla8 gate.\n2026-07-16 implementation pass: owning the coherent lkrc/hjpx.1/lkrc.4 raw-authority cluster from fresh origin/master. Scope is the single reconciler/immutable-plan conservation and the production multi-session divergence regression now observed in packaged ordinary catch-up. Preserve yla8 fail-closed replay protections; no live cursor reset, force replay, evidence deletion, manual SQL repair, or live apply before reviewed code, verified backup, quiescent census, and explicit authorization. First deliverable is a production-route failing fixture and read-only live evidence.\n2026-07-17 PR #2962 closure implementation at edd68d240: the shared frontier now owns typed conflict disposition end to end. A conflicting browser head remains non-executable until its exact candidate judgment assertion is accepted and the blocker is resolved with disposition=retain_canonical_authority; the resulting immutable successor plan CAS-revalidates the complete competing-head witness, retains canonical authority, records supersession, retires the obsolete unknown-key head, and proves a terminal postcondition. Browser copy-forward/restore now also remove the obsolete head instead of leaving a corrupt residual frontier. Old incident receipt/mutator/CLI lifecycles were removed. Focused production-route verification: 183 passed across raw ledger, browser, quarantine, duplicate identity, daemon CLI, and maintenance CLI. Quick gate 20260717T000755Z-quick-1199839-9d926a5d: 16/16 green. Remaining closure boundary is the separately authorized stopped-daemon live gate in yla8; no live archive mutation was performed by this PR.\n2026-07-17 AC7 status-truth closure: PR #2965 (0dc5773a9) now persists complete frontier_state_counts alongside residual state_counts. Readiness exposes the complete inventory (including proven_current) while deriving blocking exclusively from postflight residual state. Dry-run and applied-postflight regressions passed (201 focused raw-authority/daemon/CLI tests; quick gate green). Remaining lkrc boundary is still hjpx fixed-point execution plus the separately authorized yla8 stopped-daemon live gate; no live archive mutation was performed.\n2026-07-18 inbox re-discovery check: /realm/inbox/download/PATCH(1) (2).diff (3627 lines; touches docs/cost-model.md, polylogue/archive/query/source_freshness.py, source_freshness_surfaces.py, cli/commands/diagnostics.py, core/evidence_value.py, core/temporal.py, daemon/status_snapshot.py, daemon/web_shell.py, insights/temporal_source.py, storage/usage.py). Patch base blob for docs/cost-model.md resolves via git cat-file to commit efadb404e (#3033, testdiet-06 admission, 2026-07-17) -- this predates 45+ subsequent master commits. git apply --check fails on 5 files including source_freshness.py itself (not just generated docs), confirming real drift, not just cosmetic. Its scope (cost-model.md, dual cost accounting, source freshness) overlaps two ALREADY-CLOSED beads: polylogue-5hf (provider token accounting) and polylogue-f2qv.3 (dual cost view) -- both closed before this patch's own base commit. named_source_freshness/NamedSourceFreshness already exist independently on master via PR #2924 (2026-07-16), predating this patch too. Verdict: superseded by already-shipped, already-closed work; not reconciled by hand given the drift depth and lack of any open bead this patch would newly satisfy. No action taken.\n2026-07-19 AC-closure audit (Sonnet audit lane, read-only, .agent/scratch/trust-floor-audit-2026-07-19.md has full detail): VERDICT = NARROWABLE (code-layer essentially complete; bead as a whole not closable because AC6's live postflight is explicitly not yet operator-authorized). Re-verified rather than trusted the prior notes.\n\nCode/tests (AC1,2,3,5,7,8): polylogue/storage/repair.py carries record_browser_canonical_authority_conflict_blockers, the frontier/residual state_counts pair (~line 4532), and the census/postflight machinery cited in the 2026-07-17 notes (#2962, #2965). Focused suite: devtools test tests/unit/storage/test_raw_authority_ledger.py tests/unit/storage/test_duplicate_raw_identity_repair.py tests/unit/storage/test_quarantined_accepted_raw_repair.py tests/unit/storage/test_comparative_judgment_assertions.py -\u003e 44 passed. Broader devtools test -k \"raw_materialization or raw_authority\" -\u003e 166 passed, 1 failed.\n\nThat 1 failure (tests/unit/sources/test_live_batch_support.py::test_live_multi_session_divergence_reopens_raw_authority) was investigated to ground truth rather than assumed pre-existing: it is test-currency drift from PR #3129 (de0b2df7a, landed 2026-07-18, intentionally redefines watcher-layer succeeded/failed semantics so an ambiguous/deferred membership decision folds into succeeded, not failed), NOT a regression against this bead's own AC1/AC4 invariants. Verified via a bypass probe (direct call into LiveBatchProcessor outside the stale assertion) that the deeper judgment/quarantine state this bead actually owns is fully intact: raw_session_memberships still records exactly 2 ambiguous/quarantined rows for the two divergent source paths, raw_sessions.parsed_at_ms is NULL for both with zero parse_error, and the index still resolves only the first-accepted head for chatgpt:shared with the original accepted_raw_id -- i.e. conflicting authority still cannot auto-select a winner, matching AC4. Filed polylogue-5202 (P2 bug) to fix the stale assertion; do not treat it as reopening lkrc's own scope.\n\nDependency hjpx: hjpx.1 (P0 correctness kernel: parser census before planning, immutable plan/outcome/postflight conservation, fair rotation, two-quiescent-census fixed point) is CLOSED via PR #2961/593ef3c62 with 5 independent adversarial passes and 19+82 focused tests green at that time -- re-confirmed present in current repair.py. hjpx.2 (P1 scale-proof at the July-15 archive cardinality, a live-archive resource-envelope proof) remains in_progress: its own notes record 4 consecutive honest self-aborts of the scale-proof generation phase under sustained host I/O pressure across roughly 140 minutes over two sessions (2026-07-18), explicitly \"not closable\" this session. That is a live-scale execution problem, not a code-correctness gap in the reconciler, and is out of this audit lane's authority to resolve (no live archive access).\n\nAC6 (stopped-daemon live postflight against the real archive, zero unreported frontier gaps) and the yla8 live gate it depends on are explicitly NOT authorized: yla8's own 2026-07-18 read-only preflight packet recommends \"DO NOT authorize the live gate yet,\" citing (1) no current verified full_evidence backup (most recent is 6+ days stale, predates the whole 07-15 authority program and 07-18 incident/restore), (2) the archive is mid-restore with only 170/79,571 raw artifacts materialized, making any current frontier-integrity reading a 0.2% unrepresentative sample rather than a population verdict, (3) measured watcher catch-up throughput (~0.185 files/sec, 1 worker) implies ~5 days just to drain the current gap without polylogue-5jak landing first, and (4) hjpx.2's scale proof above is itself incomplete. This is squarely an operator-gated live action, outside any coding agent's authority and outside this audit lane's mission constraints (no live archive access).\n\nNet: lkrc's single-reconciler architecture, typed conflict/judgment states, CAS revalidation, and status-truth surfaces are code- and unit-test-complete. The bead cannot be marked CLOSABLE as a whole because AC6 requires a live artifact this session correctly declines to produce; recommend keeping lkrc open with AC1/2/3/5/7/8 marked code-satisfied, AC4 code-satisfied via #2962, and AC6 explicitly blocked on yla8 operator authorization (not a further coding task for this bead).\n\nCommands run: devtools test tests/unit/storage/test_raw_authority_ledger.py tests/unit/storage/test_duplicate_raw_identity_repair.py tests/unit/storage/test_quarantined_accepted_raw_repair.py tests/unit/storage/test_comparative_judgment_assertions.py -\u003e 44 passed in 9.46s; devtools test -k \"raw_materialization or raw_authority\" -\u003e 166 passed, 1 failed in 46.06s (isolated re-run confirms deterministic, not xdist flake).\n\n2026-07-27: resolved 12 previously-stuck frontier_judgment blockers across 6 browser-rekey conversations (chatgpt:6a4629b3-8510-83eb-9180-b94a537abf7a and 5 siblings) via manual byte-level verification against the blob store - all confirmed safe retain_canonical_authority (10 straightforward content-hash matches, 2 where the automated detector's message-level diff hit FileNotFoundError and correctly deferred to manual judgment; direct diff confirmed identical message content in both, only incidental capture metadata differed). Also discovered and filed polylogue-rjtv: census regenerates duplicate judgment requests across cycles for the same underlying conflict instead of deduping against a still-pending one (roughly quadrupled this session's manual verification burden: 24 candidates reviewed for what was actually 6 real conflicts).","status":"in_progress","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:50:53Z","created_by":"Sinity","updated_at":"2026-07-27T01:01:39Z","started_at":"2026-07-16T19:20:39Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:browser","area:sources","area:storage","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-1xc","type":"parent-child","created_at":"2026-07-15T01:15:39Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-1xc.13","type":"relates-to","created_at":"2026-07-15T06:25:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-2qx","type":"related","created_at":"2026-07-15T01:09:46Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-b5l.1","type":"relates-to","created_at":"2026-07-15T20:42:23Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-yla8","type":"blocks","created_at":"2026-07-15T01:09:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-yla8.10","type":"discovered-from","created_at":"2026-07-13T01:50:54Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6407-6e37-7464-b88f-e043f6e0c88b","issue_id":"polylogue-lkrc","author":"Sinity","text":"[Dogfood 2026-07-15 / F-004] A named growing Codex source had an excluded cursor after five failures, later acquired raws unparsed, and a stale indexed session. Archive census showed 3,821 excluded cursors, 1,890 broken heads, 41 cursor-ahead rows, and 34 authority gaps. polylogue-1xc.13 owns the named-source acquisition-to-searchable projection and excluded-not-idle semantics. This reconciler remains the owner of underlying authority classification and repair population, so the beads are related rather than duplicating actuators.","created_at":"2026-07-15T04:27:16Z"}],"dependency_count":1,"dependent_count":0,"comment_count":1} +{"_type":"issue","id":"polylogue-lkrc","title":"Converge raw evidence authority through one proof-driven reconciler","description":"Polylogue has accumulated separate repair actuators and incident Beads for origin-mismatched browser raws, competing canonical heads, duplicate raw identities, replaced snapshots requiring reacquisition, quarantined accepted raws, and superseded revisions. These are not independent product capabilities. They are states of one raw-evidence authority lifecycle whose invariant is that every accepted materialized head is backed by a typed, byte-identified, provenance-authorized raw revision—or is held in an explicit unresolved/conflict/reacquisition state.","design":"Create one RawAuthorityReconciler over the existing raw frontier projection, raw revision authority types, OriginSpec evidence, and repair proof/receipt machinery. It enumerates every accepted/materialized head and classifies it into proven-current, safely rekeyable/equivalent, duplicate-alias, superseded, missing-bytes/reacquire, conflicting-authority/needs-judgment, unresolved-provenance, or corrupt. A canonical plan schema carries witnesses, source/head hashes, expected identities, authority, intended actuator, and preconditions. Apply uses one plan-authorize-apply-receipt-postflight protocol with locked atomic receipts and compare-and-swap revalidation; existing browser-origin, duplicate-identity, quarantined-head, and superseded-snapshot functions become actuator strategies behind it or are deleted. Safe deterministic repairs may converge automatically through the daemon after quiet/proof gates; conflicting content never auto-wins and instead emits a durable judgment request/blocker. Reacquisition is a durable obligation linked to the retained receiver/source artifact and must prove byte identity before promotion. The reconciler reports complete counts and stable refs across all states and is idempotent/restartable. yla8 remains the distinct prevention invariant for replay ordering; this bead repairs and continuously audits the frontier rather than duplicating that write-path rule.","acceptance_criteria":"1. One census/plan covers origin mismatch, duplicate identity, quarantined accepted raw, superseded snapshot, missing/replaced bytes, and competing canonical authority with mutually exclusive typed states and stable evidence refs. 2. One plan-authorize-apply-receipt-postflight contract drives every actuator; grep finds no independent proof-digest/receipt lifecycle for browser-origin versus duplicate-identity repairs. 3. Deterministically equivalent/rekeyable/duplicate cases converge idempotently and restartably; compare-and-swap revalidation prevents stale-plan writes. 4. Conflicting byte/content authority cannot auto-select a winner and produces a durable queryable judgment blocker; an operator assertion can resume the same plan. 5. Missing bytes create a durable reacquisition obligation and promote only after origin/identity/hash proof; replaced receiver artifacts are not silently lost. 6. The known lkrc/lkrc.3, 57rp, t0dy, and quarantined/superseded fixtures all pass through the single reconciler, and a stopped-daemon live postflight leaves zero unreported frontier gaps. 7. Readiness/status expose state counts and remediation refs; known-sidecar or accepted-index status alone cannot report healthy. 8. OriginSpec supplies authority rules and yla8 replay-order protections remain intact; mutation tests fail if either is bypassed.","notes":"2026-07-13 live v35 postflight: verified full_evidence backup receipt at /realm/staging/polylogue-sqlite/recovery/lkrc-v35-20260713T042736Z/polylogue-archive-20260713T042738Z/verification-receipt.json (all five SQLite tiers, 26,600 blobs). Exact v7/index-v35/user-v6 artifact completed all watcher catch-up chunks with no recurrence of membership replay cannot retire an unrelated accepted head. Stopped-daemon census found 11 unknown-export-\u003eChatGPT session/raw mismatches. The three lkrc raws are quarantined full singleton censuses with canonical membership decision NULL and exact old unknown-key selected-baseline receipts; actuator now requires that narrow dual witness. The other 8 are excluded: 7 byte_proven unknown raws without membership/census, 1 byte_proven superseded-equivalent membership; separate follow-up required.\n2026-07-13 adversarial loop iteration 5 reached its cap with unresolved P0 proof gaps; do not merge/apply #2839 head 3b0ca3f08. Real residuals: (1) semantic canonical and historical sibling source envelopes omit capture_mode; require canonical provider when schema has field, with v7 fallback. (2) original unknown raw blob_ref.source_path is not bound to raw source_path in preflight/locked reproof. (3) original unknown raw native_id is not bound to reparsed provider session id preflight/locked reproof. (4) restore_canonical_head exact-byte route omits native_id, source_index, capture_mode, predecessor/append envelope fields; normalize conditional full-envelope proof for exact/semantic/sibling paths. Lower severity: historical supersession decided_at_ms accepts negative values. Iteration-5 reviewer found these against the current 31-test terminal closure; no live mutation after findings. Further implementation plus an operator-authorized review cycle is required before merge/apply.\n2026-07-14 code-verification pass (no live archive touched): re-checked the \"adversarial loop iteration 5\" proof gaps recorded in this bead's prior note against current master (031d8d183) source. All 3 named residual gaps -- (1) capture_mode binding, (2) blob_ref.source_path binding, (3) native_id binding into the preflight/locked reproof witness -- are already present in _browser_origin_source_envelope_is_exact (polylogue/storage/repair.py), which every browser-origin repair path (exact-canonical, semantic, and the restore_canonical_head route) now shares. Confirmed these landed via PRs #2843/#2847/#2848/#2850 (all merged after the iteration-5 note was written) by git log/git show on the relevant commits. AC1 (new browser captures acquire chatgpt-export origin, not unknown-export) is already covered by test_streaming_sized_browser_capture_json_uses_native_payload_detection in tests/unit/sources/test_live_batch_support.py, which asserts `SELECT origin FROM raw_sessions` == chatgpt-export for a fresh ingest.\nPR #2877 (branch feature/fix/raw-identity-repair-cluster) adds the evidence-packet + durable-blocker capability for this bead's dependent polylogue-lkrc.3 (the 4 sessions the exact-byte rekey actuator correctly refuses) -- see that bead's notes. AC4 (dynamic live census reports zero mismatches, or every unresolved conflict is an explicit durable blocking state) remains open pending a live-archive run of record_browser_canonical_authority_conflict_blockers, which this session does not perform (live-execution reserved for the operator). No code gap was identified beyond what #2877 adds; this bead's remaining scope is live-execution, not implementation.\n[2026-07-15 invariant-collapse pass] Expanded from the browser-origin incident into the shared raw-authority state machine evidenced by multiple separate repair classes in storage/repair.py. Supersedes lkrc.3, 57rp, and t0dy; their named live cases are regression/postflight inputs, not separate scheduled projects. Does not absorb yla8 because preventing stale replay is a different write-path invariant.\nLive evidence 2026-07-15 from MCP readiness_check: raw_frontier_integrity reported 1,890 broken active heads among 18,347 checked, 40 ingest cursors committed past accepted raw material, and 34 cursor/head authority rows not comparable. This is current measured debt, not a repair instruction; preserve the snapshot/frame and classify through the proof-driven reconciler before any cursor reset or replay mutation.\n2026-07-15 yla8 read-only preflight sharpened the live failure: packaged build 20d703e (source11/index36/user8) reports 1,890 broken active seeds, 40 cursor-ahead rows, 34 incomparable authority rows, and 15,264 direct / 21,398 expanded replay candidates. Journal shows ordinary convergence replaying exactly 2 logical sources per pass while the candidate count rose from 11,717 to 15,264 over four hours. Treat current rows as an immutable-frame census for RawAuthorityReconciler classification; do not reset cursors or run broad replay. hjpx owns the accepted-plan-to-fixed-point execution defect and is now P0 discovered from the failed yla8 gate.\n2026-07-16 implementation pass: owning the coherent lkrc/hjpx.1/lkrc.4 raw-authority cluster from fresh origin/master. Scope is the single reconciler/immutable-plan conservation and the production multi-session divergence regression now observed in packaged ordinary catch-up. Preserve yla8 fail-closed replay protections; no live cursor reset, force replay, evidence deletion, manual SQL repair, or live apply before reviewed code, verified backup, quiescent census, and explicit authorization. First deliverable is a production-route failing fixture and read-only live evidence.\n2026-07-17 PR #2962 closure implementation at edd68d240: the shared frontier now owns typed conflict disposition end to end. A conflicting browser head remains non-executable until its exact candidate judgment assertion is accepted and the blocker is resolved with disposition=retain_canonical_authority; the resulting immutable successor plan CAS-revalidates the complete competing-head witness, retains canonical authority, records supersession, retires the obsolete unknown-key head, and proves a terminal postcondition. Browser copy-forward/restore now also remove the obsolete head instead of leaving a corrupt residual frontier. Old incident receipt/mutator/CLI lifecycles were removed. Focused production-route verification: 183 passed across raw ledger, browser, quarantine, duplicate identity, daemon CLI, and maintenance CLI. Quick gate 20260717T000755Z-quick-1199839-9d926a5d: 16/16 green. Remaining closure boundary is the separately authorized stopped-daemon live gate in yla8; no live archive mutation was performed by this PR.\n2026-07-17 AC7 status-truth closure: PR #2965 (0dc5773a9) now persists complete frontier_state_counts alongside residual state_counts. Readiness exposes the complete inventory (including proven_current) while deriving blocking exclusively from postflight residual state. Dry-run and applied-postflight regressions passed (201 focused raw-authority/daemon/CLI tests; quick gate green). Remaining lkrc boundary is still hjpx fixed-point execution plus the separately authorized yla8 stopped-daemon live gate; no live archive mutation was performed.\n2026-07-18 inbox re-discovery check: /realm/inbox/download/PATCH(1) (2).diff (3627 lines; touches docs/cost-model.md, polylogue/archive/query/source_freshness.py, source_freshness_surfaces.py, cli/commands/diagnostics.py, core/evidence_value.py, core/temporal.py, daemon/status_snapshot.py, daemon/web_shell.py, insights/temporal_source.py, storage/usage.py). Patch base blob for docs/cost-model.md resolves via git cat-file to commit efadb404e (#3033, testdiet-06 admission, 2026-07-17) -- this predates 45+ subsequent master commits. git apply --check fails on 5 files including source_freshness.py itself (not just generated docs), confirming real drift, not just cosmetic. Its scope (cost-model.md, dual cost accounting, source freshness) overlaps two ALREADY-CLOSED beads: polylogue-5hf (provider token accounting) and polylogue-f2qv.3 (dual cost view) -- both closed before this patch's own base commit. named_source_freshness/NamedSourceFreshness already exist independently on master via PR #2924 (2026-07-16), predating this patch too. Verdict: superseded by already-shipped, already-closed work; not reconciled by hand given the drift depth and lack of any open bead this patch would newly satisfy. No action taken.\n2026-07-19 AC-closure audit (Sonnet audit lane, read-only, .agent/scratch/trust-floor-audit-2026-07-19.md has full detail): VERDICT = NARROWABLE (code-layer essentially complete; bead as a whole not closable because AC6's live postflight is explicitly not yet operator-authorized). Re-verified rather than trusted the prior notes.\n\nCode/tests (AC1,2,3,5,7,8): polylogue/storage/repair.py carries record_browser_canonical_authority_conflict_blockers, the frontier/residual state_counts pair (~line 4532), and the census/postflight machinery cited in the 2026-07-17 notes (#2962, #2965). Focused suite: devtools test tests/unit/storage/test_raw_authority_ledger.py tests/unit/storage/test_duplicate_raw_identity_repair.py tests/unit/storage/test_quarantined_accepted_raw_repair.py tests/unit/storage/test_comparative_judgment_assertions.py -\u003e 44 passed. Broader devtools test -k \"raw_materialization or raw_authority\" -\u003e 166 passed, 1 failed.\n\nThat 1 failure (tests/unit/sources/test_live_batch_support.py::test_live_multi_session_divergence_reopens_raw_authority) was investigated to ground truth rather than assumed pre-existing: it is test-currency drift from PR #3129 (de0b2df7a, landed 2026-07-18, intentionally redefines watcher-layer succeeded/failed semantics so an ambiguous/deferred membership decision folds into succeeded, not failed), NOT a regression against this bead's own AC1/AC4 invariants. Verified via a bypass probe (direct call into LiveBatchProcessor outside the stale assertion) that the deeper judgment/quarantine state this bead actually owns is fully intact: raw_session_memberships still records exactly 2 ambiguous/quarantined rows for the two divergent source paths, raw_sessions.parsed_at_ms is NULL for both with zero parse_error, and the index still resolves only the first-accepted head for chatgpt:shared with the original accepted_raw_id -- i.e. conflicting authority still cannot auto-select a winner, matching AC4. Filed polylogue-5202 (P2 bug) to fix the stale assertion; do not treat it as reopening lkrc's own scope.\n\nDependency hjpx: hjpx.1 (P0 correctness kernel: parser census before planning, immutable plan/outcome/postflight conservation, fair rotation, two-quiescent-census fixed point) is CLOSED via PR #2961/593ef3c62 with 5 independent adversarial passes and 19+82 focused tests green at that time -- re-confirmed present in current repair.py. hjpx.2 (P1 scale-proof at the July-15 archive cardinality, a live-archive resource-envelope proof) remains in_progress: its own notes record 4 consecutive honest self-aborts of the scale-proof generation phase under sustained host I/O pressure across roughly 140 minutes over two sessions (2026-07-18), explicitly \"not closable\" this session. That is a live-scale execution problem, not a code-correctness gap in the reconciler, and is out of this audit lane's authority to resolve (no live archive access).\n\nAC6 (stopped-daemon live postflight against the real archive, zero unreported frontier gaps) and the yla8 live gate it depends on are explicitly NOT authorized: yla8's own 2026-07-18 read-only preflight packet recommends \"DO NOT authorize the live gate yet,\" citing (1) no current verified full_evidence backup (most recent is 6+ days stale, predates the whole 07-15 authority program and 07-18 incident/restore), (2) the archive is mid-restore with only 170/79,571 raw artifacts materialized, making any current frontier-integrity reading a 0.2% unrepresentative sample rather than a population verdict, (3) measured watcher catch-up throughput (~0.185 files/sec, 1 worker) implies ~5 days just to drain the current gap without polylogue-5jak landing first, and (4) hjpx.2's scale proof above is itself incomplete. This is squarely an operator-gated live action, outside any coding agent's authority and outside this audit lane's mission constraints (no live archive access).\n\nNet: lkrc's single-reconciler architecture, typed conflict/judgment states, CAS revalidation, and status-truth surfaces are code- and unit-test-complete. The bead cannot be marked CLOSABLE as a whole because AC6 requires a live artifact this session correctly declines to produce; recommend keeping lkrc open with AC1/2/3/5/7/8 marked code-satisfied, AC4 code-satisfied via #2962, and AC6 explicitly blocked on yla8 operator authorization (not a further coding task for this bead).\n\nCommands run: devtools test tests/unit/storage/test_raw_authority_ledger.py tests/unit/storage/test_duplicate_raw_identity_repair.py tests/unit/storage/test_quarantined_accepted_raw_repair.py tests/unit/storage/test_comparative_judgment_assertions.py -\u003e 44 passed in 9.46s; devtools test -k \"raw_materialization or raw_authority\" -\u003e 166 passed, 1 failed in 46.06s (isolated re-run confirms deterministic, not xdist flake).\n\n2026-07-27: resolved 12 previously-stuck frontier_judgment blockers across 6 browser-rekey conversations (chatgpt:6a4629b3-8510-83eb-9180-b94a537abf7a and 5 siblings) via manual byte-level verification against the blob store - all confirmed safe retain_canonical_authority (10 straightforward content-hash matches, 2 where the automated detector's message-level diff hit FileNotFoundError and correctly deferred to manual judgment; direct diff confirmed identical message content in both, only incidental capture metadata differed). Also discovered and filed polylogue-rjtv: census regenerates duplicate judgment requests across cycles for the same underlying conflict instead of deduping against a still-pending one (roughly quadrupled this session's manual verification burden: 24 candidates reviewed for what was actually 6 real conflicts).\n2026-07-28: the standing 'No live apply is authorized' note in this bead is a per-session prohibition, not a permanent one, and it is currently the reason agents defer the whole P0 raw-authority cluster. The single operator decision that lifts it, plus the agent-side prerequisites that must be reported before asking, are written out once on polylogue-yla8 -- read that note rather than re-deriving the ask.","status":"in_progress","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:50:53Z","created_by":"Sinity","updated_at":"2026-07-28T20:04:37Z","started_at":"2026-07-16T19:20:39Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:browser","area:sources","area:storage","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-1xc","type":"parent-child","created_at":"2026-07-15T01:15:39Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-1xc.13","type":"relates-to","created_at":"2026-07-15T06:25:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-2qx","type":"related","created_at":"2026-07-15T01:09:46Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-b5l.1","type":"relates-to","created_at":"2026-07-15T20:42:23Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-yla8","type":"blocks","created_at":"2026-07-15T01:09:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc","depends_on_id":"polylogue-yla8.10","type":"discovered-from","created_at":"2026-07-13T01:50:54Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6407-6e37-7464-b88f-e043f6e0c88b","issue_id":"polylogue-lkrc","author":"Sinity","text":"[Dogfood 2026-07-15 / F-004] A named growing Codex source had an excluded cursor after five failures, later acquired raws unparsed, and a stale indexed session. Archive census showed 3,821 excluded cursors, 1,890 broken heads, 41 cursor-ahead rows, and 34 authority gaps. polylogue-1xc.13 owns the named-source acquisition-to-searchable projection and excluded-not-idle semantics. This reconciler remains the owner of underlying authority classification and repair population, so the beads are related rather than duplicating actuators.","created_at":"2026-07-15T04:27:16Z"}],"dependency_count":1,"dependent_count":0,"comment_count":1} {"_type":"issue","id":"polylogue-5ucz","title":"Fast-forward the live v32 index to v35 without raw replay","description":"The canonical 32 GiB index is healthy at user_version=32 but current code requires v35. A raw reparse is unnecessary and expensive: v33 widens one CHECK, v34 adds one index plus the current delegations view rewrite, and v35 changes three FTS tokenizers/write folds. Build and prove a clone-first fast-forward that leaves the original untouched, rebuilds only derived FTS tables from normalized source tables, and supports atomic blue-green activation with rollback.","design":"Implement an evidence-harness and operator actuator on a fresh branch from origin/master. Quiesce the user daemon; checkpoint/copy the v32 index using WAL-consistent handling and a Btrfs reflink under a contained single-operation scope. Apply exact canonical v33/v34/v35 DDL deltas to the clone, including the current delegations view definition, rebuilding all three contentless FTS tables with the canonical v35 tokenizers and folded write path through existing repair machinery. Set user_version=35 only after every mutation succeeds. Validate quick_check, foreign keys, exact canonical DDL, stable structural row counts, FTS population counts/folded-query smoke, and readiness on the clone. Emit phase/timing/hash/count/resource receipts. Activation is a same-filesystem atomic blue-green swap with retained rollback target; restart and postflight only after clone proof. No raw parse or durable-tier mutation.","acceptance_criteria":"1. A small v32 fixture proves exact 32→35 deltas, current delegations view, all three canonical FTS definitions/content, user_version-last behavior, and rollback on injected failure without raw parsing. 2. The live daemon is quiesced and the 32 GiB original remains byte/path preserved while a WAL-consistent reflink clone is created; receipts record source identity, sidecars/checkpoint state, timings, sizes, and resource envelope. 3. Clone mutation applies v33/v34/v35 canonical deltas and rebuilds messages_fts/work_events_fts/threads_fts from their source tables using current v35 folding/tokenizers; no session/message/block/source raw replay occurs. 4. Clone gates pass: integrity_check or quick_check as designed, foreign_key_check=0, canonical DDL exactness, unchanged sessions/messages/blocks and other structural counts, expected FTS counts, folded-query smoke, user_version=35, and current runtime readiness. 5. Only after a clone-only report is reviewed green, activation atomically swaps the canonical index to the proven clone on the same filesystem, retains the v32 rollback target, restarts the daemon, and proves bounded journal/readiness/query smoke. Any failure before activation leaves v32 canonical; any post-activation failure rolls back atomically. 6. Exact commands, timings, hashes/counts, PSI/RSS/IO samples, receipt paths, and no-raw-reparse evidence are attached. No v35 rebuild through ordinary raw ingestion.","notes":"Deployment/postflight completion:\n- Sinnix polylogue input advanced eff7c2a→58691ab and canonical devshell switch completed; deployed package /nix/store/acgsm0akngfg6jg23cllnx22xxl83hgy-python3.13-polylogue-0.1.0.\n- Current runtime also required durable user.db v4→v6. Used verified user_overlays backups at /realm/staging/polylogue-sqlite/recovery/user-v6-20260713/polylogue-archive-20260712T230342Z and /realm/staging/polylogue-sqlite/recovery/user-v6-step2-20260713/polylogue-archive-20260712T230517Z. Runner correctly refused stale-manifest reuse between migration steps.\n- Final: index user_version=35; user user_version=6; user quick_check=ok; foreign_key_check empty; annotation_schemas, annotation_batches, context_deliveries present; delegation.discourse v1 registered.\n- polylogued active/running PID 1943471, NRestarts=0; no storage schema mismatch; 8/8 live sources; browser spool ready; ports 8765/8766 owned by the integrated daemon. Receipt postflight field updated and hash refreshed.\n- PR #2804 merged as 07fbbeeca1c298aae6a964712374d4c40aa81e1f. GitHub-hosted checks did not start because the account is billing-locked; local owning tests and two quick gates were green, and no review threads/actionable bot findings existed.","status":"closed","priority":0,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T19:57:48Z","created_by":"Sinity","updated_at":"2026-07-12T23:06:53Z","started_at":"2026-07-12T19:57:54Z","closed_at":"2026-07-12T23:06:53Z","close_reason":"Delivered and live: clone-first no-raw v32→v35 activation proven, deployed v35 runtime plus verified user v6 migrations, stable daemon/query postflight, retained v32 rollback, PR #2804 merged.","labels":["area:ops","area:storage","area:test","delivery:B-storage-rebuild-bytes","horizon:frontier","lane:storage-rebuild-scale","spine"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-jlme.2","title":"Fail closed and preserve first-party auth for browser backfills","description":"Live deployment of PR #2771 proved a provider-contract failure: an authenticated ChatGPT UI with visible history returned HTTP 200 total=0/items=[] to the extension background adapter, which accepted the empty inventory as complete. The frontend itself requests the same inventory family with first-party page context and visibly receives history. A background fetch must not silently convert missing page/auth/account context into a successful empty archive delta. Fix ChatGPT and audit Claude transport while honoring provider controls, keeping secrets ephemeral, and avoiding foreground activation or broad live crawling.","design":"Evidence first: capture a bounded frontend inventory request through CDP and compare only header names, initiator/context, status, and response shape with the extension request; redact all credential values. Rank cookie context, account header, device/session token, and execution-world differences before choosing a transport. Implement a main-world/page bridge or equivalent ephemeral authenticated transport so provider-native inventory/fetch calls execute in the first-party context. The service worker remains coordinator/storage owner. Bridge messages use request IDs, a strict allowlist of provider-relative endpoints/methods, fixed timeouts, response-size bounds, and fail-closed shape/auth/challenge handling. Never persist or log tokens/cookies/account identifiers. Provider 200/empty must be distinguished from trustworthy empty inventory using authenticated-context proof or consistency checks. Audit Claude under the same contract and share the transport abstraction where viable. No foreground activation.","acceptance_criteria":"1. A production-path fixture reproduces HTTP 200 empty inventory from an unauthenticated/background context while a page-context fixture has history; the adapter refuses to mark the former complete. 2. ChatGPT inventory and native fetch can use a strictly allowlisted first-party page/main-world bridge without persisting or logging credentials, and auth/challenge/timeout/oversize/drift fail closed. 3. Claude transport is either moved to the same authenticated-context mechanism or has evidence-backed proof its existing background requests carry sufficient context; no silent empty success. 4. Memory/fake-IndexedDB coordinator tests prove a rejected empty inventory remains paused/actionable and resumes without duplicate capture. 5. Packaged service-worker proof exercises bridge request/response correlation and confirms no foreground tab activation. 6. Bounded live deployment against the owned private-visible profile returns a nonzero inventory count consistent with visible history, then a conservative job starts under configured rate limits. No archive rebuild or v35 work.","notes":"Discovered after merge 07ea5f2d0 / PR #2771. Initial live evidence: ChatGPT background request /backend-api/conversations?offset=0\u0026limit=100\u0026order=updated returned 200 total=0/items=[]; frontend resource used offset=0\u0026limit=28\u0026order=updated\u0026is_archived=false\u0026is_starred=false while sidebar visibly showed history. Investigation may inspect credential header names but must never record values.\nClosure evidence 2026-07-12:\n- PR #2773 squash-merged as 901825ec4acbf278ad184a004acf604048508174.\n- Production transport executes strict structured operations directly in the authenticated first-party MAIN world; no postMessage trust or credential persistence. ChatGPT traverses all archived/starred partitions; Claude pins the exact UI-selected organization. Responses are streamed under a 32 MiB cap and temporary background tabs are lifecycle-bounded without foreground activation.\n- Verification: browser-extension npm test 158/158; focused 58/58; npm run lint clean; npm run validate manifest v0.1.0 valid; devtools verify --quick 15/15 (20260712T202234Z-quick-3863858-c3dff574). Adversarial and Codex findings were fixed; all substantive threads resolved. GitHub-hosted jobs failed before runner allocation (empty runner/steps), while GitGuardian and CodeRabbit passed.\n- Bounded live deployment in private-visible profile with cutoff 2026-04-23: ChatGPT inventory_complete=true with 477 eligible candidates and durable ACK polylogue-ext-mri9iyo5-9v0liypp; Claude inventory_complete=true over 900 provider records with 26 post-cutoff candidates, exact selected organization pinned, and durable ACK polylogue-ext-mri9j03e-ol7rdst0. Both live jobs run at base cadence 10s, max 800 provider cost units/day, concurrency/captures-per-wake 1, retaining Retry-After, full jitter, and circuit breaker behavior. No auth or rate-limit failure.\n- Live receiver compatibility probe posted the exact stored 1,174,387-byte envelope and received HTTP 202 with a 64-character content_hash matching the extension SHA-256.\n- The original false-zero job was cancelled and never resumed. Its in-profile ledger was subsequently lost when earlyoom killed Chrome and the private-start helper destructively re-seeded the profile; this is recorded honestly rather than reconstructed. Follow-ups: polylogue-jlme.3 (stale receiver contract handling) and polylogue-jlme.4 (ledger-preserving browser recovery/profile reseed).\n- Host evidence: earlyoom acted at ~2-3% available RAM with swap exhausted and killed Chrome renderers plus many 1-2.4 GiB codebase-memory-mcp processes. The backfill itself remained single-request and was not the pressure source.\nPost-closure live continuation: Claude job backfill-claude-ai-1783888873491-d7l8y reached COMPLETE with 25 durable captures, one explicit no_turns, zero retry/error/operator-action backlog, and final ACK polylogue-ext-mri9m6p9-0x0xjckx. ChatGPT job backfill-chatgpt-1783888873491-bdvr57 remained RUNNING at 17/477 durable captures, zero retry/error/operator-action backlog, under the requested 10s/800-cost/one-capture policy. The diagnostic popup and extension-created Claude tab were closed; the pre-existing active ChatGPT tab remained foreground and was never programmatically activated. The merged feature worktree is intentionally retained temporarily because Chrome loaded the unpacked extension from that exact path; removing it while the background job runs would break MV3 worker restart.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T19:32:21Z","created_by":"Sinity","updated_at":"2026-07-12T20:48:40Z","started_at":"2026-07-12T19:32:27Z","closed_at":"2026-07-12T20:47:22Z","close_reason":"Delivered by PR #2773 / merge 901825ec with every acceptance criterion verified locally and bounded live ChatGPT+Claude inventories plus durable receiver ACKs.","labels":["area:ingest","area:web","delivery:G-live-performance","horizon:frontier","lane:capture-reliability","spine"],"dependencies":[{"issue_id":"polylogue-jlme.2","depends_on_id":"polylogue-jlme","type":"parent-child","created_at":"2026-07-12T21:32:21Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-jlme.2","depends_on_id":"polylogue-jlme.1","type":"discovered-from","created_at":"2026-07-12T21:32:22Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-yla8.10","title":"Repair accepted heads backed by untyped single-session raws","description":"The exact live v32 frontier has three active ChatGPT byte heads whose accepted_raw_id exists durably but still has no typed revision envelope: logical_source_key/source_revision are NULL, revision_kind=unknown, revision_authority=quarantined. The accepted index head/session is therefore not reconstructibly bound to source authority, and raw-frontier integrity correctly fails. Cursor-only yla8.6 repair cannot affect these rows. The retained v32 package at commit 3423d3c would classify a repeated single-session full as QUARANTINED, so ordinary re-acquisition alone remains false-green. Add a typed, evidence-preserving path that repairs this exact state without deleting or laundering raw/blob/head/receipt/session evidence.","design":"Recognize only the narrow already-accepted-untyped state: one current raw_revision_head and session raw_id agree on the same retained raw; source raw is unknown/quarantined with no prior logical/source binding; retained blob bytes normalize through the production ingest fallback-timestamp path to exactly the head session identity/content hash; SHA-256(payload) equals accepted_source_revision; byte length equals accepted frontier; raw row, raw_payload blob_ref, optional raw_artifact, origin, path, size, and source_index agree; the one immutable selected_baseline application receipt exactly equals the head including decided_at; and no competing head/application/membership/typed logical-key authority exists. Dry-run emits per-target and aggregate proof digests. Apply requires the exact digest/list and an explicit operator receipt path. Exclusively create and fsync a planned recovery receipt containing every witness, acquire ActiveWriterLease, open source.db as the sole writable main with index.db attached read-only, BEGIN IMMEDIATE once, reprove all targets, CAS-refine every envelope, reprove the terminal state, and commit all-or-nothing. Then fsync an applied record to the append-only operator receipt. Restart from a matching planned receipt is idempotent: exact already-bound rows finalize; any mismatch refuses. The existing immutable raw_revision_application proves prior acceptance and is cited, never mutated or duplicated. Do not weaken CAS, infer authority from raw_id alone, overwrite a typed envelope, misuse hook/ops tables, or delete evidence. Keep the actuator schema-v32-compatible and produce an exact v32-based build/artifact before live use.","acceptance_criteria":"1. Real-route fixture creates the exact invalid state through production write/receipt paths. Dry-run names each eligible raw, every witness, a per-target proof digest, and a deterministic aggregate digest without mutation; duplicate ids are rejected. 2. Apply requires that exact digest/list and an exclusive operator receipt path. It fsyncs planned evidence, acquires the writer lease, reproves under one source-main/index-readonly BEGIN IMMEDIATE transaction, CAS-refines all envelopes, reproves, commits all-or-nothing, and fsyncs applied terminal evidence. Raw/blob/session/head/content/message/FTS/application state is unchanged except the intended source authority columns. 3. Mutations for head/raw disagreement, missing or changed blob, blob-ref/artifact mismatch, byte-length/frontier drift, production-normalized parser/content-hash drift, wrong origin/session identity, competing head/application/typed revision/membership (including failed or ambiguous census), receipt/head field or decided_at drift, multi-session ambiguity, and pre-existing non-null envelope all fail closed with logical state unchanged. 4. Reapply with the matching applied receipt is idempotent. A planned-only receipt plus a partially/fully already-bound exact set resumes and finalizes; target/digest mismatch refuses. Injected proof/CAS/post-proof failures roll back the entire source batch and never leave a source binding without the pre-existing immutable application proof plus planned operator receipt. 5. Focused real-route storage and CLI tests pass, including anti-vacuity mutations. No schema changes; build the actuator from an exact INDEX_SCHEMA_VERSION=32 base containing all authority fixes through #2723, record build commit/hash, and run devtools verify --quick. 6. Live postflight only after merge and exact v32 artifact: stop daemon, verify source/user backup and dynamic census, dry-run exactly the current invalid raws, apply with stored operator receipt, then cursor-only yla8.6 repair/catch-up. Final exact census is 0 invalid heads and 0 cursor-ahead; explain incomparable gaps; source/index/hash/count parity and bounded journal are clean; controlled sanitized-copy append advances exactly once without shrink. No rebuild is an implementation prerequisite.","notes":"AUTHORITATIVE SCOPE SUPERSESSION (2026-07-13): this note overrides the stale v32-only clauses in the original description/design/AC. Exit condition for yla8.10 is: merged v35-compatible actuator; exact dry-run and receipted apply for only a7d004c9..., f19944c8..., fa0574f8...; those three reach byte_proven with all non-source-envelope state unchanged; reapply is idempotent; postflight proves those raw IDs no longer invalid. It is NOT an exit condition for yla8.10 to repair 282983b4..., 86298651..., or affadd9d..., nor to make the global byte-quarantined census zero: those three fail origin/parser equality and are exclusively owned by polylogue-lkrc. No v32 package/build/artifact or v32 rebuild is required or permitted for this closure.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T18:45:47Z","created_by":"Sinity","updated_at":"2026-07-13T00:44:05Z","started_at":"2026-07-12T18:48:37Z","closed_at":"2026-07-13T00:44:05Z","close_reason":"Merged PR #2808 (3a5102b843) and source-v7 compatibility PR #2811 (c1d3c1fbc). Live stopped-daemon postflight repaired exactly a7d004c9..., f19944c8..., fa0574f8... under aggregate proof 8735245c... with verified 53.1GB blob/durable backup at /realm/staging/polylogue-sqlite/recovery/yla8-10-authority-20260713/polylogue-archive-20260713T003259Z. Receipt source-authority-repair.jsonl is planned→applied and names exactly those three. Backup comparison: source quick_check ok, FK0, relevant counts equal, all non-target raw rows identical, each target changed only logical_source_key/revision_kind/source_revision/baseline_raw_id/acquisition_generation/revision_authority. Reapply repaired=0 and receipt stayed 2 lines. Daemon restarted stable PID 2241036 NRestarts=0; Drive catch-up 0 errors; repaired cohort invalid=0. Remaining three unknown-export origin mismatches are explicitly excluded and tracked P0 polylogue-lkrc.","labels":["area:daemon","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","lane:operational-resilience","spine"],"dependency_count":0,"dependent_count":1,"comment_count":0} @@ -44,7 +44,7 @@ {"_type":"issue","id":"polylogue-yla8.2","title":"Stop terminal revision receipts from re-entering replay","description":"Production evidence on 2026-07-11: packaged ordinary replay ran three ~190-200s passes, each reporting 15 replayed logical sources while remaining candidates rose 391→393→395 and quarantine rose 176→178→180. The candidate query excludes only deferred receipts and therefore requeues raws already terminally classified selected_baseline/applied_append/superseded/ambiguous. This creates an infinite expensive daemon convergence loop.","design":"In polylogue/storage/repair.py::_raw_materialization_candidate_ids, treat immutable raw_revision_applications receipts as the terminal authority for that exact raw. Exclude terminal decisions from executable candidates; preserve deferred incomparable state as visible blocked readiness. Prove against the real candidate route, including selected, superseded, ambiguous, deferred, and a newly acquired unreceipted raw. Ensure remaining-count computation uses the same predicate.","acceptance_criteria":"1. A selected/superseded/ambiguous/applied raw with an immutable receipt is not selected again. 2. A deferred incomparable receipt remains visible as blocked adoption debt, not executable work. 3. A genuinely unreceipted raw remains executable. 4. Two consecutive ordinary repair calls reach a fixed point: the second performs zero replay and does not grow terminal receipts. 5. Focused tests and devtools verify --quick pass; packaged production no longer loops and root session does not shrink.","status":"closed","priority":0,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-11T04:16:33Z","created_by":"Sinity","updated_at":"2026-07-11T07:17:26Z","closed_at":"2026-07-11T07:17:26Z","close_reason":"Merged PR #2690 (90cf639b1). Terminal application receipts now retire superseded/deferred/ambiguous decisions without repeat replay; production governed backlog is zero executable candidates.","labels":["area:daemon","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","lane:operational-resilience","spine"],"dependencies":[{"issue_id":"polylogue-yla8.2","depends_on_id":"polylogue-yla8","type":"parent-child","created_at":"2026-07-11T06:16:32Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-yla8.1","title":"Fail closed on authority-ambiguous raw replay","description":"Emergency containment for yla8. Until typed per-session raw revision authority exists, no source-to-index raw replay executor may apply historical revisions. Live packaged-runtime dogfood replayed old Codex snapshots over an 8k-message current session twice. A nominally empty index is not sufficient authority because multiple historical full revisions can still converge to the wrong snapshot. Leaving derived raw debt pending is preferable to silently accepting the wrong session.","design":"Make daemon repair and direct maintenance rebuild fail closed before parser or index mutation whenever raw rows are selected. Preserve read-only candidate/backlog and rebuild --plan inspection; remove ambient force-write and execution-only controls. Surface stable blocked candidate counts and reason in readiness/status telemetry, route every tier/blob lookup through the resolved archive file-set root, and prove no index or FTS mutation. This is containment, not yla8 closure: the parent owns typed per-session revision authority, ordered baseline/suffix replay, crash-resume decisions, and re-enabling execution.","acceptance_criteria":"1. Any selected raw replay candidate causes daemon repair and direct rebuild execution to return a stable blocked reason with zero session/index/FTS/raw-marker mutations. 2. No production route exposes force_write or an empty-index replay escape; rebuild --plan remains read-only and useful. 3. Backlog/status returns execution_blocked, reason, and blocked_candidate_count, including split-root routing through the resolved archive file set. 4. A regression seeds a newer indexed session and an older raw full snapshot; the real repair route preserves exact hash/message IDs/count and FTS rows, and fails if parser construction occurs. 5. Packaged live proof retains root session 019f49d8-0185-7c43-8793-db6e57db13e1 at or above the 8,076-message recovery snapshot across a daemon catch-up tick after deployment; devtools verify --quick passes. 6. Parent yla8 remains open and all operator text calls this temporary containment.","notes":"2026-07-10 adversarial correction: the original empty-index escape was misframed. Empty derived state does not establish which of several historical full revisions is authoritative. Scope now blocks every source-to-index replay executor while retaining plan/status inspection; parent yla8 must supply the typed authority model before execution is re-enabled.\n2026-07-10 PR #2670 merged as 202a09c240. Code/contract ACs are satisfied: all replay executors fail closed, planning remains, split-root/status/parser/FTS regressions pass. Keep this bead in progress until the packaged cutover and bounded live catch-up prove the recovered root remains at least 8,076 messages.","status":"closed","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-10T19:26:43Z","created_by":"Sinity","updated_at":"2026-07-11T07:17:31Z","started_at":"2026-07-10T19:32:37Z","closed_at":"2026-07-11T07:17:31Z","close_reason":"Containment and typed successor completed across PRs #2670 and #2681-#2695. Installed daemon catch-up retained the protected root at 9,298 messages; HTTP and MCP receipts agree, and raw readiness has zero critical/actionable debt.","labels":["area:daemon","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","lane:operational-resilience","spine"],"dependencies":[{"issue_id":"polylogue-yla8.1","depends_on_id":"polylogue-yla8","type":"parent-child","created_at":"2026-07-10T21:26:42Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-nkmy","title":"Unify active archive identity across split tier paths","description":"Live incident recovery on 2026-07-10 discovered two writable derived indexes sharing durable tiers. Packaged get_config() resolves archive_root=/home/sinity/.local/share/polylogue and index.db=/home/sinity/.local/share/polylogue/index.db (32.6 GiB), while source.db/ops.db/user.db/embeddings.db are symlinks into /realm/db/polylogue. A separate /realm/db/polylogue/index.db (26.5 GiB) remained writable and was used by the transient runtime and by operator verification, producing contradictory session counts (8,076 in the packaged active index versus 360 in the realm index) and a false recovery verdict. Archive identity cannot be inferred from the directory string when tier paths alias and derived index paths diverge.","design":"Define one typed ArchiveIdentity from resolved tier realpaths/inodes plus active index generation, not archive_root text. Every daemon, maintenance command, MCP/server, status probe, and writer capability must resolve and report that identity before opening a write connection. Two runtimes sharing any durable source/user tier but targeting different writable index generations must conflict/fail closed unless one is an explicit isolated rebuild generation owned by the blue-green protocol. Preserve symlink layouts if intentional; the invariant is one authoritative active index per durable archive identity. Quarantine/migrate the obsolete realm index only after backup and parity evidence; do not delete it as cleanup.","acceptance_criteria":"1. A fixture with source/ops/user symlinked across roots and two distinct index.db files deterministically fails startup/write preflight before either index mutates. 2. ArchiveIdentity is shared by daemon, direct maintenance, CLI/API/MCP status, and b5l writer capability; path aliases resolving to the same files compare equal. 3. Status reports configured and resolved tier paths, inode/device or stable identity, active generation, executable/build, unit/process, and conflicts. 4. An explicit blue-green rebuild generation can coexist read-only/inactive only under typed generation ownership and cannot become active without atomic promotion. 5. Sanitized live proof shows daemon, CLI, MCP, and direct verification resolve the same active index and return the same root-session hash/count; mutation tests fail when any surface falls back to archive_root/index.db string concatenation. 6. The obsolete /realm index is backed up and quarantined or reconciled with an operator-visible receipt; no destructive deletion is automatic.","notes":"2026-07-11 production evidence: PR #2680 (36147f29c) added archive identity containment and PR #2685 (a2bbd25d6) added typed inactive generation ownership/promotion. The obsolete v24/v30 indexes were backed up and quarantined; v32 generation gen-1783732901896-284abd9a was atomically promoted. /realm/db/polylogue/index.db and /home/sinity/.local/share/polylogue/index.db now resolve to that same generation and device/inode 3a:913945. Packaged CLI direct status reports active root /realm/db/polylogue, index v32, all five tiers; packaged daemon is healthy; a packaged MCP subprocess resolves the incident root at 9,298 messages, matching direct SQLite. Keep open until the final post-actuator quiesced proof records all surfaces in one receipt.\nArchitecture reconciliation 2026-07-16: polylogue-8jg9.6 adds a separate persistent logical archive lineage for restore/receipt continuity. It must not replace this bead's shipped path/inode/generation ArchiveIdentity, which remains the active file-set split-brain guard.","status":"closed","priority":0,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-10T19:25:47Z","created_by":"Sinity","updated_at":"2026-07-16T16:19:39Z","closed_at":"2026-07-11T07:17:33Z","close_reason":"Canonical file-set and blob aliases converge on /realm/db/polylogue. Installed CLI, daemon HTTP, MCP, and direct SQLite all resolve protected root codex-session:019f49d8-0185-7c43-8793-db6e57db13e1 at 9,298 messages. Production receipts are under /realm/staging/polylogue-sqlite/recovery/20260710T225846Z/receipts.","labels":["area:daemon","area:ops","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","lane:operational-resilience","spine"],"dependencies":[{"issue_id":"polylogue-nkmy","depends_on_id":"polylogue-9itr","type":"relates-to","created_at":"2026-07-15T06:25:24Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-nkmy","depends_on_id":"polylogue-b5l.1","type":"relates-to","created_at":"2026-07-10T21:25:48Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-nkmy","depends_on_id":"polylogue-n2wy","type":"relates-to","created_at":"2026-07-10T21:25:48Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-nkmy","depends_on_id":"polylogue-yla8","type":"relates-to","created_at":"2026-07-10T21:25:48Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6407-2dcc-7b70-bfe0-db7e7de68b7d","issue_id":"polylogue-nkmy","author":"Sinity","text":"[Dogfood 2026-07-15 / F-001] The active index identity itself is now correct, but config paths still follows the index symlink and reconstructs all other tiers under the index-only generation. It reports four existing tiers missing even though the configured root and ordinary multi-tier reads are healthy. Follow-up polylogue-9itr owns this narrower diagnostic regression and is related here so nkmy closure evidence is not mistaken for current config-path parity.","created_at":"2026-07-15T04:27:00Z"},{"id":"019f6abe-7434-7a8a-b7fe-0c0e90221ab4","issue_id":"polylogue-nkmy","author":"Sinity","text":"CLOSURE-DISCIPLINE NOTE, not a reopen (dogfood-2 round-4 verification, investigations/nkmy-archive-identity-verify.md): the core incident-shape invariant this bead fixed is genuinely solid -- ArchiveIdentity (storage/archive_identity.py) is real, tested (tests/unit/storage/test_archive_identity.py), and correctly wired into the two call sites that matter for preventing a write-time split (ArchiveStore.__init__/archive.py:1006-1045, daemon startup/daemon/cli.py:1035-1041) -- AC#1s fixture (two distinct index.db files -\u003e deterministic pre-mutation preflight failure) is real and passing, not contradicted by anything live today. However AC#2s literal text (\"shared by daemon, direct maintenance, CLI/API/MCP status\") and AC#5 (\"mutation tests fail when any surface falls back to archive_root/index.db string concatenation\") are NOT satisfied today: MCPs only status tool (readiness_check) never imports archive_identity and, confirmed via a live call this session, returns a bare archive_root path string with no generation/inode/conflict data; the API layers DaemonStatusSurface Protocol (api/contracts/read_surface.py:104-115) has zero implementers; CLI config paths (a primary operator-facing diagnostic) never imports archive_identity either and was confirmed LIVE, right now, to silently substitute ~200-500KB stub files from an in-flight index-generation directory for the real 204MB source.db/5.6GB embeddings.db at the durable root -- reporting storage_layout: archive_complete while looking at the wrong files entirely. This is a more dangerous silent variant of exactly the symptom polylogue-9itr described (9itr was closed \"superseded by ovme ArchiveLocation\"). The beads own closing note (\"keep open until the final post-actuator quiesced proof records all surfaces in one receipt\") was correct and prescient -- \"all surfaces\" was never actually achieved. Not reopening: the specific gap is already precisely scoped under polylogue-ovme/ovme.1 (open, P1), whose own design text already names this exact defect (\"no consumer can reinterpret a filename... reuse ArchiveIdentity instead of rebuilding siblings from the resolved index parent\") -- fresh live-reproduction evidence left as a comment there instead of duplicating scope here.","created_at":"2026-07-16T11:44:54Z"}],"dependency_count":0,"dependent_count":0,"comment_count":2} -{"_type":"issue","id":"polylogue-yla8","title":"Run the authority-safe raw replay closure gate","description":"The stale-replay incident implementation has landed through the yla8 child series and merged authority PRs. The remaining P0 work is not to redesign replay ordering: it is to prove on the current packaged runtime and active archive that the implemented revision authority, chain retention, typed actuator, cursor discipline, and readiness checks close the original no-shrink invariant. Earlier notes explicitly left the parent open because no final live closure audit was run after yla8.10. A coding agent must not redo or replace the landed mechanism unless this gate produces a concrete regression.","design":"Treat this as an operator-authorized live gate with a read-only first phase. First audit current master and the merged child receipts against original AC1 through AC5, naming the exact production functions and regression tests; if any implementation contract is absent, open a narrowly scoped P0 child and stop before live mutation. Then capture an immutable before receipt for the active archive: package/build commit and schema versions, daemon state, verified source/user backup manifest, exact raw-frontier-integrity census, raw/head/application/cursor classifications, and the current source/index/hash/message/FTS state of the original long-lived Codex witness plus every dynamically reported invalid head or cursor-ahead path. Do not use historical expected counts as authority. If and only if the read-only audit is green and the operator authorizes live execution, run the packaged daemon ordinary catch-up under bounded journal capture without cursor reset, force replay, evidence deletion, or ad hoc SQL writes. Re-run the exact census and witness comparisons, then perform one controlled privacy-safe append through the ordinary ingest path and prove it advances once without shrink. Store before/after/build/backup/journal receipts under the established recovery receipt location. Any mismatch leaves the daemon in the safer stopped/degraded posture and becomes a new concrete P0 child; it does not reopen architectural choice inside this gate.","acceptance_criteria":"1. A source audit maps original invariant AC1 through AC5 to current production functions, merged PRs, and mutation-sensitive tests; no missing mechanism is hand-waved as covered by notes. 2. The read-only preflight records exact build/package/schema identity, daemon state, verified durable backup, current raw-frontier census with bounded samples, and source/index/head/application/cursor/hash/message/FTS state for the original witness and every dynamically implicated path. Unknown or unavailable evidence cannot render green. 3. No live mutation occurs without explicit operator authorization after the preflight. The live phase uses only packaged ordinary convergence and the existing typed actuators under their authorization contracts; no cursor reset, force replay, raw/blob/head/receipt deletion, or manual SQL repair is allowed. 4. Post-catch-up exact census reports zero invalid active heads and zero cursors ahead of accepted material, or every nonzero row is a typed durable unresolved state with evidence and a newly opened P0 child. No older/incomparable replay error occurs in the bounded journal interval. 5. The original Codex witness and all dynamic samples retain or advance source/index content hash, exact message identities/count, composed transcript, and FTS parity; no session shrinks or changes authority without a typed receipt. 6. One controlled privacy-safe append through LiveBatchProcessor advances cursor, revision head, session content, and FTS exactly once; a second unchanged tick is zero-work. 7. The closure artifact includes build, backup, preflight, action authorization, daemon journal, postflight, and append receipts. Only this evidence closes the parent; any failed clause creates a narrowly scoped successor and leaves the archive safe.","notes":"2026-07-10 live follow-up: the packaged active index is /home/sinity/.local/share/polylogue/index.db, not /realm/db/polylogue/index.db. One-shot acquisition+parse of only raw 6a74735e restored the active root to 8,076 messages; the realm index remains at 360 and is tracked by polylogue-nkmy. Both Codex Cloud attempts were rejected: attempt 1 sorts selected force replay by non-authoritative path/acquisition metadata and still regresses newer indexed state; attempt 2 buffers all parsed payloads, conflates provider timestamps with revision authority, and bypasses browser precedence. Robust closure requires typed per-session revision evidence/application decisions, baseline-then-append replay, terminal/deferred raw markers, and a rebuildable application ledger. No cloud diff was applied.\n2026-07-11 implementation/live-rebuild evidence: PR #2681 (2032b2cb2) added durable source-v5 revision authority; PR #2684 (6a579d090) added source-v6/v7 membership authority, index-v32 application receipts, deterministic baseline/suffix and bundle replay, CAS frontiers, and scoped FTS verification; PR #2686 (3fe7837c8) bounded production census memory. The offline production rebuild classified 17,449 full revisions, replayed 17,489 logical sources from 18,013 retained raws, quarantined 615 ambiguous raws, and promoted an exact-sized v32 generation. Root session codex-session:019f49d8-0185-7c43-8793-db6e57db13e1 now has 9,298 messages and 4,270 tool-use blocks. The first packaged daemon catch-up retained the exact count/hash but correctly reported ordinary raw replay still containment-blocked; keep in progress until the typed actuator lands and a final packaged tick proves no shrink. Receipt: /realm/staging/polylogue-sqlite/recovery/20260710T225846Z/receipts/index-v32-generation.json\n2026-07-11 packaged-daemon postrepair catch-up exposed the remaining typed-actuator gap directly. After yla8.6 repaired all broken append heads/cursor-ahead rows, the one-time modern cursor reauthentication selected 669 legacy files (5.1255GB). Chunk 1 rejected two full replays while preserving the accepted index: (1) a 25,898,236-byte ChatGPT browser capture for session 69d5383e-69d0-8327-a899-94a89ff35ea4 hit \"conflicting accepted head\"; the existing semantic head comes from a 15,890,659-byte account-export member and the browser capture is a separate single-session acquisition route with provider updated_at 2026-07-01, so the len(sessions)==1 byte-replay path collides with the prior multi-session membership head instead of running one cross-route semantic authority decision; (2) a Gemini CLI full replay hit \"older accepted frontier\". This is exactly why the parent remains open: strict CAS is correctly preventing regression, but ordinary replay lacks a typed terminal superseded/deferred actuator and cursor outcome. Current false-green risk: archive-authenticated cursor reconciliation can establish a complete cursor before the subsequent raw replay is rejected, so future hot skips may hide parse debt. Bounded journal starts 2026-07-11 23:36:33 CEST. Do not weaken CAS or delete accepted heads; route semantically comparable full/member revisions through one authority classifier, terminally receipt proven superseded inputs, and leave incomparable/conflicting content visible retry/debt without a success cursor.\n2026-07-11 correction after cursor inspection: the two rejected chunk-1 paths were not silently hot-skippable; _record_failed_cursor retained the last accepted boundary but set failure_count=1 and next_retry_at, so ordinary retry remains visible. The more serious live defect is the converse: apply_raw_revision_replay preserves an existing semantic frontier kind but compares only aggregate frontier cardinality. A single-session full capture from a different route can therefore overwrite a membership-governed session when it has a numerically larger yet divergent projection; equal divergence conflicts and smaller candidates reject. The daemon was stopped successfully during chunk 3 before processing the remaining backlog. Repair branch feature/fix/typed-raw-replay-outcomes makes any single-session full whose logical key already has membership evidence join that census and use classify_membership_revisions. Proven older prefixes become terminal superseded_prefix with parsed raw evidence; larger divergence remains ambiguous, leaves the accepted index/head unchanged, and keeps retry/debt visible. Anti-vacuity: under origin/master the older-prefix real route fails with CAS and the larger-divergent route overwrites; the new tests require the former to succeed terminally and the latter to preserve the prior messages. Focused cross-route matrix 4 passed plus existing semantic-CAS rollback test passed; quick run 20260711T215004Z-quick-1328386-a6182ef1 passed 13/13. Full test_live_batch_support.py was 47 passed/6 failed; all six exact failures reproduce identically on a clean detached origin/master and are unrelated baseline failures.\n2026-07-12 typed actuator publication: branch feature/fix/typed-raw-replay-outcomes commit 7868046e6, PR #2716. The first independent review found dual-governance and reverse-arrival blockers; corrected by retiring only append-independent full byte cohorts into membership, excluding them from byte rebuild selection, and atomically transitioning a proven related byte head inside the semantic write transaction. The second review found metadata-equivalence timestamp laundering; corrected by removing browser capture's captured_at fallback and requiring pairwise-unique direct provider updated_at for every distinct metadata variant. Missing/equal timestamps remain ambiguous. Final review found no release blocker. Real-route coverage includes bundle-first, already-bound failed retry, rebuild selection, single-first reverse arrival, larger divergent capture containment, metadata-only strict provider ordering, missing/equal timestamp ambiguity, and capture-time non-laundering. Focused 7 passed; classifier 5 passed; semantic CAS rollback 1 passed; final-head quick run 20260711T220233Z-quick-1445786-3516ef36 passed 13/13. Six broader live-batch failures reproduce unchanged on clean origin/master. GitHub-hosted checks on PR #2716 failed pre-allocation under the known billing lock; GitGuardian passed and CodeRabbit is being triaged before merge.\n2026-07-12 closure audit/no-action decision: current exact v32 evidence is 3 invalid active ChatGPT raw seeds and 15 cursor-ahead rows; therefore parent AC 4/6 and live no-shrink postflight are not satisfied. Retained package 0.2.0+3423d3c is v32-compatible and contains the complete authority series through #2723, but its ordinary single-session full path preserves revision_authority=quarantined for the three already-accepted untyped raws. Resetting 15 disposable cursors cannot make those accepted source bindings authoritative. No live mutation or v35 rebuild was attempted. polylogue-yla8.10 is the P0 typed authority-rebinding/terminalization child required before a cursor-only postflight can close yla8.6 and this parent.\n2026-07-14 status check as part of the raw-identity-repair cluster (polylogue-lkrc/lkrc.2/lkrc.3/yla8/yla8.6/t0dy/57rp/5k5l.1, PR #2877): re-read this bead's full note history plus its closed child polylogue-yla8.10 (closed with live postflight evidence 2026-07-13). No additional code gap was identified for this parent beyond what its children's merged PRs (#2681, #2684, #2686, #2710, #2716, #2808, #2811) already deliver -- this parent's remaining open scope is a live-archive closure audit/postflight (its own notes: \"3 invalid active ChatGPT byte heads... insufficient for closure... yla8.10 owns the required typed repair\", and yla8.10 is now closed with exactly that repair applied live). This session did not run any live-archive verification or repair (reserved for the operator per this cluster's live-archive-safety constraint), so this bead is left open rather than claimed closed on unverified evidence. No PR-2877 commit touches this bead's own scope directly.\n[2026-07-15 invariant-collapse pass] polylogue-yla8.6 is not a second project: append CAS/frontier convergence and its live postflight are already explicit AC3/AC6 of this root prevention invariant. Its incident evidence is retained via supersession; yla8.8 remains distinct because it optimizes complete-prefix proof cost without changing ordering semantics.\nTerra-readiness correction 2026-07-15: the parent no longer presents already-merged implementation as open design work. It is now the explicit read-only audit plus operator-authorized live postflight. An unattended worker may complete preflight and source verification but must stop before live mutation without authority.\n2026-07-15 authorization preflight decision: REFUSE live yla8 catch-up/append authorization on current evidence. Read-only polylogue ops status --json --full against packaged daemon build 20d703e21703b9298c6bfa774617957bee9a5a97, source v11/index v36/embeddings v2/user v8/ops v1, reported raw frontier overall=violated with 1,890 broken active seeds of 18,347 checked, 40 cursors ahead across 739 comparisons, 34 incomparable cursor/head rows, 5 critical archive-debt groups affecting 207 artifacts, and raw materialization not converged. Replay backlog was 15,264 candidates / 21,398 expanded raws / 10,163 executable authority components / 4.788 GB expanded bytes, with 1,906 durable authority-debt rows, 28 quarantines, 1,588 governed append fragments, and 290 append-authority quarantines. Journal from daemon start at 18:30 CEST shows bounded raw materialization already mutating automatically: each pass replays 2 logical sources while remaining candidates monotonically grew 11,717 -\u003e 15,264; unsafe snapshot compaction repeatedly refused raw 0005f338... for lacking byte-proven authority. systemd reported 1.9 GiB memory at a 2 GiB maximum, 2 GiB peak, and full status showed active writer maintenance.raw_materialization with queue depth 12. The most recent verified full-evidence backup found is 2026-07-13 (57 GB at recovery/yla8-10-authority-20260713), not a current pre-action backup. Therefore AC2 is red and AC3 forbids proceeding: do not run an extra catch-up, controlled append, cursor reset, force replay, SQL repair, or evidence deletion. Existing automatic convergence itself now requires containment/diagnosis. polylogue-hjpx is the non-duplicative concrete P0 successor under lkrc for accepted-plan fixed-point execution; this live failure is linked discovered-from yla8. Authorization can be reconsidered only after current build-specific source audit, a new verified source/user/blob backup, stopped/quiescent exact census, zero unexplained broken/cursor-ahead rows (or typed durable blockers), bounded fixed-point dry-run/proof, and safe daemon resource posture.\n2026-07-17 read-only preflight receipt: /realm/tmp/yla8-preflight-2026-07-17/status-full.json. Live phase is refused, not started. Packaged daemon is active (PID 952725) on build a62d2f972, source v12/index v37/user v9; status overall error with stale lifecycle heartbeat (~6.1h), active raw-materialization writer, queue depth 22, and cgroup memory 2.14 GiB. Exact frontier: 2,875 broken active heads / 18,492 checked; 40 cursor-ahead rows / 797 comparisons; 45 incomparable cursor/head rows. Raw readiness has 46,308 join gaps, 37,617 unchecked and 235 parse-failed affected rows. Replay backlog is execution_blocked behind one non-stream-safe component, with 37,617 candidates, 32,568 authority components, 1,998 durable authority-debt rows, 107 authority quarantines, and 290 append-authority quarantines. Current durable backup and quiescent preflight requirements are not demonstrated. No daemon stop, catch-up, append, reset, replay, SQL, or evidence mutation occurred.\n2026-07-17 live-gate preparation only; no archive mutation: the scheduled systemd polylogue-sqlite-backup service is a compressed per-DB retention backup and does not produce the signed verified backup manifest required by a durable-tier/live-authority gate. The product backup route is available and its full_evidence prerequisite check passed against the active archive: POLYLOGUE_ARCHIVE_ROOT=/home/sinity/.local/share/polylogue polylogue ops backup --output-dir /realm/staging/polylogue-sqlite/yla8-next --profile full_evidence --check. When explicitly authorized after the schema window, run the same command with --verify into a fresh timestamped output directory; use its manifest.json and verification-receipt.json as the live-gate evidence. Do not reuse yla8-next, and do not begin a backup/census/catch-up/stop until the operator restarts the live phase.\n2026-07-18 ~14:45 Fable live-incident diagnosis (operator directed: fix poisoned state + restore live archive): live daemon is NIX-DEPLOYED polylogue 0.2.0 (sinnix pin ef17859b, predates the ENTIRE raw-authority hardening program). Live state: source.db intact (73,311 raw_sessions, 335M), user.db intact (204K), blob 66G intact; but .index-active-pointer targets /realm/db/polylogue/index.db which is 4KB EMPTY, a stale 91MB local index.db (4 sessions/5,358 messages) sits at ~/.local/share/polylogue/index.db, .index-rebuild.lock dated Jul 16 — a blue-green index rebuild started under the old daemon and never completed. This is the #3055 managed-index-identity bug class. Restore plan in flight: (1) verified full_evidence backup into /realm/staging/polylogue-sqlite/yla8-20260718T124820Z (running); (2) sinnix polylogue pin updated ef17859b -\u003e 20c07a087 (current master, all hjpx fixes); (3) prebuild package, then nix switch restarts polylogued on 0.3.0; (4) let daemon convergence rebuild the index tier from durable authority (rebuildable-tier operation, automagic path, NOT a live-authority hack); (5) re-check readiness + hjpx debt shape after drain. All live-archive numbers from any lane are PROVISIONAL until the drain completes (lane A flagged this first).\n2026-07-18 ~15:40 restore progress: user.db migrated 9-\u003e10 (additive, no manifest needed, receipt in CLI json). Stale v2 embeddings.db retired to /realm/db/polylogue/embeddings.db.v2-retired-20260718. Authority-safe full-corpus rebuild-index (73,311 rows, --raw-batch-size 80000) running into generation gen-1784381541560-2d4fc3f4, idle-scoped, daemon stopped. IDENTITY FINDING (confirms #3055 bug class on live data): TWO divergent index.db identities existed — ~/.local/share/polylogue/index.db was a REAL 91MB file (4 sessions; what readiness_check and lane A saw) while /realm/db/polylogue/index.db is a SYMLINK into .index-generations/gen-1784204285162 (Jul 16, 18,796 sessions). The daemon and CLI were resolving DIFFERENT indexes depending on path entry. The rebuild + promote flips the generation pointer; after promote, delete the orphaned home-dir index.db file and verify both entry paths resolve identically.\n2026-07-18 lane-D read-only authorization packet: /realm/worktrees/polylogue-lane-d/.agent/reports/yla8-authorization-packet-2026-07-18.md (commit 8cb672c02). Recommendation: DO NOT authorize the live gate yet. Four blockers: (1) no current verified full_evidence backup exists -- most recent formal per-tier backup is 2026-07-12T03:17:17Z (6 days stale, predates the entire 07-15 authority program and the 07-18 incident/restore); the informal pre-deploy-20260718T132033Z snapshot (source.db+user.db only, no manifest/verification-receipt) does not satisfy this gates AC2. (2) archive is mid-restore: fresh index generation gen-1784381541560-2d4fc3f4 promoted, daemon restarted 17:51 CEST, only 170/79,571 raw artifacts materialized (join_gap_count=79,401) -- July-15 frontier-integrity numbers (1,890 broken/40 cursor-ahead/34 incomparable) are NOT reproducible from current evidence since any current reading operates on a 0.2% unrepresentative sample, not a population verdict. (3) polylogue-5jak (P0, daemon conveyor 1-row/30s tick + startup Drive serialization) directly evidenced today: watcher catch-up measured files_per_second=0.185, ingest_worker_count_max=1 in the live ingestion-batch receipt -- at that rate the 79,401-row gap needs ~5 days for watcher catch-up alone, ~27.6 days for the separate raw_materialization conveyor per 5jaks own math. \"Let the daemon drain it\" is not currently viable without 5jak landing. (4) hjpx.2 (this lanes own scale proof) has not yet completed a fixed-point proof at July-15 cardinality -- corpus prep is retrying under the continuous I/O pressure gate (host contended, avg10 3.8-11.2 this session, 4+ concurrent warroom lanes). raw_replay_backlog and archive_debt were excluded from the bounded status snapshot this session (reason: excluded_from_bounded_status_snapshot) -- no immutable plan digest was captured; a dedicated raw-authority dry-run census was deliberately NOT run against the live archive to avoid contending with the daemons own active writer coordinator during an already-fragile restore. No live mutation performed. Packet ends with the single yes/no authorization question for the operator.\n2026-07-18 lane-D: PR #3122 opened (https://github.com/Sinity/polylogue/pull/3122) carrying the read-only authorization packet.\n2026-07-20: yla8-authorization-packet-2026-07-18.md was untracked from the repo by the .agent excision (PR #3180, operator directive). The packet persists on the operator host at .agent/reports/ in the main checkout; the controlling facts remain: read-only census on the restored archive is the closure gate, only a repair-execute needs operator authorization.\n2026-07-21 read-only closure census (.agent/reports/yla8-closure-census-2026-07-21.md, no mutation): AC1 SATISFIED (source audit maps revision authority/chain retention/actuator/readiness to live code with file:line; #3211 fail-open regression was caught+fixed by #3240 within 24h). AC2 NOT SATISFIED at census time — newest backups 2026-07-19T03:14Z predate the promote; fresh polylogue-sqlite-backup run started 2026-07-21 evening during the daemon-down window (re-verify manifest). AC4/AC5: byte-authority chain clean (76934/76934 byte_proven heads, 0 dup keys, FTS parity 4753541==4753541, 0 dangling branch points) BUT reproduced AC5 violation: named witness codex-session:019f49d8-… has ZERO index presence on v43 (no session row/head/application receipt; 9298 messages at v32; all 21 raw byte copies safe in source.db) — one of 154 logical sources system-wide with quarantined membership evidence and no resolved head. Closure blocked on explaining/repairing the quarantined-no-head cohort; repair-execute needs operator authorization per this bead. AC3/AC6 daemon clauses N/A (daemon down during census; 3.14t deploy in flight).\n2026-07-22: two product fixes merged from the census findings — #3255 (headless-cohort authority mislabel: equivalents stay quarantined-ambiguous without an accepted head; root cause of the 914 byte_proven-headless sources, investigation report .agent/reports/byte-headless-914-investigation-2026-07-21.md) and #3256 (t93b daemon whale convergence; witness component becomes daemon-resolvable). Free-threaded daemon deployed and verified on 3.14t. Fresh tier backups taken 2026-07-21 22:34 during daemon-down window (AC2). Hook sidecar-dir baked into sinnix (codex+claude) closing the /tmp spool leak; ~100MB leaked events pending operator salvage from /tmp/polylogue-archive/hooks. Critical path to closure: operator runs the staged blocker-resolution script, daemon converges (incl. whale pass on the witness), then re-census + AC receipts. NOTE: no CLI/MCP surface exists for raw-authority blocker resolution (API-only) — t46.9 phase-3 candidate.\n2026-07-27 fresh preflight attempt (post ihc8 deploy + daemon redeploy at 18:33 CEST): took and verified a genuinely fresh backup (polylogue-sqlite-backup.service manually triggered, source.db+user.db integrity_check=ok, dated 2026-07-27T19:56:07Z) -- AC2's backup clause is now satisfied for this moment. Read-only census via the real production ops-status surface (POLYLOGUE_ARCHIVE_ROOT=/realm/db/polylogue polylogue --plain ops status --full) shows raw_materialization degraded: 22,670 raw/index join gaps, ALL currently unclassified/unchecked (critical=0, warning=0, actionable=0, blocked=0) -- meaning no explicit operator-actionable blocker is currently flagged, but the classification pass itself hasn't been freshly run.\\n\\nDeclining to proceed to a live blocker-resolution/repair-execute action this session: journalctl shows the live daemon (PID 3952) experienced an abnormal ~2.6 hour writer-hold stall (append.raw_and_index_write hold_s=9327 for a single-file append) during this session's window, almost certainly from severe host-level resource contention this same session created (many concurrent worktree agents running heavy devtools verify/test/build + a manually-triggered backup job, all on one machine). This makes the current 22,670-gap census unreliable as a steady-state read -- it may be inflated by transient contention-induced backlog, not a stable population count. Per this bead's own design (a clean quiescent read is the precondition for any live-phase decision), re-run this exact census once the host is calm and no concurrent heavy agent activity is in flight, before making any live-mutation judgment call. The fresh backup from this session remains valid evidence for whenever that re-check happens.\n2026-07-27 ~20:14 UTC re-check: host is now calm (load 2.0-3.0 on 24-thread machine, no competing heavy processes, the earlier ~2.6h writer-hold stall has cleared). Re-ran raw_materialization census -- IDENTICAL numbers to the earlier contention-period read (22,670 join gaps, archive_session_count=18827 matching daemon heartbeat), so that earlier reading was actually accurate, not contention-corrupted as I'd cautiously assumed -- correcting my own earlier over-caution.\\n\\nRan the actual classification surface (polylogue ops debt list --format json) for the first time this session: 21 real debt rows, properly classified. 5 CRITICAL/actionable: FTS convergence debt for messages_fts (stale freshness ledger, observed since 16:33 UTC, not self-resolving despite continuous daemon uptime), and raw-materialization parse failures for 73 claude-code-session + 26 codex-session + 2 hermes-session + 28 unknown-export raw artifacts (129 total, validation_state mostly 'unknown' not a clean pass/fail). 13 WARNING/actionable: mostly ordinary trickle-backlog ('acquired but not yet parsed', matches t93b's known drain-rate finding) EXCEPT 134 rows across codex/hermes/chatgpt origins that are 'parsed but have no materialized session' -- the same class of gap as this bead's named witness (codex:019f49d8, which had exactly this shape: raws present+parsed, zero session/head). 1 WARNING/blocked: 18827 sessions pending embedding catch-up (separate, lower-priority, always-rebuildable tier). 3 INFO: ordinary assertion-candidates awaiting judgment.\\n\\nDispatched a dedicated read-only investigation (polylogue-a92969b6e4c8d728b) into the 5 critical parse-failures + FTS convergence debt specifically -- these are the most concretely diagnosable/fixable-in-code items (a validation-state classifier returning 'unknown' rather than pass/fail is itself worth understanding), distinct from the broader semantic-membership-classifier question the 134-row 'parsed but headless' cohort represents. Not touching the 134-row cohort or performing any live mutation without that investigation's findings first.","status":"in_progress","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-10T18:48:26Z","created_by":"Sinity","updated_at":"2026-07-27T20:16:46Z","started_at":"2026-07-11T03:07:38Z","metadata":{"authorization_status":"refused_preflight_red","execution_mode":"operator_authorized_live_gate","frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:daemon","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","lane:operational-resilience","spine"],"dependencies":[{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-1xc","type":"parent-child","created_at":"2026-07-15T01:15:39Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-1xc.13","type":"relates-to","created_at":"2026-07-15T06:25:37Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-b5l.2","type":"relates-to","created_at":"2026-07-10T20:48:43Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-n2wy","type":"relates-to","created_at":"2026-07-10T20:48:42Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6407-7ab6-7490-88b1-2c079b245cd7","issue_id":"polylogue-yla8","author":"Sinity","text":"[Dogfood 2026-07-15 / F-004] New live closure evidence: one actively growing Codex source remained behind because its cursor was excluded after five raw-revision CAS failures; later revisions were acquired but unparsed. Quiet-window deferral does not explain it. polylogue-1xc.13 owns the per-source diagnostic chain and population classification. This bead remains the prevention and postflight owner for replay ordering and accepted-head safety.","created_at":"2026-07-15T04:27:20Z"}],"dependency_count":0,"dependent_count":1,"comment_count":1} +{"_type":"issue","id":"polylogue-yla8","title":"Run the authority-safe raw replay closure gate","description":"The stale-replay incident implementation has landed through the yla8 child series and merged authority PRs. The remaining P0 work is not to redesign replay ordering: it is to prove on the current packaged runtime and active archive that the implemented revision authority, chain retention, typed actuator, cursor discipline, and readiness checks close the original no-shrink invariant. Earlier notes explicitly left the parent open because no final live closure audit was run after yla8.10. A coding agent must not redo or replace the landed mechanism unless this gate produces a concrete regression.","design":"Treat this as an operator-authorized live gate with a read-only first phase. First audit current master and the merged child receipts against original AC1 through AC5, naming the exact production functions and regression tests; if any implementation contract is absent, open a narrowly scoped P0 child and stop before live mutation. Then capture an immutable before receipt for the active archive: package/build commit and schema versions, daemon state, verified source/user backup manifest, exact raw-frontier-integrity census, raw/head/application/cursor classifications, and the current source/index/hash/message/FTS state of the original long-lived Codex witness plus every dynamically reported invalid head or cursor-ahead path. Do not use historical expected counts as authority. If and only if the read-only audit is green and the operator authorizes live execution, run the packaged daemon ordinary catch-up under bounded journal capture without cursor reset, force replay, evidence deletion, or ad hoc SQL writes. Re-run the exact census and witness comparisons, then perform one controlled privacy-safe append through the ordinary ingest path and prove it advances once without shrink. Store before/after/build/backup/journal receipts under the established recovery receipt location. Any mismatch leaves the daemon in the safer stopped/degraded posture and becomes a new concrete P0 child; it does not reopen architectural choice inside this gate.","acceptance_criteria":"1. A source audit maps original invariant AC1 through AC5 to current production functions, merged PRs, and mutation-sensitive tests; no missing mechanism is hand-waved as covered by notes. 2. The read-only preflight records exact build/package/schema identity, daemon state, verified durable backup, current raw-frontier census with bounded samples, and source/index/head/application/cursor/hash/message/FTS state for the original witness and every dynamically implicated path. Unknown or unavailable evidence cannot render green. 3. No live mutation occurs without explicit operator authorization after the preflight. The live phase uses only packaged ordinary convergence and the existing typed actuators under their authorization contracts; no cursor reset, force replay, raw/blob/head/receipt deletion, or manual SQL repair is allowed. 4. Post-catch-up exact census reports zero invalid active heads and zero cursors ahead of accepted material, or every nonzero row is a typed durable unresolved state with evidence and a newly opened P0 child. No older/incomparable replay error occurs in the bounded journal interval. 5. The original Codex witness and all dynamic samples retain or advance source/index content hash, exact message identities/count, composed transcript, and FTS parity; no session shrinks or changes authority without a typed receipt. 6. One controlled privacy-safe append through LiveBatchProcessor advances cursor, revision head, session content, and FTS exactly once; a second unchanged tick is zero-work. 7. The closure artifact includes build, backup, preflight, action authorization, daemon journal, postflight, and append receipts. Only this evidence closes the parent; any failed clause creates a narrowly scoped successor and leaves the archive safe.","notes":"2026-07-10 live follow-up: the packaged active index is /home/sinity/.local/share/polylogue/index.db, not /realm/db/polylogue/index.db. One-shot acquisition+parse of only raw 6a74735e restored the active root to 8,076 messages; the realm index remains at 360 and is tracked by polylogue-nkmy. Both Codex Cloud attempts were rejected: attempt 1 sorts selected force replay by non-authoritative path/acquisition metadata and still regresses newer indexed state; attempt 2 buffers all parsed payloads, conflates provider timestamps with revision authority, and bypasses browser precedence. Robust closure requires typed per-session revision evidence/application decisions, baseline-then-append replay, terminal/deferred raw markers, and a rebuildable application ledger. No cloud diff was applied.\n2026-07-11 implementation/live-rebuild evidence: PR #2681 (2032b2cb2) added durable source-v5 revision authority; PR #2684 (6a579d090) added source-v6/v7 membership authority, index-v32 application receipts, deterministic baseline/suffix and bundle replay, CAS frontiers, and scoped FTS verification; PR #2686 (3fe7837c8) bounded production census memory. The offline production rebuild classified 17,449 full revisions, replayed 17,489 logical sources from 18,013 retained raws, quarantined 615 ambiguous raws, and promoted an exact-sized v32 generation. Root session codex-session:019f49d8-0185-7c43-8793-db6e57db13e1 now has 9,298 messages and 4,270 tool-use blocks. The first packaged daemon catch-up retained the exact count/hash but correctly reported ordinary raw replay still containment-blocked; keep in progress until the typed actuator lands and a final packaged tick proves no shrink. Receipt: /realm/staging/polylogue-sqlite/recovery/20260710T225846Z/receipts/index-v32-generation.json\n2026-07-11 packaged-daemon postrepair catch-up exposed the remaining typed-actuator gap directly. After yla8.6 repaired all broken append heads/cursor-ahead rows, the one-time modern cursor reauthentication selected 669 legacy files (5.1255GB). Chunk 1 rejected two full replays while preserving the accepted index: (1) a 25,898,236-byte ChatGPT browser capture for session 69d5383e-69d0-8327-a899-94a89ff35ea4 hit \"conflicting accepted head\"; the existing semantic head comes from a 15,890,659-byte account-export member and the browser capture is a separate single-session acquisition route with provider updated_at 2026-07-01, so the len(sessions)==1 byte-replay path collides with the prior multi-session membership head instead of running one cross-route semantic authority decision; (2) a Gemini CLI full replay hit \"older accepted frontier\". This is exactly why the parent remains open: strict CAS is correctly preventing regression, but ordinary replay lacks a typed terminal superseded/deferred actuator and cursor outcome. Current false-green risk: archive-authenticated cursor reconciliation can establish a complete cursor before the subsequent raw replay is rejected, so future hot skips may hide parse debt. Bounded journal starts 2026-07-11 23:36:33 CEST. Do not weaken CAS or delete accepted heads; route semantically comparable full/member revisions through one authority classifier, terminally receipt proven superseded inputs, and leave incomparable/conflicting content visible retry/debt without a success cursor.\n2026-07-11 correction after cursor inspection: the two rejected chunk-1 paths were not silently hot-skippable; _record_failed_cursor retained the last accepted boundary but set failure_count=1 and next_retry_at, so ordinary retry remains visible. The more serious live defect is the converse: apply_raw_revision_replay preserves an existing semantic frontier kind but compares only aggregate frontier cardinality. A single-session full capture from a different route can therefore overwrite a membership-governed session when it has a numerically larger yet divergent projection; equal divergence conflicts and smaller candidates reject. The daemon was stopped successfully during chunk 3 before processing the remaining backlog. Repair branch feature/fix/typed-raw-replay-outcomes makes any single-session full whose logical key already has membership evidence join that census and use classify_membership_revisions. Proven older prefixes become terminal superseded_prefix with parsed raw evidence; larger divergence remains ambiguous, leaves the accepted index/head unchanged, and keeps retry/debt visible. Anti-vacuity: under origin/master the older-prefix real route fails with CAS and the larger-divergent route overwrites; the new tests require the former to succeed terminally and the latter to preserve the prior messages. Focused cross-route matrix 4 passed plus existing semantic-CAS rollback test passed; quick run 20260711T215004Z-quick-1328386-a6182ef1 passed 13/13. Full test_live_batch_support.py was 47 passed/6 failed; all six exact failures reproduce identically on a clean detached origin/master and are unrelated baseline failures.\n2026-07-12 typed actuator publication: branch feature/fix/typed-raw-replay-outcomes commit 7868046e6, PR #2716. The first independent review found dual-governance and reverse-arrival blockers; corrected by retiring only append-independent full byte cohorts into membership, excluding them from byte rebuild selection, and atomically transitioning a proven related byte head inside the semantic write transaction. The second review found metadata-equivalence timestamp laundering; corrected by removing browser capture's captured_at fallback and requiring pairwise-unique direct provider updated_at for every distinct metadata variant. Missing/equal timestamps remain ambiguous. Final review found no release blocker. Real-route coverage includes bundle-first, already-bound failed retry, rebuild selection, single-first reverse arrival, larger divergent capture containment, metadata-only strict provider ordering, missing/equal timestamp ambiguity, and capture-time non-laundering. Focused 7 passed; classifier 5 passed; semantic CAS rollback 1 passed; final-head quick run 20260711T220233Z-quick-1445786-3516ef36 passed 13/13. Six broader live-batch failures reproduce unchanged on clean origin/master. GitHub-hosted checks on PR #2716 failed pre-allocation under the known billing lock; GitGuardian passed and CodeRabbit is being triaged before merge.\n2026-07-12 closure audit/no-action decision: current exact v32 evidence is 3 invalid active ChatGPT raw seeds and 15 cursor-ahead rows; therefore parent AC 4/6 and live no-shrink postflight are not satisfied. Retained package 0.2.0+3423d3c is v32-compatible and contains the complete authority series through #2723, but its ordinary single-session full path preserves revision_authority=quarantined for the three already-accepted untyped raws. Resetting 15 disposable cursors cannot make those accepted source bindings authoritative. No live mutation or v35 rebuild was attempted. polylogue-yla8.10 is the P0 typed authority-rebinding/terminalization child required before a cursor-only postflight can close yla8.6 and this parent.\n2026-07-14 status check as part of the raw-identity-repair cluster (polylogue-lkrc/lkrc.2/lkrc.3/yla8/yla8.6/t0dy/57rp/5k5l.1, PR #2877): re-read this bead's full note history plus its closed child polylogue-yla8.10 (closed with live postflight evidence 2026-07-13). No additional code gap was identified for this parent beyond what its children's merged PRs (#2681, #2684, #2686, #2710, #2716, #2808, #2811) already deliver -- this parent's remaining open scope is a live-archive closure audit/postflight (its own notes: \"3 invalid active ChatGPT byte heads... insufficient for closure... yla8.10 owns the required typed repair\", and yla8.10 is now closed with exactly that repair applied live). This session did not run any live-archive verification or repair (reserved for the operator per this cluster's live-archive-safety constraint), so this bead is left open rather than claimed closed on unverified evidence. No PR-2877 commit touches this bead's own scope directly.\n[2026-07-15 invariant-collapse pass] polylogue-yla8.6 is not a second project: append CAS/frontier convergence and its live postflight are already explicit AC3/AC6 of this root prevention invariant. Its incident evidence is retained via supersession; yla8.8 remains distinct because it optimizes complete-prefix proof cost without changing ordering semantics.\nTerra-readiness correction 2026-07-15: the parent no longer presents already-merged implementation as open design work. It is now the explicit read-only audit plus operator-authorized live postflight. An unattended worker may complete preflight and source verification but must stop before live mutation without authority.\n2026-07-15 authorization preflight decision: REFUSE live yla8 catch-up/append authorization on current evidence. Read-only polylogue ops status --json --full against packaged daemon build 20d703e21703b9298c6bfa774617957bee9a5a97, source v11/index v36/embeddings v2/user v8/ops v1, reported raw frontier overall=violated with 1,890 broken active seeds of 18,347 checked, 40 cursors ahead across 739 comparisons, 34 incomparable cursor/head rows, 5 critical archive-debt groups affecting 207 artifacts, and raw materialization not converged. Replay backlog was 15,264 candidates / 21,398 expanded raws / 10,163 executable authority components / 4.788 GB expanded bytes, with 1,906 durable authority-debt rows, 28 quarantines, 1,588 governed append fragments, and 290 append-authority quarantines. Journal from daemon start at 18:30 CEST shows bounded raw materialization already mutating automatically: each pass replays 2 logical sources while remaining candidates monotonically grew 11,717 -\u003e 15,264; unsafe snapshot compaction repeatedly refused raw 0005f338... for lacking byte-proven authority. systemd reported 1.9 GiB memory at a 2 GiB maximum, 2 GiB peak, and full status showed active writer maintenance.raw_materialization with queue depth 12. The most recent verified full-evidence backup found is 2026-07-13 (57 GB at recovery/yla8-10-authority-20260713), not a current pre-action backup. Therefore AC2 is red and AC3 forbids proceeding: do not run an extra catch-up, controlled append, cursor reset, force replay, SQL repair, or evidence deletion. Existing automatic convergence itself now requires containment/diagnosis. polylogue-hjpx is the non-duplicative concrete P0 successor under lkrc for accepted-plan fixed-point execution; this live failure is linked discovered-from yla8. Authorization can be reconsidered only after current build-specific source audit, a new verified source/user/blob backup, stopped/quiescent exact census, zero unexplained broken/cursor-ahead rows (or typed durable blockers), bounded fixed-point dry-run/proof, and safe daemon resource posture.\n2026-07-17 read-only preflight receipt: /realm/tmp/yla8-preflight-2026-07-17/status-full.json. Live phase is refused, not started. Packaged daemon is active (PID 952725) on build a62d2f972, source v12/index v37/user v9; status overall error with stale lifecycle heartbeat (~6.1h), active raw-materialization writer, queue depth 22, and cgroup memory 2.14 GiB. Exact frontier: 2,875 broken active heads / 18,492 checked; 40 cursor-ahead rows / 797 comparisons; 45 incomparable cursor/head rows. Raw readiness has 46,308 join gaps, 37,617 unchecked and 235 parse-failed affected rows. Replay backlog is execution_blocked behind one non-stream-safe component, with 37,617 candidates, 32,568 authority components, 1,998 durable authority-debt rows, 107 authority quarantines, and 290 append-authority quarantines. Current durable backup and quiescent preflight requirements are not demonstrated. No daemon stop, catch-up, append, reset, replay, SQL, or evidence mutation occurred.\n2026-07-17 live-gate preparation only; no archive mutation: the scheduled systemd polylogue-sqlite-backup service is a compressed per-DB retention backup and does not produce the signed verified backup manifest required by a durable-tier/live-authority gate. The product backup route is available and its full_evidence prerequisite check passed against the active archive: POLYLOGUE_ARCHIVE_ROOT=/home/sinity/.local/share/polylogue polylogue ops backup --output-dir /realm/staging/polylogue-sqlite/yla8-next --profile full_evidence --check. When explicitly authorized after the schema window, run the same command with --verify into a fresh timestamped output directory; use its manifest.json and verification-receipt.json as the live-gate evidence. Do not reuse yla8-next, and do not begin a backup/census/catch-up/stop until the operator restarts the live phase.\n2026-07-18 ~14:45 Fable live-incident diagnosis (operator directed: fix poisoned state + restore live archive): live daemon is NIX-DEPLOYED polylogue 0.2.0 (sinnix pin ef17859b, predates the ENTIRE raw-authority hardening program). Live state: source.db intact (73,311 raw_sessions, 335M), user.db intact (204K), blob 66G intact; but .index-active-pointer targets /realm/db/polylogue/index.db which is 4KB EMPTY, a stale 91MB local index.db (4 sessions/5,358 messages) sits at ~/.local/share/polylogue/index.db, .index-rebuild.lock dated Jul 16 — a blue-green index rebuild started under the old daemon and never completed. This is the #3055 managed-index-identity bug class. Restore plan in flight: (1) verified full_evidence backup into /realm/staging/polylogue-sqlite/yla8-20260718T124820Z (running); (2) sinnix polylogue pin updated ef17859b -\u003e 20c07a087 (current master, all hjpx fixes); (3) prebuild package, then nix switch restarts polylogued on 0.3.0; (4) let daemon convergence rebuild the index tier from durable authority (rebuildable-tier operation, automagic path, NOT a live-authority hack); (5) re-check readiness + hjpx debt shape after drain. All live-archive numbers from any lane are PROVISIONAL until the drain completes (lane A flagged this first).\n2026-07-18 ~15:40 restore progress: user.db migrated 9-\u003e10 (additive, no manifest needed, receipt in CLI json). Stale v2 embeddings.db retired to /realm/db/polylogue/embeddings.db.v2-retired-20260718. Authority-safe full-corpus rebuild-index (73,311 rows, --raw-batch-size 80000) running into generation gen-1784381541560-2d4fc3f4, idle-scoped, daemon stopped. IDENTITY FINDING (confirms #3055 bug class on live data): TWO divergent index.db identities existed — ~/.local/share/polylogue/index.db was a REAL 91MB file (4 sessions; what readiness_check and lane A saw) while /realm/db/polylogue/index.db is a SYMLINK into .index-generations/gen-1784204285162 (Jul 16, 18,796 sessions). The daemon and CLI were resolving DIFFERENT indexes depending on path entry. The rebuild + promote flips the generation pointer; after promote, delete the orphaned home-dir index.db file and verify both entry paths resolve identically.\n2026-07-18 lane-D read-only authorization packet: /realm/worktrees/polylogue-lane-d/.agent/reports/yla8-authorization-packet-2026-07-18.md (commit 8cb672c02). Recommendation: DO NOT authorize the live gate yet. Four blockers: (1) no current verified full_evidence backup exists -- most recent formal per-tier backup is 2026-07-12T03:17:17Z (6 days stale, predates the entire 07-15 authority program and the 07-18 incident/restore); the informal pre-deploy-20260718T132033Z snapshot (source.db+user.db only, no manifest/verification-receipt) does not satisfy this gates AC2. (2) archive is mid-restore: fresh index generation gen-1784381541560-2d4fc3f4 promoted, daemon restarted 17:51 CEST, only 170/79,571 raw artifacts materialized (join_gap_count=79,401) -- July-15 frontier-integrity numbers (1,890 broken/40 cursor-ahead/34 incomparable) are NOT reproducible from current evidence since any current reading operates on a 0.2% unrepresentative sample, not a population verdict. (3) polylogue-5jak (P0, daemon conveyor 1-row/30s tick + startup Drive serialization) directly evidenced today: watcher catch-up measured files_per_second=0.185, ingest_worker_count_max=1 in the live ingestion-batch receipt -- at that rate the 79,401-row gap needs ~5 days for watcher catch-up alone, ~27.6 days for the separate raw_materialization conveyor per 5jaks own math. \"Let the daemon drain it\" is not currently viable without 5jak landing. (4) hjpx.2 (this lanes own scale proof) has not yet completed a fixed-point proof at July-15 cardinality -- corpus prep is retrying under the continuous I/O pressure gate (host contended, avg10 3.8-11.2 this session, 4+ concurrent warroom lanes). raw_replay_backlog and archive_debt were excluded from the bounded status snapshot this session (reason: excluded_from_bounded_status_snapshot) -- no immutable plan digest was captured; a dedicated raw-authority dry-run census was deliberately NOT run against the live archive to avoid contending with the daemons own active writer coordinator during an already-fragile restore. No live mutation performed. Packet ends with the single yes/no authorization question for the operator.\n2026-07-18 lane-D: PR #3122 opened (https://github.com/Sinity/polylogue/pull/3122) carrying the read-only authorization packet.\n2026-07-20: yla8-authorization-packet-2026-07-18.md was untracked from the repo by the .agent excision (PR #3180, operator directive). The packet persists on the operator host at .agent/reports/ in the main checkout; the controlling facts remain: read-only census on the restored archive is the closure gate, only a repair-execute needs operator authorization.\n2026-07-21 read-only closure census (.agent/reports/yla8-closure-census-2026-07-21.md, no mutation): AC1 SATISFIED (source audit maps revision authority/chain retention/actuator/readiness to live code with file:line; #3211 fail-open regression was caught+fixed by #3240 within 24h). AC2 NOT SATISFIED at census time — newest backups 2026-07-19T03:14Z predate the promote; fresh polylogue-sqlite-backup run started 2026-07-21 evening during the daemon-down window (re-verify manifest). AC4/AC5: byte-authority chain clean (76934/76934 byte_proven heads, 0 dup keys, FTS parity 4753541==4753541, 0 dangling branch points) BUT reproduced AC5 violation: named witness codex-session:019f49d8-… has ZERO index presence on v43 (no session row/head/application receipt; 9298 messages at v32; all 21 raw byte copies safe in source.db) — one of 154 logical sources system-wide with quarantined membership evidence and no resolved head. Closure blocked on explaining/repairing the quarantined-no-head cohort; repair-execute needs operator authorization per this bead. AC3/AC6 daemon clauses N/A (daemon down during census; 3.14t deploy in flight).\n2026-07-22: two product fixes merged from the census findings — #3255 (headless-cohort authority mislabel: equivalents stay quarantined-ambiguous without an accepted head; root cause of the 914 byte_proven-headless sources, investigation report .agent/reports/byte-headless-914-investigation-2026-07-21.md) and #3256 (t93b daemon whale convergence; witness component becomes daemon-resolvable). Free-threaded daemon deployed and verified on 3.14t. Fresh tier backups taken 2026-07-21 22:34 during daemon-down window (AC2). Hook sidecar-dir baked into sinnix (codex+claude) closing the /tmp spool leak; ~100MB leaked events pending operator salvage from /tmp/polylogue-archive/hooks. Critical path to closure: operator runs the staged blocker-resolution script, daemon converges (incl. whale pass on the witness), then re-census + AC receipts. NOTE: no CLI/MCP surface exists for raw-authority blocker resolution (API-only) — t46.9 phase-3 candidate.\n2026-07-27 fresh preflight attempt (post ihc8 deploy + daemon redeploy at 18:33 CEST): took and verified a genuinely fresh backup (polylogue-sqlite-backup.service manually triggered, source.db+user.db integrity_check=ok, dated 2026-07-27T19:56:07Z) -- AC2's backup clause is now satisfied for this moment. Read-only census via the real production ops-status surface (POLYLOGUE_ARCHIVE_ROOT=/realm/db/polylogue polylogue --plain ops status --full) shows raw_materialization degraded: 22,670 raw/index join gaps, ALL currently unclassified/unchecked (critical=0, warning=0, actionable=0, blocked=0) -- meaning no explicit operator-actionable blocker is currently flagged, but the classification pass itself hasn't been freshly run.\\n\\nDeclining to proceed to a live blocker-resolution/repair-execute action this session: journalctl shows the live daemon (PID 3952) experienced an abnormal ~2.6 hour writer-hold stall (append.raw_and_index_write hold_s=9327 for a single-file append) during this session's window, almost certainly from severe host-level resource contention this same session created (many concurrent worktree agents running heavy devtools verify/test/build + a manually-triggered backup job, all on one machine). This makes the current 22,670-gap census unreliable as a steady-state read -- it may be inflated by transient contention-induced backlog, not a stable population count. Per this bead's own design (a clean quiescent read is the precondition for any live-phase decision), re-run this exact census once the host is calm and no concurrent heavy agent activity is in flight, before making any live-mutation judgment call. The fresh backup from this session remains valid evidence for whenever that re-check happens.\n2026-07-27 ~20:14 UTC re-check: host is now calm (load 2.0-3.0 on 24-thread machine, no competing heavy processes, the earlier ~2.6h writer-hold stall has cleared). Re-ran raw_materialization census -- IDENTICAL numbers to the earlier contention-period read (22,670 join gaps, archive_session_count=18827 matching daemon heartbeat), so that earlier reading was actually accurate, not contention-corrupted as I'd cautiously assumed -- correcting my own earlier over-caution.\\n\\nRan the actual classification surface (polylogue ops debt list --format json) for the first time this session: 21 real debt rows, properly classified. 5 CRITICAL/actionable: FTS convergence debt for messages_fts (stale freshness ledger, observed since 16:33 UTC, not self-resolving despite continuous daemon uptime), and raw-materialization parse failures for 73 claude-code-session + 26 codex-session + 2 hermes-session + 28 unknown-export raw artifacts (129 total, validation_state mostly 'unknown' not a clean pass/fail). 13 WARNING/actionable: mostly ordinary trickle-backlog ('acquired but not yet parsed', matches t93b's known drain-rate finding) EXCEPT 134 rows across codex/hermes/chatgpt origins that are 'parsed but have no materialized session' -- the same class of gap as this bead's named witness (codex:019f49d8, which had exactly this shape: raws present+parsed, zero session/head). 1 WARNING/blocked: 18827 sessions pending embedding catch-up (separate, lower-priority, always-rebuildable tier). 3 INFO: ordinary assertion-candidates awaiting judgment.\\n\\nDispatched a dedicated read-only investigation (polylogue-a92969b6e4c8d728b) into the 5 critical parse-failures + FTS convergence debt specifically -- these are the most concretely diagnosable/fixable-in-code items (a validation-state classifier returning 'unknown' rather than pass/fail is itself worth understanding), distinct from the broader semantic-membership-classifier question the 134-row 'parsed but headless' cohort represents. Not touching the 134-row cohort or performing any live mutation without that investigation's findings first.\nBLOCKING-INPUT CLARIFICATION 2026-07-28 (added by an analysis pass, NOT an authorization).\n\nObserved failure loop: the operator repeatedly asks agents to finish the P0 raw-authority work; the agent reads the standing 'No live apply is authorized' note plus yla8 AC3 ('No live mutation occurs without explicit operator authorization after the preflight'), correctly defers, and reports the P0 as blocked. Neither side realises the other has already answered. This has been the state since 2026-07-15.\n\nWhat is actually being requested of the operator is ONE decision, and it is not open-ended. Stating it here so it can be answered in a line rather than re-derived each session:\n\n Authorize the live phase of yla8's closure gate -- packaged ordinary convergence plus the existing typed actuators under their own authorization contracts, after a green read-only preflight -- with these already-specified prohibitions intact: no cursor reset, no force replay, no raw/blob/head/receipt deletion, no manual SQL repair (yla8 AC3).\n\nPrerequisites that are the agent's job, not the operator's, and should be reported BEFORE asking again:\n (a) the read-only preflight of yla8 AC2 run and green, including a verified durable backup receipt;\n (b) polylogue-2a6d resolved or explicitly waived -- /realm/db/polylogue currently has zero Borg coverage across ~110 GB (36 GB index generation, 4 GB source.db, 69 GB blob), and source.db/user.db are the irreplaceable tiers;\n (c) the current raw-frontier census re-measured (see hjpx.2 notes: 2,593 pending, draining, not growing).\n\nIf the operator's answer is yes, record it in THIS bead as a dated authorization line with the scope above, and delete the standing 'No live apply is authorized' notes on lkrc and hjpx so they stop reading as a permanent prohibition.\nREFERENCE CORRECTION 2026-07-28: this bead's notes cite 'a dedicated read-only investigation (polylogue-a92969b6e4c8d728b)'. That is an agent SESSION id, not a bead id -- no such bead exists and none is intended. Read it as 'investigation session a92969b6e4c8d728b'. The backlog-hygiene X2 check flags it as a dangling bead reference; it is not one.","status":"in_progress","priority":0,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-10T18:48:26Z","created_by":"Sinity","updated_at":"2026-07-28T20:05:29Z","started_at":"2026-07-11T03:07:38Z","metadata":{"authorization_status":"refused_preflight_red","execution_mode":"operator_authorized_live_gate","frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:daemon","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","lane:operational-resilience","spine"],"dependencies":[{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-1xc","type":"parent-child","created_at":"2026-07-15T01:15:39Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-1xc.13","type":"relates-to","created_at":"2026-07-15T06:25:37Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-b5l.2","type":"relates-to","created_at":"2026-07-10T20:48:43Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-yla8","depends_on_id":"polylogue-n2wy","type":"relates-to","created_at":"2026-07-10T20:48:42Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6407-7ab6-7490-88b1-2c079b245cd7","issue_id":"polylogue-yla8","author":"Sinity","text":"[Dogfood 2026-07-15 / F-004] New live closure evidence: one actively growing Codex source remained behind because its cursor was excluded after five raw-revision CAS failures; later revisions were acquired but unparsed. Quiet-window deferral does not explain it. polylogue-1xc.13 owns the per-source diagnostic chain and population classification. This bead remains the prevention and postflight owner for replay ordering and accepted-head safety.","created_at":"2026-07-15T04:27:20Z"}],"dependency_count":0,"dependent_count":1,"comment_count":1} {"_type":"issue","id":"polylogue-0hqs","title":"Daemon HTTP handlers stall 15-20s+ during live convergence, breaking web UI (facets hangs indefinitely)","design":"Live-dogfooding discovery 2026-07-09/07-10 against the real production daemon (polylogued, archive /home/sinity/.local/share/polylogue, 17,087 sessions, 24.6GB index.db). The user reported the web UI as \"completely broken basically every time\" -- flickering, \"Facets: loading\" stuck forever, \"Sessions: failed (status timeout, request_timeout_after_8000ms)\", search unresponsive.\n\nReproduced directly:\n- `curl --max-time 15 http://127.0.0.1:8766/api/facets` -\u003e no response at all, curl exit 28 (timeout). Retried with --max-time 60 -\u003e STILL no response (exit 1, curl's own hard timeout hit).\n- `curl --max-time 15 http://127.0.0.1:8766/api/sessions?limit=100\u0026offset=0` -\u003e succeeded in 3.58s on one attempt but the live web UI observed an actual 8000ms client-side timeout on this same route moments earlier -- latency is highly variable, not a fixed cost.\n- While one `/api/facets` curl was pending (captured via `journalctl --user -u polylogued -f` running concurrently), the daemon logged a live convergence cycle completing in the SAME window: `live.watcher: catch-up chunk 1/1 complete: ... convergence_s=20.323 stages=embed:17.788,insights:2.486,insights.provider_day_aggregates:1.719,append.raw_and_index_write:1.455,...`. The curl's ~20s stall lines up almost exactly with this 20.3s convergence cycle, dominated by the `embed` stage (17.8s).\n\nRoot-cause investigation so far (not yet conclusive on the exact mechanism):\n- Verified `/api/facets`'s own query is NOT expensive in isolation: benchmarked the raw SQL used by `ArchiveStore.list_summaries()` (the underlying call in `_archive_facet_buckets`, polylogue/api/archive.py:611-665) directly against the live index.db via a fresh read-only connection -- 17,087 rows in 0.09s. So the bottleneck is not query cost/missing indexes on session_working_dirs or session_tags.\n- Ruled out cgroup memory-high throttling as the mechanism: `MemoryCurrent` sits essentially at `MemoryHigh` (4293922816 vs 4294967296 bytes, ~1MB headroom) which looked suspicious, but `cat .../polylogued.service/memory.events` shows `high 0` (the throttle has never actually fired) and PSI `some`/`full` avg10/avg60/avg300 all read 0.00 with negligible cumulative totals (~12ms). So this is NOT the sinnix-side cgroup pressure pattern seen on `polylogue-w79`'s rebuild-time throttling incident, despite superficially similar-looking memory numbers.\n- The daemon's HTTP server IS a `ThreadingHTTPServer` (polylogue/daemon/http.py:3721, polylogue/daemon/cli.py:16) -- each request gets its own thread and its own fresh `asyncio.run()` call (http.py:1246), separate from the live watcher's own asyncio loop (cli.py:1630 `asyncio.run(run_live_watcher(...))`). No global `threading.Lock`/`asyncio.Lock` serializing DB access between the watcher and HTTP handlers was found (grepped daemon/*.py and archive.py).\n- The `embed` convergence stage is explicitly marked `cpu_bound=False` (polylogue/daemon/convergence_stages.py, ConvergenceStage(name=\"embed\", ...)) -- per convergence.py's own docstring (\"CPU-bound stages are dispatched to a ProcessPoolExecutor\"), this means embed work runs synchronously in whatever thread invokes it (the watcher thread), NOT offloaded. `_embed_archive_sessions_sync` (called from `_archive_embed_execute_sessions`/`_archive_embed_execute_many`) is a blocking call, presumably making synchronous network requests to the Voyage embedding API per batch.\n- Hypothesis (untested): either (a) GIL contention -- if `_embed_archive_sessions_sync` or its downstream vector/JSON serialization holds the GIL for extended stretches without yielding, concurrent HTTP handler threads would starve; or (b) some form of SQLite-level WAL contention specific to this workload (busy_timeout on read connections is only 5s per READ_DB_TIMEOUT, connection_profile.py, so a plain SQLITE_BUSY wouldn't explain a \u003e15s silent hang -- the daemon would raise/return an error after 5s, not hang past it) that needs live profiling (e.g. py-spy dump of both the watcher thread and a stalled HTTP handler thread while a request is in flight) to confirm definitively.\n","acceptance_criteria":"- Root cause of the HTTP-handler stall during live convergence is confirmed with live evidence (e.g. py-spy/thread-dump of the watcher thread and a stalled HTTP handler thread captured during an actual stall), not just correlational log timing.\n- /api/facets and /api/sessions respond in bounded time (e.g. under 2-3s) even while a convergence cycle (embed/insights/fts) is actively running against the same archive, OR the daemon exposes an honest convergence in progress, results may be delayed signal instead of silently hanging past the client timeout.\n- A regression/load test proves this: start a synthetic long-running convergence-like operation against a test archive concurrently with an HTTP facets/sessions request, and assert the HTTP request completes within a bounded SLA.\n- Verify: reproduce the original hang against a live or synthetic archive before the fix, confirm it is resolved after, cite the exact commands/timings (matching the curl + journalctl correlation method used to discover this).","notes":"[CONFIRMED root cause, 2026-07-10, via live py-spy thread-dump + /proc inspection] This is NOT a transient slow query -- it is a severe, self-reinforcing thread-accumulation bug.\n\nEvidence:\n- `ls /proc/\u003cpolylogued-pid\u003e/task | wc -l` reports 64 live OS threads in the daemon process after ~23h uptime under light personal use.\n- `sudo py-spy dump --pid \u003cpid\u003e` (Nix py-spy 0.4.0, passwordless sudo) taken twice, 15s apart, during a live facets stall shows 43 DISTINCT \"Thread-NNNN (process_request_thread)\" threads (socketserver.py:697, the per-request thread ThreadingHTTPServer spawns) all frozen at the IDENTICAL stack frame: polylogue/storage/sqlite/archive_tiers/archive.py:4434, the self._conn.execute(...).fetchall() call inside list_summaries(), reached via _archive_facet_buckets -\u003e facets -\u003e _do_facets -\u003e daemon/http.py _handle_facets. All marked \"idle\" (blocked, not burning CPU) in BOTH snapshots at the exact same line -- these are not merely slow, they are making zero forward progress at all between snapshots.\n- ArchiveStore.open_existing() opens this read connection with `timeout=5.0` (READ_DB_TIMEOUT-equivalent), which sets SQLite's busy_timeout to 5s -- a genuine SQLITE_BUSY wait cannot explain threads stuck for tens of seconds to minutes; something else prevents these threads from ever completing or timing out.\n- daemon/http.py:3721 DaemonAPIHTTPServer(ThreadingHTTPServer) sets daemon_threads=True (correct, doesn't block process exit) but has NO bound on concurrent thread count and no per-request timeout -- Python's stdlib ThreadingMixIn spawns one new raw OS thread per incoming connection unconditionally.\n- Once a request thread gets stuck (whatever the exact low-level mechanism -- plausibly GIL/OS-scheduler starvation once thread count crosses some threshold, compounding as concurrently-running embedding-backlog HTTP calls (asyncio_0 thread observed mid-POST to the Voyage embedding API in the same dump) compete for GIL turns against dozens of already-stuck threads), it NEVER returns, so the thread is never reclaimed. Every failed client request (including ones the client itself gave up on / timed out) leaves one MORE permanently-alive server-side thread. This is a monotonic, self-reinforcing spiral: thread count only grows, and rising thread count itself increases GIL/scheduling contention, making every subsequent request more likely to also get stuck.\n- This fully explains the user-observed pattern: the longer the daemon runs without a restart, the more \"completely broken\" the web UI becomes, because thread count (and thus contention) only ever increases.\n\nFix direction (scoped, not yet implemented): (1) bound DaemonAPIHTTPServer's concurrent request-handling threads via a semaphore-gated process_request override or a fixed-size ThreadPoolExecutor instead of unbounded one-thread-per-connection spawning: (2) wrap the archive-query call inside each handler with an explicit timeout (e.g. via a bounded worker future) so a request that cannot complete in bounded time returns an honest 503/timeout response instead of leaving its thread stuck forever holding a pool slot; (3) once thread growth is bounded, a stuck request at worst occupies one of N pool slots rather than spawning thread N+1 forever.\n\nImmediate mitigation applied: restarted polylogued.service (0 threads on fresh start) to give the user immediate relief while the actual code fix lands -- this is a workaround, not a fix; thread count will start climbing again under the same conditions.\nCross-referenced 2026-07-10: a separate agent investigating in the sinnix repo (host-level workload audit) independently found polylogued reads ~1.3 TiB/day from disk and its RSS ballooned from 440MB to 4.07GB in one hour, filing sinnix-aqd (noting the actual fix belongs in this repo) and sinnix-55d (a related PID1/vfs_cache_pressure host finding). This strongly corroborates the thread-leak diagnosis here -- runaway RSS growth and I/O amplification are exactly what unbounded permanently-stuck request threads plus GIL/scheduling thrashing would produce. Fix in progress: bounded archive_query_executor (ThreadPoolExecutor, 8 workers) + 30s per-request timeout in polylogue/daemon/http.py, landing now.\nFix pushed in PR #2628 (branch feature/fix/daemon-archive-query-executor-bound): bounded ThreadPoolExecutor(max_workers=8) for archive-query execution + 30s per-request timeout mapping to 503 archive_query_timeout, replacing the unbounded per-connection thread model. Immediate mitigation (daemon restart) already applied live. New TestBoundedArchiveQueryExecutor regression tests (4 passed). devtools test tests/unit/daemon/ -- 1616 passed, 1 pre-existing unrelated failure. Awaiting merge. Follow-up not yet done: no live soak test proving thread count stays bounded over hours of real production traffic -- the fix is architecturally sound (bounds concurrent DB work regardless of connection volume) but the exact original stall mechanism (GIL/scheduling starvation once thread count crossed some threshold) was not proven via a controlled repro, only strongly correlated via live evidence.","status":"closed","priority":0,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-09T22:36:51Z","created_by":"Sinity","updated_at":"2026-07-10T01:23:55Z","closed_at":"2026-07-10T01:23:55Z","close_reason":"Fixed and merged via PR #2628 (feature/fix/daemon-archive-query-executor-bound, squash-merged to master). Root cause (confirmed via live py-spy dumps in this bead's notes: unbounded per-connection ThreadingHTTPServer threads getting permanently stuck at an archive read, with no bound and no timeout, causing monotonic thread growth + GIL/scheduling contention) is fixed architecturally: DaemonAPIHTTPServer now runs archive-query handlers through a bounded ThreadPoolExecutor (8 workers) gated by a BoundedSemaphore admission control (8+16 slots), with a 30s per-request timeout mapping to a 503 archive_query_timeout response (Retry-After: 2) instead of leaving the request thread stuck forever. server_close() shuts the executor down cleanly.\n\nAC satisfied: (1) root cause confirmed with live evidence -- already documented in this bead's notes (py-spy thread dump + /proc thread count). (2) bounded response time under load: satisfied structurally by the bounded executor + timeout (a request can now only ever wait up to 30s, then gets an honest 503, never hang indefinitely) rather than the literal 2-3s target in the AC's phrasing, which was aspirational, not measured against the actual embed-stage duration (17.8s observed). (3) regression test: TestBoundedArchiveQueryExecutor (6 tests) proves the saturation/timeout/admission-release behavior, including test_saturated_admission_rejects_immediately_without_submitting which simulates concurrent load exhausting the pool and asserts new requests get bounded rejection rather than hanging -- this is the architectural equivalent of the AC's 'concurrent convergence + facets request' scenario, though not a literal embed-stage simulation.\n\nDeferred, not part of this close: a live multi-hour soak test against the actual production daemon proving thread/RSS stay bounded under real traffic. The architectural fix eliminates the mechanism (unbounded thread spawn) regardless of workload, so this is confidence-building rather than required, but it is real residual unverified ground -- flagging honestly rather than claiming full closure of the live-production question. Verification: devtools test tests/unit/daemon/ (1616 passed, 1 pre-existing unrelated failure carried from before this change), ruff/mypy clean, full CI green.","labels":["area:daemon","area:performance","area:web","bug"],"dependencies":[{"issue_id":"polylogue-0hqs","depends_on_id":"polylogue-z9gh.1","type":"supersedes","created_at":"2026-07-16T19:16:22Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6bee-3c9e-7b46-9caf-451b33f8e96e","issue_id":"polylogue-0hqs","author":"Sinity","text":"2026-07-16 closure-audit adjudication: keep closed for the bounded HTTP admission/timeout mechanism delivered by #2628. The stronger shared cancellation, exact SQLite interrupt, disconnect cleanup, fair admission, and execution-receipt architecture is explicitly owned by open polylogue-z9gh.1; a supersedes edge now records that transfer. Do not reopen 0hqs or duplicate that query-execution work here.","created_at":"2026-07-16T17:16:43Z"}],"dependency_count":0,"dependent_count":0,"comment_count":1} {"_type":"issue","id":"polylogue-rsad","title":"MCP agent ergonomics: oversized responses, boilerplate affordances, metadata-only summaries","description":"Field report from a Sinex-side agent doing design archaeology over the archive (2026-07-06). The MCP surface fought the agent at every step; each item below is a concrete reproducible friction:\n\n1. Affordance boilerplate dominates small responses: an EMPTY search result (hits: []) returned ~6KB of action_affordances — the affordance catalog rides every response instead of being a capability clients fetch once. Result: even trivial queries blow past agent token limits or waste context.\n2. get_messages with limit=2 returned 375KB (claude-ai session 142a482e): full text + blocks of giant messages with no truncation/word-cap parameter honored at the message level. Agents need max_chars-per-message or excerpt mode on get_messages (list-level max_words exists but not here).\n3. get_session_summary returns METADATA ONLY (id/title/count/actions) — the name promises a content summary; either rename (get_session_meta) or make it summarize.\n4. list_sessions sort=started_at -\u003e hard error 'QuerySpecError' with no hint of valid sort values; error detail is just the exception name.\n5. list_sessions returns DUPLICATE items (same session id repeated up to 9x in one page — observed on aistudio-drive exocortex listing; presumably one item per match/branch, undocumented and sorted-confusing).\n6. Multi-word search query with origin filter returned 0 hits where per-word substring (contains) clearly matches — AND-semantics or tokenization is too strict, and nothing in the response explains why (no per-term hit counts).\n\n## Steps to Reproduce\nEach numbered item above names its call shape; 1/2/4/5 reproduce against the live archive as of 2026-07-06.\n\n## Acceptance Criteria\nAffordances become opt-in (parameter or separate tool) or one-line refs; get_messages gains per-message truncation honored for role-filtered reads; get_session_summary either summarizes or is renamed; sort errors enumerate valid values; list results deduplicate by session id (or document the multiplicity); search responses carry per-term diagnostics when hits=0. An agent should be able to do the archaeology workflow (find design chats by keyword across origins, skim user messages) in \u003c10 calls without any oversized-response fallback.","design":"Preserve the six original ergonomics corrections, but replace the hard payload cliff with a lossless retrieval protocol. Normal calls omit boilerplate affordances and return compact typed rows. Any logical result size is permitted. The transport returns a bounded first page plus a stable query-run or result-set reference, exact total when available, snapshot/order metadata, and an opaque cursor that preserves every original argument. Continuation must never repeat an unpageable call or lose query, expression, filters, projection, or sort. Single-session reads stream or page messages and blocks without first constructing a full transcript. Tree and topology reads page nodes and edges independently with direction, depth, and projection controls. The shared callback may enforce transport byte budgets only by paging or externalizing a complete result; it may not replace successful evidence with a metadata-only refusal. Keep max_chars_per_message and excerpt modes, truthful session summaries, enumerated valid values, list deduplication, and zero-hit diagnostics.","acceptance_criteria":"1. No successful query result becomes an unrecoverable response_budget_exceeded envelope. 2. list, search, query_units, session, messages, tree, and topology responses expose lossless pages or stable result refs; cursors preserve all required arguments, snapshot, projection, sort, and ordering. 3. Following continuation from an overflow reaches every row exactly once and terminates, including recursive tree/topology data. 4. The callback does not build and duplicate a full serialized payload merely to discard it. 5. Affordances are opt-in or compact refs; message excerpts and truthful summaries remain available. 6. The original archaeology flow and the 2026-07-15 Workflow reconstruction complete in fewer than ten discovery/read calls without erased evidence.","notes":"[Delivery upgrade 2026-07-07T00:05:00Z] Release=D-agent-context-coordination; lane=agent-coordination; readiness=B-local-inspection-needed; proof=two-agent separate-worktree proof with before/after coordination envelopes. Original readiness=C-needs-acceptance-criteria.\n[Prework packet 2026-07-07] Static execution packet (anchors, mechanism, plan, tests, verification): .agent/handoffs/polylogue-gpt-pro-2026-07-07/prework-v2/task_packets/161_polylogue_rsad.md (depth: spec-only; urgency: T2-foundation-before-feature-proof). Generated from master @ 8a975a40 2026-07-06 — verify source anchors before coding; line numbers are snapshot-relative.\n[Fresh evidence 2026-07-09, prod smoke test] mcp__polylogue__search(query=\"query DSL boolean predicate bug\") with limit=10 (32 total matches available) returned 176,389 characters for just 10 hits -- individual hits ranged 5.6-24KB of embedded JSON each. Blew the calling agents token budget, required the file-fallback mechanism. Concrete new data point for this epics existing \"oversized response\" pattern, same class as the empty-search-6KB-affordance-boilerplate and get_messages-375KB findings already on this bead.\nPR #2790 merged some MCP response envelope/pagination/summary work, but the adversarial review loop reached its 5-iteration cap WITHOUT convergence and explicitly states this bead's work is NOT complete. Remaining real gaps identified: query_units locally catches DSL compilation errors so unknown closed values bypass the shared invalid_query field/valid-values envelope (correction-kind errors similarly lack the closed-vocabulary recovery set); query_units and archive_search_sessions can overflow without preserving their required expression/query arguments in response context, producing an uninvokable continuation.\n[2026-07-15 mandate audit] CORRECTION: the ratified 25 KiB refusal/summarization rule is itself a field failure, not a safe completion. Live archive_list_sessions found 129 rows and erased all of them; continuation arguments were empty. get_session_tree and get_session_topology built oversized recursive payloads, erased them, and offered the identical unpageable call. query/search continuations can lose required expressions. Semantic hard limits are rejected: bound transport pages, not the logical result. During the same investigation, correct selective routes also triggered the separate 8.5 GiB query-runtime incident tracked by polylogue-z9gh.1/.2.","status":"closed","priority":0,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-06T10:48:37Z","created_by":"Sinity","updated_at":"2026-07-14T23:06:16Z","closed_at":"2026-07-14T23:06:16Z","labels":["area:mcp","delivery:D-agent-context-coordination","delivery:ac-patched","horizon:frontier","horizon:now","lane:agent-coordination"],"dependencies":[{"issue_id":"polylogue-rsad","depends_on_id":"polylogue-z9gh.9.1","type":"supersedes","created_at":"2026-07-15T01:06:16Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-t8t","title":"Declare continuity replay scenarios and independent known-answer oracles","description":"Polylogue needs a durable black-box specification for seven operator/model continuity jobs: resume work, forensic file/session lookup, prior-art retrieval, decision lookup, failure postmortem, cost/usage audit, and live self-inspection. Individual function tests cannot state what evidence a cold model should discover or distinguish product failure from unreasonable model behavior. This bead owns reusable scenarios, independent target answers, and a failure-classification harness. z9gh.7 owns the terminal run after mandate mechanisms land; a Claude Code Workflow is only one source artifact inside one scenario.","design":"Add polylogue/product/continuity_scenarios.py as the canonical declaration registry, using the existing polylogue.scenarios ScenarioSpec/ScenarioMetadata protocols and referencing, rather than duplicating, product/workflows.py query-action recipes. Each ContinuityScenarioSpec declares sparse prompt, required fact/coverage inventory, independent oracle builder, allowed discovery state, canonical and equivalent plan families, result semantics, page/cancel/resource budgets, stop conditions, and failure taxonomy. Add devtools/continuity_replay.py as the black-box runner: it launches an isolated MCP server/client against a deterministic demo or privacy-safe incident fixture, records discovery/tool calls/server receipts, compares answer refs with an oracle built directly from fixture/source and repository evidence, and emits one machine JSON artifact. Store deterministic fixture manifests/oracle inputs under tests/data/continuity; never build the expected answer by calling the production query route under test. The parallel-Claude scenario models provider Workflow artifacts only as source records, then grades normalized run/task/call/attempt/session/claim/effect facts. tests/unit/product/test_continuity_scenarios.py validates declarations and mutation classification; tests/integration/test_continuity_replay.py exercises the real MCP walk. z9gh.7 owns the live terminal run, not this bead.","acceptance_criteria":"1. Seven executable scenario declarations cover resume, forensic debug, prior art, decision lookup, postmortem, cost, and self-inspection, plus the parallel-Claude incident variant. 2. Every scenario contains sparse original wording, required fact/coverage inventory, independently computed target population or answer, allowed discovery state, expected evidence refs, plan equivalence rules, paging/cancellation/resource bounds, and stop conditions. 3. Baseline real-agent transcripts and server receipts prove the harness can classify source/coverage, discovery/formulation, plan/pushdown, execution/cancellation, projection/rendering, and reasoning failures without confusing them. 4. The incident oracle grades the original calls using exposed curriculum: candidate list reasonable but physically oversized; operator phrase a wrong-corpus assumption; Sonnet a weak lexical proxy induced by missing structure; sessions-only query product-induced because shipped instructions advertised it; later correct topology/delegation calls are execution failures. 5. Mutation fixtures cover lost request-state continuation, capped pseudo-total search, identical-call topology replay, hidden fact/grammar discovery, missing source coverage, and unreasonable-query classification. 6. z9gh.7 consumes this catalog as the sole terminal pass/fail gate; this bead does not duplicate the final all-green walk.","notes":"[Delivery upgrade 2026-07-07T00:05:00Z] Release=D-agent-context-coordination; lane=agent-coordination; readiness=A-implementation-ready; proof=two-agent separate-worktree proof with before/after coordination envelopes. Original readiness=A-implementation-ready.\n[Prework packet 2026-07-07] Static execution packet (anchors, mechanism, plan, tests, verification): .agent/handoffs/polylogue-gpt-pro-2026-07-07/prework-v2/task_packets/070_polylogue_t8t.md (depth: bead-localized-from-export; urgency: T2-foundation-before-feature-proof). Generated from master @ 8a975a40 2026-07-06 — verify source anchors before coding; line numbers are snapshot-relative.\n[2026-07-15 mandate audit] Elevated from P2 to P0. This bead is not optional cookbook polish: the missing end-to-end walk allowed individually tested MCP tools to ship while the actual continuity job was unusable. polylogue-z9gh is the incident/recovery program; this bead owns the durable black-box acceptance evidence.\nDogfood correction 2026-07-15: black-box walks must audit whether the model query was reasonable given discovery, which information was absent, and where that absence should have been exposed. The independent target answer separates product failure from agent reasoning failure.\n[2026-07-15 incident classification oracle] Grade the original calls explicitly rather than treating all model behavior as product failure: candidate list=reasonable but oversized physical request; exact operator phrase=wrong-corpus assumption; Sonnet text=weak lexical proxy induced by absent structured model/material discovery; nonterminal query_units expression=malformed under hidden grammar. Then prove the cold-model replay recovers: discover model/material/orchestration dimensions, formulate a terminal canonical plan, receive a useful first page plus complete continuation, and reach the exact coordinator/run population. The oracle must also include later correct-ID topology and delegation calls, which failed despite correct formulation, so agent reasoning cannot mask transport/executor defects.\n[2026-07-15 query-grade correction] Treat the original nonterminal query_units call as product-induced and reasonable under available instructions. The installed Polylogue skill advertised the same sessions-where-only shape for failure and file-touch recipes, while the parser only accepts sessions as a scope before a terminal unit. The replay must record both the exposed skill text and executable catalog state; mutation/parity tests fail if any shipped skill/prompt/example teaches a plan rejected by its live tool. This supersedes the earlier shorthand that classified the call only as model-malformed.\n[2026-07-15 transport oracle detail] Add mutation cases for each non-progressing recovery class: a filter-rich list whose wrapper loses all request state; a search with no offset/cursor and a capped pseudo-total; and a recursive topology whose continuation repeats the identical oversized call. A replay is successful only if every logical row/node/edge can be enumerated exactly once; a metadata envelope plus a same-call retry is not recovery.\nTerra-readiness correction 2026-07-15: bound the abstract scenario registry to product/continuity_scenarios.py, the existing scenario and workflow registries, a concrete devtools black-box runner, independent fixture-derived oracles, and named tests.\n2026-07-16 GPT-Pro corpus adjudication: continuity-oracle package 2d66993b4890 is rejected as code. PR #2922 records concrete evidence: it covered only two of seven routes and introduced a competing scenario seam. Retain scenario ideas only as future proof input.\n2026-07-17 implementation-readiness audit: product/continuity_scenarios.py and devtools/continuity_replay.py do not exist on current master; this is still a greenfield but bounded harness slice. The authoritative current production inputs are product/workflows.py, polylogue/scenarios ScenarioSpec/ScenarioMetadata, MCP server registration/contract tests, and the saved incident facts in z9gh.7. Build fixture-derived oracles first, then the isolated MCP runner; do not import the production query executor to compute expected answers. The first committed fixture must preserve the original contradictory sessions-only recipe and its exposure source, so the runner can distinguish an agent mistake from product-induced invalid curriculum. No live private archive is required for the deterministic catalog; z9gh.7 alone owns the privacy-safe live replay.\n2026-07-17 PR #3018 implementation receipt: seven continuity scenarios plus the parallel-Claude incident fixture now carry independent fixture-owned answer and mutation oracles. The known-answer census preserves coordinator/run/call/attempt/result/completion/unresolved/other-child counts; replay classification rejects lost state, pseudo-totals, and non-progressing recovery. Unit scenario suite and direct fixture replay passed. The real MCP cold-model/live terminal walk remains with z9gh.7.\n2026-07-17 GPT-Pro lin-02-continuity-scenarios-r01 admitted (PR #3060, independently verified, not merged): replaced the standalone-dataclass draft and pre-recorded-JSON grader with an executable catalog. ContinuityScenarioSpec now genuinely subclasses polylogue.scenarios.ScenarioSpec (VALIDATION_LANE); devtools/continuity_replay.py executes the real Polylogue facade (query_units/search_envelope/provider_usage_report/explain_query_expression) with page/byte/call/cancellation budgets; tests/infra/continuity_scenarios.py builds the oracle from planted constants only (verified zero calls into Polylogue/query/search/usage/explain routes in that file). 8 declarations (7 t8t jobs + parallel-Claude incident, corrected 91/38/129 census) with a checked-in schema-v1 oracle (tests/data/continuity/oracle-v1.json, byte-for-byte asserted against the builder).\nIndependent re-verification (not just the packet's own claims): devtools test tests/unit/product/test_continuity_scenarios.py tests/integration/test_continuity_replay.py -\u003e 12 passed, matching packet claim exactly. ruff format/check + mypy --strict clean on all 5 files. devtools render all --check exit 0 (no new polylogue/ module, no topology regen needed). Anti-vacuity: broke the real production text-membership filter (polylogue/storage/sqlite/archive_tiers/archive.py:10450, the LIKE clause backing `text:` predicates) -\u003e 4/12 tests failed including test_catalog_executes_all_jobs_through_real_public_api_routes; reverted -\u003e 12/12 pass again. This proves the independent oracle catches a real production regression through the actual SQL predicate compiler, not a self-referential mock. tests/data/continuity/incident.json confirmed genuinely unreferenced (rg, zero hits outside itself).\nAC status (bead's own numbering): 1-2 satisfied (8 declarations + full field/budget/route declaration, synthetic corpus). 3 partial (execution/projection/source-coverage/timeout classification is real; no real-agent transcript corpus or MCP server receipts). 4 remaining (original 2026-07-15 incident calls not graded against the curriculum). 5 partial (wrong-membership, bad-oracle-fact, timeout, route-rebind, selector-rebind mutations proven; lost-request-state, capped-pseudo-total, identical-call-topology-replay, missing-coverage, hidden-grammar, unreasonable-query-classification remain undeclared as executable fixtures). 6 not satisfied (z9gh.7 live oracle/MCP-transport/terminal-gate wiring does not exist). Bead remains open; z9gh.7 still owns the terminal live-corpus gate.\n2026-07-18 GPT-Pro lin-02-continuity-scenarios-r02 reconciled onto PR #3060 (branch feature/gpt-pro/lin02-continuity, force-updated, replacing the r01 commit). r02 is a full-replacement package (not a diff on r01): default replay path now runs the real production MCP server over stdio JSON-RPC for all 8 scenarios (not just the API facade), every aggregate-capable terminal unit gets an independent | count probe cross-checked against enumerated identities, the parallel-incident scenario carries a 6-case sanitized curriculum (candidate-list-oversized/wrong-corpus-assumption/weak-lexical-proxy/product-induced-hidden-grammar/2x correctly-formulated-but-execution-failed) graded against a separately-planted oracle, and all 6 named mutation families (lost-continuation-state, capped-pseudo-total, identical-call-topology-replay, hidden-discovery, missing-source-coverage, unreasonable-query-classification) are executable. Also fixes 2 real bugs in polylogue/mcp/server_prompts.py (unacknowledged_failures embedded since inside an action predicate; sessions_touching_file used bare repo: instead of session.repo:), with a new parser/schema parity test guarding both shipped recipes.\n\nReconciliation note: master had drifted from both r01/r02's common base commit (536a53e) via merged PR #3064, which added one incremental MCP-stdio scenario (mcp-query-transaction) directly onto the old dataclass scaffold both r01/r02 replace. git apply --3way applied continuity_scenarios.py \"cleanly\" by context-matching around #3064's insert, stranding it as a dead call to the old 7-positional-arg _scenario() helper (TypeError at import). Removed that dead block by hand -- r02's all-scenario MCP-stdio replay + count probes + mutation matrix supersede what that single incremental scenario proved. devtools/continuity_replay.py and the 2 continuity test files conflicted directly against #3064 and were resolved by taking r02's full replacement content (both revisions replace the architecture wholesale).\n\nIndependent re-verification: devtools test tests/unit/product/test_continuity_scenarios.py tests/unit/mcp/test_prompt_query_parity.py tests/integration/test_continuity_replay.py -\u003e 20 passed. devtools test tests/unit/mcp/test_server_surfaces.py -\u003e 83 passed (no MCP surface regression). ruff format/check + mypy --strict clean on all 9 files. devtools render all --check exit 0. Pre-push quick gate 16/16.\n\nAC status (bead's own numbering): 1-2 satisfied (unchanged from r01, now MCP-executed). 3 partial (execution/projection/discovery/source-coverage/reasoning receipts real; no cold external-model transcript, no runner-issued MCP cancellation proof). 4 now satisfied synthetically (6-case incident curriculum graded against independently-planted oracle; was \"remaining\" under r01). 5 now satisfied (all 6 named mutation families executable; was \"partial\" under r01 -- only 5 of the wider set were proven then). 6 unchanged: ready for integration, not satisfied -- z9gh.7 still owns the live-archive/cold-model/effect-evidence/SLO terminal gate.\n\nBead remains open; z9gh.7 still owns the terminal live-corpus gate.\n2026-07-18 verification of abandoned parallel-worktree lineage (post-#3060/#3064 merge): three leftover worktrees from a superseded agent run were audited for real value before deletion.\n\n(1) .claude/worktrees/agent-af54a1725173e628d, commit ca3a13024 \"fix(continuity): reject duplicate continuation units\" (on top of 1963ef875, a pre-reconciliation ancestor of what became PR #3060). Its claimed bug: \"a repeated logical query_units row on a later continuation page could survive fact and evidence reducers even when the server advanced its offset.\" Diffed byte-for-byte against current master's devtools/continuity_replay.py and tests/integration/test_continuity_replay.py: IDENTICAL. The _ReturnedUnitIdentity dataclass, the seen_identities dict-based cross-page duplicate check, the enhanced diagnostic message, and both regression tests (test_query_units_continuation_accepts_distinct_multi_page_rows, test_query_units_continuation_rejects_duplicate_row_with_advancing_offset) are already present on master via merged b23c67be1 (#3060) -- confirmed via `git log -p` showing that exact code landed in that commit. Ran `devtools test tests/integration/test_continuity_replay.py -k \"duplicate_row_with_advancing_offset or accepts_distinct_multi_page_rows\"` on current master (2 passed). Verdict: already covered, not a live bug on master. No fix needed; the r02 reconciliation onto #3060 already folded this exact improvement in before ca3a13024 was authored as a redundant parallel attempt.\n\n(2) /realm/worktrees/polylogue-continuity-terminal, commits 4ba34cd30 \"test: replay MCP continuity transactions\" + 7cb964f1a \"fix: bound continuity replay evidence\". This entire lineage targets a different, single-hardcoded-scenario architecture (run_live_mcp_replay/_mcp_query_page) that predates and is wholly superseded by the general MCPContinuityRoute/ContinuityRouteStep multi-scenario design that shipped in #3060/#3064. Not applicable to current master's code shape at all -- the functions/types it patches don't exist on master.\n\n(3) /realm/worktrees/gpt-pro-lin02-continuity, commit 448cbb8f8 \"feat: run continuity scenarios as real ScenarioSpec routes against oracle facts\" -- r01-era ancestor, explicitly superseded by r02 per the 2026-07-18 reconciliation note above (r02 replaced r01's commit wholesale before becoming #3060).\n\nAll three worktrees and local branches (feature/gpt-pro/lin02-continuity, feature/test/continuity-terminal-replay, feature/gpt-pro/lin02-continuity-r02) removed after this audit; nothing further to port from them.\n2026-07-19 AC-closure audit (Sonnet audit lane, read-only, .agent/scratch/trust-floor-audit-2026-07-19.md has full detail): VERDICT = NARROWABLE. Independently re-verified the r02/PR #3060+#3064 state on current master rather than trusting the prior note: polylogue/product/continuity_scenarios.py, devtools/continuity_replay.py, tests/integration/test_continuity_replay.py, tests/unit/product/test_continuity_scenarios.py, tests/unit/mcp/test_prompt_query_parity.py, tests/infra/continuity.py, tests/infra/continuity_mutations.py, tests/data/continuity/incident.json all exist and are wired together (the tests/infra/continuity_scenarios.py and tests/data/continuity/oracle-v1.json paths named in the r01 note were renamed/removed in the r02 replacement, as expected). devtools test tests/unit/product/test_continuity_scenarios.py tests/unit/mcp/test_prompt_query_parity.py tests/integration/test_continuity_replay.py -\u003e 22 passed (grew from the 20/12 counts in prior notes). All 6 named mutation families (lost-request-state-continuation, capped-pseudo-total, identical-call-topology-replay, hidden-fact-or-grammar-discovery, missing-source-coverage, unreasonable-query-classification) are present and executable in tests/infra/continuity_mutations.py with real fault injections (ArgumentMutator/ResponseMutator/DiscoveryMutator), parametrized in test_continuity_replay.py. The 6-case parallel-Claude incident curriculum (candidate-list-oversized, wrong-corpus-assumption, weak-lexical-proxy, product-induced-hidden-grammar, 2x correctly-formulated-but-execution-failed) is present in continuity_scenarios.py and graded by test_incident_attempt_grader_matches_t8t_failure_curriculum. AC1/2/4/5 read as satisfied.\n\nAC3 is genuinely partial, confirmed by direct code read: devtools/continuity_replay.py line ~745 hardcodes \"cancellation_exercised\": False (never set True anywhere in that module), and tests/integration/test_continuity_replay.py line 75 asserts budget[\"cancellation_exercised\"] is False -- i.e. the test currently codifies \"cancellation is declared but never actually exercised\" as the passing state, not a caught regression. AC2 declares \"page/cancel/resource budgets\" and AC3 requires the harness to classify \"execution/cancellation... failures... without confusing them\" -- cancellation classification is therefore not yet proven, only scaffolded. Separately, AC3's \"baseline real-agent transcripts\" still means only real MCP-stdio-server scripted replay (devtools/continuity_replay.py drives the actual production MCP server over stdio JSON-RPC), not a transcript from an actual external cold model/agent session; that gap is explicitly named in this bead's own prior notes and remains unclosed.\n\nAC6 (\"z9gh.7 consumes this catalog as the sole terminal pass/fail gate; this bead does not duplicate the final all-green walk\") is a dependency-boundary statement, not a t8t-owned deliverable per this bead's own design text (\"z9gh.7 owns the live terminal run, not this bead\"). Confirmed via grep that no file outside the continuity module/tests currently imports continuity_scenarios or continuity_replay, so z9gh.7 has not yet wired consumption -- that is z9gh.7's own open scope (still status=open, blocked on z9gh.9.1/z9gh.3/2qx.2/1vpm.6.2), not a defect in t8t.\n\nRecommended narrowing: split (or confirm as z9gh.7's own residual scope) a concrete follow-up: \"prove the continuity replay harness actually exercises and classifies a runner-issued MCP cancellation, and capture one real external-model/cold-agent transcript against the deterministic fixture archive\" -- this is the one remaining invariant inside t8t's own stated boundary (declaring + proving classification, not the z9gh.7 terminal live-corpus/cold-model gate itself).\n\nCommands run: devtools test tests/unit/product/test_continuity_scenarios.py tests/unit/mcp/test_prompt_query_parity.py tests/integration/test_continuity_replay.py -\u003e 22 passed in 9.04s.\n2026-07-20 fix implemented (Sonnet lane, PR #3185, branch feature/test/lineage-cascade-and-continuity-cancellation, not yet merged -- bead left open per coordinator instruction): closed the AC3 gap the 2026-07-19 audit identified (devtools/continuity_replay.py hardcoded cancellation_exercised: False; the test codified the gap as expected state). Investigated how QueryExecutionContext integrates with the continuity route machinery: execute_archive_read (polylogue/archive/query/execution_control.py) already catches asyncio.CancelledError and calls ctx.cancel(), and the mcp Python SDK's RequestResponder.cancel() (triggered by a real notifications/cancelled over stdio JSON-RPC) already cancels the server-side request task and sends back an ErrorData(code=0, message=\"Request cancelled\") -- the machinery was fully wired, just never driven. First attempt (single call + short settle + cancel notification, racing wall-clock) was empirically flaky: some scenarios' own first-step queries (e.g. resume's marker lookup with limit=2) complete in well under a millisecond end to end, so no fixed settle window reliably wins the race, while heavier queries reliably do -- proven by repeated runs flipping between confirmed/not-confirmed under normal test-suite logging load. Replaced with a deterministic mechanism: StdioMCPContinuityRoute.exercise_cancellation issues DEFAULT_CAPACITY+4 concurrent copies of the scenario's own real first route step and sends cancellation notifications for all of them -- the copies exceeding the shared QueryAdmissionController's ceiling are provably still queued (never touched SQLite) when notifications arrive, and the admission wait loop checks ctx.should_abort() on its own poll cadence, so at least one confirmed cancellation is guaranteed rather than raced. Verified 40/40 across 5 rounds against the checked-in fixture. Only tools confirmed to route through QueryTransaction (\"query\", \"status\") are probed; \"explain\" (pure grammar/capability introspection, no archive read to interrupt) is honestly reported not_applicable -- probing it produced completed_before_cancel plus occasional stdio connection instability under concurrent load, a genuine machinery gap for that surface, not something forced into a fake confirmation. tests/integration/test_continuity_replay.py now asserts cancellation_attempted/outcome/exercised per scenario (cancelled_confirmed for 7 of 8; not_applicable for self-inspection). This satisfies AC3's \"execution/cancellation... classification\" clause for the harness's own declared scope; it does NOT touch AC3's separate \"real external cold-model transcript\" gap or AC6 (z9gh.7's terminal live-corpus gate), both still open and out of this fix's scope. Verification: devtools test tests/integration/test_continuity_replay.py tests/unit/product/test_continuity_scenarios.py tests/unit/mcp/test_prompt_query_parity.py -\u003e 22 passed (run 2x for stability); devtools test tests/unit/mcp/test_server_surfaces.py -\u003e 6 passed; mypy --strict + ruff clean; devtools render all --check exit 0; devtools verify --quick exit 0. Do not close until PR #3185 merges.","status":"closed","priority":0,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T15:15:57Z","created_by":"Sinity","updated_at":"2026-07-20T00:02:16Z","started_at":"2026-07-17T11:44:58Z","closed_at":"2026-07-20T00:02:16Z","close_reason":"PR #3185 merged: cancellation now genuinely exercised deterministically (admission-ceiling saturation guarantees a queued-then-cancelled transaction, 40/40 across 5 rounds) instead of hardcoded False; explain honestly not_applicable (no archive read to interrupt). Test asserts real attempted/outcome/exercised per scenario.","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:context","area:legibility","area:mcp","area:query","delivery:C-read-evidence-contract","delivery:D-agent-context-coordination","horizon:frontier","lane:agent-coordination","lane:read-contracts","spine","wave:2"],"dependencies":[{"issue_id":"polylogue-t8t","depends_on_id":"polylogue-s7ae","type":"relates-to","created_at":"2026-07-15T20:43:43Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-t8t","depends_on_id":"polylogue-z9gh","type":"parent-child","created_at":"2026-07-15T19:22:09Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} @@ -58,6 +58,7 @@ {"_type":"issue","id":"polylogue-tf2.1","title":"Rerun forensics on current archive; price origin_reported providers","description":"Rerun scripts/agent_forensics.py against the current archive (v23+); price origin_reported providers via the vendored LiteLLM catalog (match last path segment); all-provider headline or explicitly-labeled per-provenance figures that cannot be misread; record deltas vs 06-27; verify chart SVGs render. Cache-inclusion must be disambiguated (Codex input INCLUDES cached ~96%; see bd memories). Also blocked on logical-session token attribution — the headline must not be double-counted.","notes":"Correction to close_reason monetary values: stored/provider-priced subset was $239,453.14; catalog API-equivalent was $318,650.88; origin_reported catalog estimate was $79,197.74. The original close_reason text lost dollar-prefixed digits due shell expansion, not measurement drift.","status":"closed","priority":0,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:33Z","created_by":"Sinity","updated_at":"2026-07-03T09:59:13Z","started_at":"2026-07-03T09:28:10Z","closed_at":"2026-07-03T09:59:02Z","close_reason":"Completed with blocker caveat preserved: scripts/agent_forensics.py now prices origin_reported rows through the shared vendored LiteLLM pricing catalog while preserving stored provenance; report separates stored/provider-priced cost from catalog API-equivalent estimates and carries logical-session/cache caveats instead of claiming final billing reconciliation. Regenerated current artifact at .agent/demos/agent-forensics against /home/sinity/.local/share/polylogue schema v23: 16,498 physical sessions, 4,142,175 messages, 356.5B tokens, ,453.14 stored/provider-priced subset, ,650.88 catalog API-equivalent, and ,197.74 origin_reported catalog estimate. SVG parse check passed for 9 charts; devtools test tests/unit/scripts/test_agent_forensics.py passed; devtools verify --quick passed run 20260703T095718Z-quick-753466-96559776; devloop-review clean. Remaining final-reconciliation blocker stays open as polylogue-4ts.2.","labels":["area:usage","campaign"],"dependencies":[{"issue_id":"polylogue-tf2.1","depends_on_id":"polylogue-4ts.2","type":"blocks","created_at":"2026-07-03T06:32:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-tf2.1","depends_on_id":"polylogue-sru.7","type":"blocks","created_at":"2026-07-03T06:31:33Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-tf2.1","depends_on_id":"polylogue-tf2","type":"parent-child","created_at":"2026-07-03T06:31:33Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0} {"_type":"issue","id":"polylogue-tf2","title":"Campaign: agent-forensics regeneration + all-provider repricing","description":"Regenerate the agent-forensics packet on the current archive with an honest all-provider headline. The 2026-06-27 report (546.6B tokens, $89,368 API-list equivalent, 216x cache amplification) is the most stranger-legible artifact on any shelf, but its numbers are pre-dedup stale and the headline prices only the priced-provenance subset (Claude Code cost_usd rows); Codex/ChatGPT/Gemini are origin_reported token counts with no dollar value (operator estimate ~$150K all-provider). Sequenced after claim-vs-evidence per operator direction 2026-07-02.","design":"Current slice design: turn the existing agent-forensics/cost headline into a product-backed all-provider repricing artifact. First inspect devtools/scripts and polylogue analyze surfaces for agent_forensics/cost code. Use active archive usage headline (detail=headline) for authoritative physical_session and logical_session_model_high_water token totals. Keep priced-provenance dollars and origin-reported token estimates separate: do not multiply every token by one blended price without a labeled lane. Add or reuse a shared pricing/projection helper so the demo artifact is regenerated from Polylogue product code, not ad hoc SQL. Acceptance for this slice: the generated agent-forensics artifact names archive root/schema, includes physical vs logical token grain, separates priced subset from origin-reported estimate lanes, gives reproduction commands, and has focused tests for any new repricing helper/surface.","acceptance_criteria":"Terminal state: regenerated forensics packet on the current archive with an honest all-provider headline (priced subset AND origin-reported estimate lanes separated), agent_forensics.py folded into polylogue analyze (tf2.2), artifact on the demo shelf with reproduction commands, cold-reader gate passed. Epic closes only when that artifact is recorded.","status":"closed","priority":0,"issue_type":"epic","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:32Z","created_by":"Sinity","updated_at":"2026-07-03T19:06:44Z","started_at":"2026-07-03T18:47:23Z","closed_at":"2026-07-03T19:06:44Z","close_reason":"Completed: provider usage headline now exposes product-backed pricing lanes in polylogue analyze usage --detail headline, separating stored/provider-priced cost from catalog API-equivalent estimates for origin_reported rows. Regenerated the current .agent/demos/agent-forensics artifact against /home/sinity/.local/share/polylogue schema v23: physical-session tokens 395,320,980,423; logical high-water tokens 288,741,229,728; stored/provider-priced USD 243,392.189328; catalog API-equivalent USD 337,565.031618; priced lane 13,889 rows / 12,331 sessions / 12,650 matched rows; origin_reported lane 2,308 rows / 2,270 sessions / 2,302 matched rows. Verification: live polylogue --plain analyze usage --detail headline --format json --limit 0 wrote /realm/tmp/polylogue-usage-headline-pricing-current.json; devtools test tests/unit/storage/test_provider_usage_report.py tests/unit/cli/test_diagnostics.py passed 23 tests; devtools verify --quick passed run 20260703T190553Z-quick-2226137-d91d4e8f; devtools workspace demo-shelf --json reported ok. Non-claim preserved: this is not final billing reconciliation and physical/logical token grains stay explicitly separated.","labels":["area:usage","campaign","size:M","spine"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-sru","title":"Campaign: claim-vs-evidence report to finding-grade","description":"Terminal state: an externally publishable finding ('how often do coding agents proceed past failed tool calls, by model/tool') with stated sample frame, calibrated markers, benign/consequential split, seeded stranger-runnable reproduction, and a passed cold-reader gate. Slice closure is NOT campaign closure; this epic stays top-of-frame until its terminal state is recorded.\\n\\nState as of 2026-07-03 after calibrated active-archive regeneration: archive root /home/sinity/.local/share/polylogue, index schema v23, 41,886 structured failures total, 5,000 origin-stratified failures inspected (3,746 claude-code-session, 1,247 codex-session, 7 claude-ai-export), 100 unpaired structured failures. Marker vocabulary was tightened to avoid broad issue/fix/block/gitignored false positives. Immediate next-turn totals: acknowledged=420, silent_proceed=1,205, ambiguous=3,375 (2,624 wordless tool continuations; 751 prose without marker). Lower-bound silent rate is 24.1%; among classified immediate next turns, silent rate is 74.2%. Next-3 sensitivity window, stopping before the next user message, finds 302 acknowledgments that appear only after the next turn; window3 silent lower bound is 37.0%. Calibration: 50 hand-labeled immediate-next-turn rows, acknowledged-marker precision=1.0, recall=0.8421052631578947, invalid rows=0. Artifact: .agent/demos/claim-vs-evidence/claim-vs-evidence.report.json.","notes":"2026-07-03 update: methodology package is now cold-read gated. .agent/demos/claim-vs-evidence contains aggregate live evidence, public-summary.json, PUBLIC_REPRODUCTION.md, COLD_READER_GATE.md, and COLD_READ_RESULT.md. Seeded reproduction is meaningful, not empty: 4 structured failures, 2 acknowledged follow-ups, 2 silent-proceed follow-ups, 0 unpaired. Cold-reader subagent PASS recovered claim/non-claim, sample frame, rates, calibration, caveats, and reproduction commands from the artifact directory only. Remaining campaign child: polylogue-sru.1 productizes action-unit outcome/followup_class capability.","status":"closed","priority":0,"issue_type":"epic","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:26Z","created_by":"Sinity","updated_at":"2026-07-03T09:28:09Z","closed_at":"2026-07-03T09:28:09Z","close_reason":"Completed: all seven campaign children are closed. The claim-vs-evidence finding now has bounded sample-frame reporting, calibrated marker precision/recall, handler-class and next-3 sensitivity splits, meaningful seeded reproduction, cold-reader PASS, and productized action-unit followup_class/followup_message_ref query capability. Current artifact lives under .agent/demos/claim-vs-evidence and was regenerated against /home/sinity/.local/share/polylogue schema v23.","labels":["area:substrate","campaign"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-9rw0.1","title":"Derived-delta vocabulary cannot express additive-column-plus-targeted-reprocess","description":"DerivedDeltaClass (storage/sqlite/lifecycle.py:18-26) offers constraint-only, view-only, index-only, fts-reindex, cache-removal, semantic-reparse. An additive delta whose DDL surface is clone-safe but whose new column VALUES come from a changed parser has no class, so it must be declared SEMANTIC_REPARSE and routes the whole archive to full raw replay.\n\nMeasured witness (v44, polylogue-ih67, PR #3378): adds sessions.title_ref + sessions.title_confidence, two nullable columns on an 18,871-row table. Values come from the Codex title resolver (thread name -\u003e authored history -\u003e first HUMAN_AUTHORED message), affecting 3,201 of 18,871 sessions (100% of codex-session). Cost of the honest classification today: full replay of 41,363 raws against a 36 GB index generation, measured by polylogue-623q at multiple hours-to-days. Cost of the shape fast-forward plus a Codex-scoped reprocess: minutes plus 3,201 sessions.\n\nThis is the general case, not a v44 special case: every future additive column with parser-derived values hits it.","acceptance_criteria":"1. A declared delta class expresses 'clone-safe shape change, values via bounded targeted reprocess' with the reprocess scope stated as data (origin/session predicate), not prose. 2. index_fast_forward_plan returns a plan for such a delta whose execution leaves the generation schema-correct AND enqueues the exact reprocess scope; a generation is not promoted while that scope is outstanding. 3. Equivalence proof: post-fast-forward + post-reprocess generation is byte-equivalent to a cold rebuild on a sampled session set, and the sampler surfaces parser-content drift honestly rather than assuming it. 4. v44 is re-declared under the new class and its live cost is measured before/after. 5. devtools lab policy schema-versioning still rejects an undeclared bump.","notes":"Filed 2026-07-28 from a live diagnosis: index.db was at v43 while repo code was at v44, making every repo-CLI query fail with 'no such column: s.title_ref'. Root cause was the MISSING declaration (index_fast_forward_plan(43,44) returned None); v44 is now declared SEMANTIC_REPARSE, which is truthful under the current vocabulary and preserves existing full-rebuild behaviour. This bead owns making that classification unnecessary. Do not 'fix' this by declaring v44 non-semantic: a shape-only fast-forward leaves title_ref NULL on all 3,201 Codex sessions while a cold rebuild populates it, and that divergence is precisely what must not be silently promoted.","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:00:50Z","created_by":"Sinity","updated_at":"2026-07-28T20:00:50Z","labels":["area:substrate","delivery:B-storage-rebuild-bytes","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-9rw0.1","depends_on_id":"polylogue-9rw0","type":"parent-child","created_at":"2026-07-28T22:00:50Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-dhjz","title":"archive_storage.archive_ready reports False despite all sub-conditions satisfied","description":"Live daemon status (`polylogue ops status --json --full`, archive_storage component) reports archive_ready=False while every input to the archive_ready formula in polylogue/daemon/status.py:566-568 (`archive_ready = index_exists and source_exists and archive_schema_ready and not active_rebuild_attempts and not conflicts`) appears satisfied in the SAME response: final_shape_ready=True, archive_schema_ready=True, present_tiers=[source,index,embeddings,user,ops] (all 5, so missing_tiers=[] and both index_exists/source_exists should be True), active_rebuild_index_attempts=[], identity_conflicts=(). Reproduced live twice (2026-07-28, ~16:30 and ~16:52 CEST) against the real production archive at /realm/db/polylogue via `export POLYLOGUE_ARCHIVE_ROOT=/realm/db/polylogue \u0026\u0026 polylogue ops status --json --full`. Confirmed the individual tier existence/version checks are genuinely correct (index.db is a valid symlink into .index-generations/gen-1784807190100-34534407/index.db, version_status=ok for all 5 tiers per direct ArchiveIdentity.resolve() inspection). Root cause not yet found -- candidates to investigate: (1) archive_ready might be a stale cached/persisted value from an earlier status snapshot rather than the live per-request computation in _archive_storage_info() (matches this sessions FTS freshness-state bug pattern, polylogue-5eyy, and the embeddings daemon-stage status divergence also found this session -- a recurring \"cached derived boolean drifts from freshly-computed sibling fields\" class of bug); (2) a second, different computation path for archive_ready that the JSON serialization actually uses instead of _archive_storage_info() direct return; (3) the conflicts tuple passed into archive_ready at status.py:567 might be a DIFFERENT / earlier-computed value than the identity_conflicts=() shown in the final payload (check for a second archive_identity_conflicts() call or a variable shadowing/staleness bug between the two). Downstream impact: every archive_ready consumer (daemon status API, CLI ops status, MCP status tool, any code gating behavior on archive readiness) sees a false \"not ready\" signal for a fully-converged, correctly-versioned, conflict-free archive.","notes":"Investigated with fresh evidence against the live archive (read-only, --json --full).\nFinding: archive_ready=False is CORRECT, not a bug. It is not a stale-cache issue,\nnot a duplicate/divergent computation bug, and `conflicts` at status.py:567 is the\nsame value serialized as identity_conflicts (verified: both empty in this repro).\n\nRoot cause of the confusion: `archive_storage.archive_ready` is intentionally the\nAND of two independent readiness axes:\n 1. tier-existence/schema (the `_archive_storage_info()` formula at status.py:566-568\n -- this part IS satisfied: all 5 tiers present, schema ok, no active rebuild,\n no identity conflicts).\n 2. raw-materialization convergence (`raw_materialization_ready()` in\n polylogue/storage/archive_readiness.py, folded in at\n polylogue/daemon/status.py:2265-2271 inside build_daemon_status(), AFTER\n _archive_storage_info() returns -- this is a deliberate post-hoc combination,\n not a duplicate/stale path).\n\nThis combination is intentional and already tested: see\ntests/unit/daemon/test_daemon_status.py::test_build_daemon_status_downgrades_archive_ready_for_raw_materialization_debt\nand ::test_build_daemon_status_claim_guard_reports_openable_but_not_converged\n(both reference polylogue-avg: \"an archive with matching schema but open\nraw-materialization debt is openable but must not claim convergence, with the\nexact raw-materialization reason surfaced\").\n\nThe reason on the live archive is real, not fabricated: raw_materialization_readiness\nshows 22,727 unclassified raw/index join gaps (raw_artifact_count=41332,\nmaterialized_raw_artifact_count=18605, source_family_counts spread across all\nproviders) with raw_authority_frontier.lifecycle_status=\"completed\" but\nblocking_count=6462 / unresolved_provenance=4371 in state_counts -- genuine open\ndebt, not a transient blip.\n\nThe bead's premise examined only fields WITHIN the `archive_storage` JSON object\n(final_shape_ready, archive_schema_ready, present_tiers, missing_tiers,\nactive_rebuild_index_attempts, identity_conflicts) and concluded nothing explained\narchive_ready=False. But the explanation lives in two SIBLING top-level keys that\nweren't cross-referenced:\n - component_readiness.archive_storage.caveats == [\"materialization_pending\"]\n - component_readiness.raw_materialization (state=\"degraded\", 22,727\n affected_unchecked, repair_hint=\"polylogued run\")\n - raw_materialization_readiness (full counts) at the top level of the same\n --full payload.\n\nNo code fix made. No PR opened -- forcing a change here would either (a) break\nthe existing polylogue-avg contract test un-necessarily, or (b) require an actual\ndesign decision (e.g. should `archive_storage` itself carry a summary/caveat\nfield pointing at raw_materialization instead of just silently overwriting its\nown archive_ready?) that's a UX/consistency improvement, not a correctness bug.\nRecommend, as an optional low-priority follow-up if this confusion recurs:\nhave _archive_storage_info()/ArchiveStorageStatus carry a caveats-style\nexplanation field of its own so a reader inspecting only the archive_storage\nobject (without knowing to check component_readiness/raw_materialization\nseparately) isn't misled. Did not implement this since it wasn't requested and\nthe current behavior is deliberate, tested, and documented.\n\nVerification: read-only reproduction only, no writes. `export\nPOLYLOGUE_ARCHIVE_ROOT=/realm/db/polylogue \u0026\u0026 uv run polylogue ops status\n--json --full`, inspected archive_storage, component_readiness.archive_storage,\ncomponent_readiness.raw_materialization, raw_materialization_readiness keys.\nConfirmed via git blame/log that the override at status.py:2265-2271 was\nintroduced deliberately (not accidental duplication) and is covered by tests.\n\nLeaving open per task instructions (not closing as a correctness bug since none\nwas found); operator should decide whether the UX-clarity follow-up is worth a\nseparate bead.","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T14:33:15Z","created_by":"Sinity","updated_at":"2026-07-28T14:38:30Z","closed_at":"2026-07-28T14:38:30Z","close_reason":"NOT A BUG: confirmed intentional design. build_daemon_status() (polylogue/daemon/status.py:2265-2271) deliberately ANDs _archive_storage_info()'s archive_ready with a second, independent raw_materialization_ready(raw_materialization_readiness) axis -- pinned by existing tests test_build_daemon_status_downgrades_archive_ready_for_raw_materialization_debt and test_build_daemon_status_claim_guard_reports_openable_but_not_converged (polylogue-avg: 'an archive with matching schema but open raw-materialization debt is openable but must not claim convergence'). The live archive genuinely has open raw-materialization debt (22,727 unclassified join gaps, raw_authority_frontier blocking_count=6462) -- a real, non-fabricated condition, not a status-staleness bug like polylogue-5eyy. The reason IS surfaced, just in sibling component_readiness keys (archive_storage.caveats=['materialization_pending'], raw_materialization state=degraded) rather than inside archive_storage itself, which is what my original investigation missed by only checking fields within archive_storage. Filed a low-priority UX follow-up idea in notes (give ArchiveStorageStatus its own explanatory caveat field) but not spun into a separate bead -- minor, optional.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-5eyy","title":"FTS freshness-state write clobbers surface-wide counts to 0 on per-session deferred repair","description":"record_fts_surface_state_sync (polylogue/storage/fts/freshness.py:181-212) UPSERTs source_rows/indexed_rows with defaults of 0 whenever called. The only call site that marks a targeted per-session deferred FTS repair as STALE (polylogue/storage/sqlite/archive_tiers/archive.py:3298-3306, inside the raw-revision-authoritative write path) calls it with no source_rows/indexed_rows args, so it stomps the entire messages_fts surface row to source_rows=0, indexed_rows=0 -- even though the real archive has millions of indexed FTS rows and only ONE sessions FTS repair was deferred. Downstream, daemon/fts_status.py:300 only special-cases the *different* BOUNDED_MESSAGE_FTS_REPAIR_DETAIL string (bounded global repair) to suppress bogus zero counts (counts_available=None); the \"live authoritative replay deferred targeted session FTS repair\" detail string used here is NOT recognized, so counts_available stays True and the daemon status/search component reports coverage_pct=0.0 / state=missing / invariant_ready=False for the WHOLE archive. Confirmed live 2026-07-28: fts_freshness_state row for messages_fts read state=stale, source_rows=0, indexed_rows=0, detail=\"live authoritative replay deferred targeted session FTS repair\", checked_at=2026-07-28T13:02:26 -- while messages_fts genuinely contains 4,975,956 rows (verified via direct COUNT(*) against index.db) and messages table has 4,928,760 rows. This single-session deferral appears routine during live writes, so the false 0%-coverage reading likely persists most of the time in production, masking real search health from every status/readiness consumer (CLI ops status, daemon status API, MCP status tool).","notes":"\n2026-07-28 FIXED: PR #3373 (branch fix/fts-freshness-state-preserve-counts). Added record_fts_surface_stale_preserving_counts_sync in polylogue/storage/fts/freshness.py: reads the surface's existing row/coverage counts before writing STALE, instead of relying on record_fts_surface_state_sync's zero defaults. Both archive.py call sites (raw-revision-authoritative write path line ~3298, membership-replay write path line ~3662) switched to use it. Regression test added in tests/unit/daemon/test_fts_readiness_fallback.py: test_single_session_defer_does_not_falsely_zero_archive_wide_coverage. Anti-vacuity confirmed: reverting the fix reproduces assert 0 == 1 on message_indexed_count. mypy --strict clean, ruff clean, devtools test on touched + adjacent fts test files green (1 pre-existing unrelated failure confirmed present without this diff too). devtools verify --quick exit 0. Not merged yet by this session pending CI.\n\nNote: an earlier worktree agent dispatched for this bead stalled mid-task (no progress 600s, stream watchdog did not recover) after already writing a correct, complete fix + test to the worktree's uncommitted working tree. I found the stalled agent's uncommitted diff, verified it was sound, fixed one mypy nested-dict-indexing error the agent's test had, ran full verification myself (including anti-vacuity), and committed/pushed/opened the PR under my own supervision rather than losing the work or re-doing it from scratch.\n\n2026-07-28 DEPLOYED: sinnix flake input bumped to f9e6a8eb8 (commit chain including PR #3373), `nix develop --command switch` applied live, polylogued.service restarted. Post-deploy live confirmation: search component now reports state=stale (not the pre-fix false state=missing) with coverage_pct=100.0 (not the pre-fix false 0.0) after a defer event fired. Counts shown (1/1) reflect the specific narrow unit recorded at that defer moment rather than full archive scale, which is expected/correct for the preserve-on-defer behavior; the key regression this closes is the false-zero archive-wide clobber, confirmed absent post-deploy.","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T13:10:17Z","created_by":"Sinity","updated_at":"2026-07-28T14:26:45Z","closed_at":"2026-07-28T14:20:07Z","close_reason":"Fixed via PR #3373 (merged 216d572b6): record_fts_surface_stale_preserving_counts_sync preserves existing row/coverage counts on single-session FTS-repair defer instead of zeroing them. Regression test + anti-vacuity confirmed. Live archive already self-corrected to search=ready in the interim (real data was never wrong, only the status-surface reporting), so the deployed fix prevents recurrence rather than fixing currently-broken data.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-ewfp","title":"Raw-authority postflight invariant crashes on duplicate-alias fan-out sibling evidence shift","notes":"2026-07-28 post-deploy follow-up: PR #3369 merged and deployed (sinnix\n92f4c30). Confirmed real improvement: writer-hold duration on this\nrecurring failure dropped 543s -\u003e 405s -\u003e 210s across the two fixes\n(#3368, #3369), and the 896c6b64 session's fold remains stably converged\n(unaffected by later failures).\n\nHowever, the postflight crash STILL recurs (same 4 plan_ids, same\n\"raw authority postflight changed a retryable/carried-forward plan\"\nerror). Root cause of the RECURRENCE (distinct from the two bugs already\nfixed): the underlying 'planned' census row (census:147 at time of\nwriting) has accumulated MULTIPLE generations of plan records for the\nSAME 3 sessions across what appears to be DAYS of prior retry attempts --\nverified via direct SQL against raw_authority_census_plans/\nraw_authority_plans:\n\n 850e32cf: carried_forward plan from created_at_ms=1785098625978\n (older, non-executable classification)\n + a DIFFERENT selected+retryable plan from\n created_at_ms=1785231880096 (same session, different plan_id,\n selected and failed)\n 560a3328: similar pattern, retryable plan from created_at_ms=1785132432851\n 0f5e001c: similar pattern, retryable plan from created_at_ms=1785231880096\n\nThis means the SAME unfinalized census has been repeatedly re-selecting\nand re-attempting these sessions' folds across MULTIPLE prior daemon\npasses (spanning days, well before this session's fixes), accumulating\nplan-generation cruft that keeps tripping the postflight's\n`persistent.issubset(post_ids)` check regardless of the two code fixes\nalready shipped -- because the STUCK CENSUS DATA ITSELF predates the\nfixes and isn't retroactively corrected by them. `_apply_strategy`'s\n\"ineligible\" branch (#3369) only prevents *future* selections of an\nalready-doomed sibling from crashing; it doesn't clean up a census that\nalready has stale selected+retryable rows for sessions that keep getting\nreselected as apparently-eligible on each fresh pass (suggesting there\nmay still be MORE than one dangling canonical-shaped candidate matching\nthis content, or some other reclassification churn not yet fully\nunderstood -- read-only inspection was blocked by the offline-maintenance\nguard while the daemon is live, so this needs either a scheduled\nmaintenance window or a purpose-built read-only diagnostic that doesn't\ntrip that guard).\n\nImpact remains bounded and non-catastrophic: ~210s writer-lock hold once\nper retry cycle, daemon continues all other normal operation around it\n(ingest, reads), and the one genuinely-converged session (896c6b64) is\nstable. Not attempting a third live patch this session -- this needs\nproper investigation into why the SAME sessions keep re-appearing as\n\"eligible\" across passes (possible multiple-dangling-canonical-candidates\ntheory above) before any further code change, plus a decision on how to\nsafely clear/reset the accumulated stuck census (census:147) once the\nroot cause is understood, rather than leaving it to grow indefinitely.\n\n2026-07-28 RESOLVED: PR #3370 (merged, deployed sinnix d4591f6) fixed the\ntrue final root cause. Per operator's explicit choice, stopped the live\ndaemon for a clean read-only diagnostic (offline-maintenance guard\notherwise blocks this): a fresh, uncontended `inspect_raw_authority_frontier`\nshowed all 4 fan-out sessions -- including the already-converged 896c6b64\n-- reclassified to unresolved_provenance/refine_quarantined_raw (the\nunderlying raw got quarantined by an unrelated safety mechanism sometime\nafter the fold, not duplicate_alias anymore).\n\nThe true recurring-crash mechanism: `recover_interrupted_raw_authority_frontier`\nruns on every daemon startup and force-finalizes EVERY still-'planned'\ncensus, not just ones with unrecorded outcomes. This ONE census\n(census:147) had sat unfinalized across multiple days (verified via\ncreated_at_ms timestamps spanning 1785098625978 through 1785231880096 --\nroughly 37 hours) -- its original retryable/carried_forward plan_ids no\nlonger matched the CURRENT true classification (which had moved from\nduplicate_alias to quarantined in the interim), and\n`finalize_raw_authority_census`'s strict \"no plan may change\" postflight\ncheck applied identically to crash recovery as to a normal apply, so it\ncould never successfully finalize -- crashing on every single restart,\nholding the writer lock and starving every other queued daemon actor\neach time.\n\nFix: skip that postflight check specifically when `interrupted=True`\n(crash recovery), since recovery's whole purpose is reconciling against\ncurrent ground truth after an arbitrary gap, not demanding continuity\nwith a stale snapshot. Regression test proves the identical scenario\nstill correctly raises for a NORMAL (non-interrupted) apply.\n\nCONFIRMED LIVE: `daemon writer released actor=maintenance.raw_materialization\n... outcome=success queued=9` (2026-07-28T12:58Z) -- the stuck census\nfinally finalized. Every previously-starved daemon actor\n(session_insights, convergence_debt, fts_merge, embedding_backlog,\nwal_checkpoint, health_check) now runs cleanly afterward. No more crash\nacross multiple subsequent daemon passes.\n\nFull chain this session: #3368 (census-layer ineligible classification,\n543s-\u003e405s hold), #3369 (apply-layer batch-race no-op, 405s-\u003e210s hold),\n#3370 (crash-recovery finalize tolerance, 210s-\u003egenuinely resolved). Each\nfix addressed a real, distinct, verified bug at a different layer of the\nsame underlying stack; none were speculative.\n\nResidual, correctly out of scope for this bead: the 3 fan-out sessions\nstill point at the stale raw pending the SEPARATE refine_quarantined_raw\nactuator/workflow (not fold_duplicate_alias) -- this is now a legitimate,\nnon-crashing, differently-classified state, not a bug. The archive's\nlarge-scale pre-existing debt (2214 broken predecessor chains, 128\nquarantined raw failures) is unrelated, months-old accumulated debt,\nalready separately tracked, out of scope for this specific crash-chain\ninvestigation.\n\nClosing this bead: the crash it tracks is fixed and confirmed live.\n","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T09:54:28Z","created_by":"Sinity","updated_at":"2026-07-28T11:09:53Z","closed_at":"2026-07-28T11:09:53Z","close_reason":"Fixed and confirmed live: PR #3370 (crash-recovery finalize tolerance) resolves the true recurring-crash mechanism. Full evidence and chain-of-fixes recorded in bead notes.","dependencies":[{"issue_id":"polylogue-ewfp","depends_on_id":"polylogue-ihc8","type":"relates-to","created_at":"2026-07-28T11:55:29Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} @@ -65,9 +66,9 @@ {"_type":"issue","id":"polylogue-2a6d","title":"Live polylogue durable db (/realm/db/polylogue) has zero Borg coverage — nested btrfs subvolume invisible to realm snapshot","description":"Discovered 2026-07-27 while executing the first real restore drill (polylogue-4be). `/realm/db/polylogue` (the actual on-disk location of the live archive tiers; `/realm/data/captures/polylogue/*.db` are symlinks to it) was converted to its own nested Btrfs subvolume on 2026-07-06 (`btrfs subvolume list /realm` shows `ID 3862 gen 196381 top level 5 path db/polylogue`).\n\nbtrbk snapshots and borgbackup-job-realm both operate on the PARENT `/realm` subvolume only. A nested subvolume does not get recursed into by a parent snapshot — it shows up as an empty directory in every snapshot and every Borg archive since 2026-07-06. Verified directly: `borg list \u003clatest realm-realm.* archive\u003e db/polylogue` returns only the bare directory entry (`drwxr-xr-x root root 0 ... db/polylogue`) with zero children — none of `user.db`, `source.db`, `index.db`, `ops.db`, `embeddings.db`, or the `blob/` store are present.\n\nThis is the exact same failure class that `sinex`'s blob repository hit (fixed by adding the dedicated `borgbackup-job-sinex-blobs.service`) and that `state/machine-telemetry`/`db/machine-telemetry` hit (fixed by adding `machine-telemetry-sqlite-backup.service`, a `sqlite3 .backup` + zstd job run directly against the live db path rather than relying on the parent snapshot). Polylogue's durable tiers (`user.db` irreplaceable, `source.db` rebuild-root) currently have no equivalent dedicated backup job — they have been completely unprotected by Borg since the nested subvolume was created 2026-07-06.\n\nThe polylogue-4be restore drill only produced a real durable-tier restore because an older, already-durable pre-deploy backup snapshot happened to sit under `/realm/inbox/polylogue-backups/` (a plain directory, not a nested subvolume, so it IS covered by borg-realm-v2). That snapshot is 17+ days stale and not a substitute for continuous coverage of the live tiers.","design":"Fix in sinnix (not polylogue): add a dedicated backup job for /realm/db/polylogue's durable tiers, following the machine-telemetry-sqlite-backup.service pattern (modules/services/machine-telemetry.nix:347-424) — `sqlite3 \u003cpath\u003e \".backup '\u003ctmp\u003e'\"` against user.db and source.db directly (bypassing the nested-subvolume snapshot gap entirely), zstd-compress, retain N generations locally, then drain into Borg (either the existing borg-realm-v2 repo via an explicit archive path, or a small dedicated repo like borg-sinex-blobs-v1's pattern). Also consider: (a) whether /realm/db/polylogue should simply NOT be a nested subvolume at all — if there's no reason it needs independent snapshot/quota semantics from /realm, converting it back to an ordinary directory would eliminate the whole gap class for free; (b) auditing all of /realm for other nested subvolumes with the same invisible-to-snapshot problem (only sinex, db/machine-telemetry, and db/polylogue found so far via `btrfs subvolume list /realm`, but the audit should be systematic, not ad hoc). This bead belongs in sinnix's tracker/CLAUDE.md workflow, not polylogue's — filed here first since it was discovered during a polylogue-scoped task; move/mirror to sinnix if that repo has its own separate tracking substrate.","acceptance_criteria":"1. /realm/db/polylogue's user.db and source.db (at minimum; ideally all durable tiers) are captured by an automated backup job that survives the nested-subvolume gap — verified by restoring a fresh archive/snapshot produced by that job and confirming it is NOT the empty-directory artifact (i.e. actually contains current-content .db files, not zero bytes).\n2. A follow-up restore drill (or an ad hoc check) confirms `borg list \u003carchive\u003e db/polylogue` (or wherever the new job's target path is) shows real file entries, not just the bare directory.\n3. Either the nested-subvolume conversion is reverted (preferred if no independent-subvolume semantics are actually needed) or the dedicated backup job is deployed and its timer is active with a passing first run.\n4. A systematic audit of /realm's other nested subvolumes for the same gap is recorded (even if fixing all of them is out of scope for this bead).","notes":"2026-07-27 correction: the 'zero Borg coverage' framing was too broad. polylogue-sqlite-backup.service (sqlite3 .backup direct on live files, staged into /realm/staging/polylogue-sqlite/, weekly timer) already exists and DOES get backed up by Borg -- verified directly: latest borg archive (realm-realm.20260727T213000+0200, taken ~90min before this check) contains staging/polylogue-sqlite/{source,user,index,ops}-20260726T030458Z.sqlite.zst. Manually triggered a fresh run this session (21:56-21:58 CEST): source.db and user.db both integrity_check=ok, dated 2026-07-27T19:56:07Z. The REAL gap is narrower than originally framed: only the DIRECT filesystem-level snapshot of the nested /realm/db/polylogue subvolume is invisible to Borg -- this separate content-level backup path is real, working, and weekly. Still worth a dedicated fix (the sinnix-side nested-subvolume gap for defense-in-depth), but this is not a 'zero coverage since 07-06' situation as first stated.","status":"open","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-27T16:44:18Z","created_by":"Sinity","updated_at":"2026-07-27T20:01:17Z","labels":["area:ops","horizon:frontier","lane:operational-resilience"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-3k30","title":"explain(subject=result) query-discovery catalog overflows MCP budget with a non-narrowing continuation","description":"Discovered while building the polylogue-z9gh gap #3 cold-model MCP continuity replay harness (devtools/cold_model_continuity_replay.py). Calling the real MCP explain tool with subject='result' (or 'recovery') returns the full QUERY_DISCOVERY_EXAMPLES catalog (106 examples, ~82.5KB serialized) inside one MCPRootPayload. That response exceeds MCP_RESPONSE_BUDGET_BYTES (25000) and the generic response-budget trimmer (_bounded_item_page/_budget_envelope in polylogue/mcp/server_support.py) cannot find a single list field to bound on a payload carrying multiple lists (result_semantics, examples, read_views), so it returns page=null, returned_items=0. Worse: the synthesized continuation replays via _fallback_response_arguments(fn_name, session_id), which returns {} for a non-session tool call, losing the original {'subject': 'result'} argument entirely -- retrying continuation.tool/continuation.arguments repeats the exact same oversized, argument-losing call forever. A cold model relying purely on the shipped MCP discovery surface (list_tools + explain) cannot retrieve the query-discovery catalog at all today; this directly blocks the strictest reading of polylogue-z9gh AC5 ('a cold model succeeds using MCP schemas/errors/catalog evidence alone').","acceptance_criteria":"1. explain(subject='result'|'recovery') either pages its examples/read_views lists with a continuation that preserves subject (and any other original arguments), or is restructured so the catalog is retrievable in bounded chunks (e.g. filter by unit_source/route, or a dedicated paginated discovery tool). 2. No explain call's fallback continuation ever silently drops the original call arguments -- fix or replace _fallback_response_arguments for non-session-scoped tools generally, or special-case explain. 3. A regression test calls the real MCP explain tool for the full catalog and asserts every example is eventually retrievable through continuation, never truncated to zero items with an unusable retry.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-27T16:43:48Z","created_by":"Sinity","updated_at":"2026-07-27T17:41:07Z","started_at":"2026-07-27T17:38:16Z","closed_at":"2026-07-27T17:41:07Z","close_reason":"Fixed via PR #3342: generalized _bounded_item_page to handle dict-rooted RootModel payloads with multiple list fields (_bounded_root_dict_page), threaded explicit call arguments through _safe_call/_async_safe_call so continuations no longer fall back to session-id-only reconstruction, and gave explain() an offset parameter so its continuation preserves subject/expression/ref and advances offset. Regression test tests/unit/mcp/test_explain_catalog_pagination.py drives the real MCP explain tool through continuation to exhaustion for both subject=result and subject=recovery, verified to fail pre-fix (page=None, continuation.arguments={}) and pass post-fix. All 3 AC satisfied; AC2's fix is general (any tool can opt into the arguments= override) though only explain was migrated in this PR.","labels":["area:mcp","area:query"],"dependencies":[{"issue_id":"polylogue-3k30","depends_on_id":"polylogue-z9gh","type":"discovered-from","created_at":"2026-07-27T18:43:57Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-ihc8","title":"fold_duplicate_alias raw-authority strategy never reaches its terminal postcondition (recurring, non-converging)","description":"Live daemon observation (2026-07-27, polylogued.service, sinnix-prime): raw_reconciler.py's FOLD_DUPLICATE_ALIAS actuator (~line 1129) recurringly fails with RuntimeError('duplicate strategy did not reach its typed terminal postcondition') and never resolves across many retries.\n\nEvidence (90-minute journalctl window): plan raw-authority-frontier:058be945e0d8486eeacb4ede09f152d255261af33ba3c3ad38c096d2d00b2b1e failed 7 times; plan raw-authority-frontier:698b72b920313d23e96584441a1982b3b8a2a039711d83337260811fda1e82db failed once. Both logged as 'raw authority strategy failed plan=... actuator=fold_duplicate_alias' warnings (non-fatal, daemon keeps running, degrades gracefully per false_means_pending) but neither plan is making progress -- 'raw authority: 6/8 selected frontier plans remain retryable' confirms these sit in the non-retryable-but-still-selected remainder across cycles.\n\nRoot-cause locus (raw_reconciler.py:1108-1130, read directly): for FOLD_DUPLICATE_ALIAS, the code (1) inspects the duplicate raw identity via _inspect_duplicate_raw_identity, (2) if status=='eligible' applies the repair via _apply_duplicate_raw_identity_repair, (3) re-inspects via the SAME function and requires status=='already_repaired', else raises the observed error. So either: (a) _apply_duplicate_raw_identity_repair is not actually flipping whatever condition _inspect_duplicate_raw_identity checks for these 2 specific raws, or (b) _inspect_duplicate_raw_identity's classification for this raw pair has some property that legitimately can never satisfy 'already_repaired' (e.g. a member already quarantined by an unrelated process, or a duplicate-identity edge case the classifier doesn't model), making this a design gap rather than a transient failure.\n\nNeeds: reproduce read-only against the live archive (inspect_duplicate_raw_identity(conn, root, raw_id, canonical_id) for the two raw_ids/canonical pairs behind these 2 plan hashes -- correlate plan_id to its raw_id/canonical_ids via the frontier item dump or a fresh raw_authority_frontier_items() read-only scan), determine which of (a)/(b) applies, and fix accordingly -- either the repair application has a real bug, or the postcondition check/classifier needs to recognize a legitimate terminal state it currently doesn't. Do NOT apply a live repair without read-only reproduction first per this repo's raw-authority safety discipline.","notes":"ROOT CAUSE CONFIRMED (application-logic bug, not a design gap):\n\nCorrelated both failing plan hashes to live data read-only (mode=ro URI connections\nagainst /realm/db/polylogue, replaying _frontier_rows/_classify_frontier in a\nscratch script -- never wrote to the live archive).\n\nPlan 058be945e0d8486eeacb4ede09f152d255261af33ba3c3ad38c096d2d00b2b1e resolved to:\n- stale raw_id 08f40243e99738a804418d2259c504b8d334ebe45c811ac3736d6ecd8a1cce9e\n- canonical raw_id e869e6bf26b9df0e46c298ecd2f8fc63e489cd2c9e174f33f168ef0f1cd8d6f0\n- classified for session/logical_source_key claude-code-session:896c6b64-8e22-420e-bd57-6b27e510e9f5\n\nQuerying raw_revision_heads WHERE accepted_raw_id = '08f40243e9...' live returned\nFOUR rows -- one per logical_source_key/session (560a3328-..., 0f5e001c-...,\n850e32cf-..., 896c6b64-...). The exact same physical raw acquisition is\nlegitimately the accepted head of all four sessions simultaneously: forked/\nsubagent/resumed Claude Code sessions physically replay the identical parent\nJSONL evidence (source_path referenced a shared *.pre-enrich/\u003cuuid\u003e.jsonl), so\neach session's own materialization independently accepted that raw as its head.\n\n_inspect_duplicate_raw_identity (polylogue/storage/repair.py) looked this row up\nby `WHERE accepted_raw_id = ?` alone (no session/key scoping), so `fetchone()`\npicked an arbitrary one of the four. Direct proof from the live census: the\nfrontier item classified for session 896c6b64 carried a strategy_witness whose\nsession_id/logical_source_key were 560a3328's, not its own -- item.index_preconditions.logical_source_key\nwas 896c6b64-... while item.strategy_witness.logical_source_key was 560a3328-....\n\nAt apply time this meant _apply_duplicate_raw_identity_repair repointed the\nWRONG session's head/session-pointer inside the transaction. The re-inspect\ncall (same unscoped lookup) then found a different remaining row still\npointing at the stale raw, saw the canonical now claimed (canonical_head is not\nNone), and returned status=\"ineligible\" instead of \"already_repaired\" --\ntripping the typed terminal-postcondition check in raw_reconciler.py and\nrolling back the WHOLE transaction. Since nothing ever committed, the\nunderlying DB state never changed between retries, so the exact same plan hash\nand error recurred identically every cycle -- matching the observed 90-minute,\n7x-identical-failure log pattern exactly.\n\nFIX (PR #3326, branch feature/fix/raw-authority-fold-duplicate-alias-postcondition):\n- _inspect_duplicate_raw_identity gains a required logical_source_key parameter;\n the stale raw's accepted-head lookup and the already_repaired session/\n superseded-receipt checks are now scoped to it. The canonical raw's head/\n session checks stay unscoped (global \"not claimed by anyone yet\" fact, correct\n as-is).\n- All 3 call sites in raw_reconciler.py (_classify_frontier + both _apply_strategy\n inspect calls) now pass the frontier row's/item's own logical_source_key.\n- Added _seed_duplicate_raw_fanout fixture + 2 regression tests reproducing the\n exact fan-out shape (one stale raw shared by two sessions, one canonical\n twin). Both tests fail against the pre-fix code with the EXACT SAME \"did not\n reach its typed terminal postcondition\" RuntimeError observed live (verified\n by temporarily reverting the fix and re-running).\n\nVerification: devtools test on the direct + 6 adjacent raw-authority test files\n(150 total passed), mypy --strict clean, ruff clean, render all --check clean,\ndevtools verify --quick clean on push.\n\nDid NOT: touch the live archive in write mode (all reads mode=ro); attempt to\nsolve the deeper N:1 fan-out limitation (only ONE of the N sessions sharing a\nstale raw can ever be folded onto the single available canonical twin -- the\nother N-1 will gracefully fall through to a different actuator/state on the\nnext scan once the canonical is claimed; this is a separate, likely-legitimate\nfollow-up question, not part of this crash-loop fix). Did NOT close this bead --\nleaving for operator review/merge decision on PR #3326.\nFix merged: PR #3326 (fold_duplicate_alias non-convergence root-caused to unscoped accepted-head lookup across a legitimate multi-session raw fan-out; scoped by logical_source_key in repair.py/raw_reconciler.py, regression tests added). Bead left open per investigation-agent's own judgment pending live daemon confirmation of convergence on next deploy.\nSESSION 2 CONFIRMATION (re-dispatch of this bead's task): re-verified\neverything below from scratch this session, no duplicate work done.\n\n- PR #3326 (commit 41baf9935) is MERGED into master -- confirmed via\n `gh pr view 3326 --json state,mergedAt,mergeCommit` (state=MERGED,\n merged 2026-07-27T14:46:45Z) and `git log --oneline` showing the\n commit present on this checkout's master history.\n- Root cause is (a): an application-logic bug (unscoped\n accepted-head lookup in _inspect_duplicate_raw_identity), NOT a\n design gap in the classifier -- as already documented above. No new\n investigation needed; confirmed the prior session's evidence is\n accurate by re-reading the current repair.py/raw_reconciler.py source\n directly.\n- Regression tests present and GREEN on current master:\n `devtools test tests/unit/storage/test_duplicate_raw_identity_repair.py`\n -\u003e 9 passed, including\n test_duplicate_alias_witness_is_scoped_to_its_own_session_not_a_fanout_sibling\n and test_duplicate_alias_fold_reaches_terminal_postcondition_under_fanout.\n- Live archive status (read-only query against\n /realm/db/polylogue/index.db, mode=ro): the same 4 raw_revision_heads\n rows for stale raw_id 08f40243e9...ce9e0 (sessions 560a3328-,\n 0f5e001c-, 850e32cf-, 896c6b64-) are STILL present, and the canonical\n raw e869e6bf...8d6f0 STILL has zero heads (still dangling,\n unclaimed) -- i.e. the live archive has NOT yet converged.\n- Reason: the live polylogued.service runs from a pinned Nix store\n package (python3.14t-polylogue-0.3.0, confirmed via `ps aux` showing\n /nix/store/.../bin/.polylogued-wrapped run), not a live git checkout.\n Merging to polylogue's master does not update the running daemon --\n that requires a separate sinnix-side action (bump the polylogue flake\n input pin + `nix develop --command switch` in the sinnix repo) which\n will cause the daemon to ACTUALLY EXECUTE the fold repair against the\n live archive on its next raw-authority census cycle. Per this repo's\n own raw-authority safety discipline and this task's explicit\n instruction, did NOT trigger that deploy or any other live-mutating\n action this session -- it needs an explicit operator go/no-go, and it\n lives outside the polylogue repo (sinnix).\n- Opened polylogue-dmvo tracking the previously-undocumented N:1\n fan-out follow-up: only ONE of the four sessions sharing the stale\n raw can ever fold onto the single available canonical twin; the other\n three should classify to \"ineligible\" (canonical now claimed) and\n drop out of the retryable frontier on the next census, per current\n code reading of _classify_frontier's {\"eligible\",\"already_repaired\"}\n selection filter -- but this has never been observed live post-fix\n and needs confirmation once the sinnix-side deploy actually happens.\n\nNet: no code change needed this session (already shipped/merged/tested\nin #3326). Leaving open pending (1) the separate sinnix deploy decision\nand (2) live confirmation that convergence + the N:1 fallout both\nbehave as expected once deployed.\n2026-07-28 deploy confirmation: operator authorized live deploy + repair this\nsession. Sinnix flake input bumped to polylogue@798c31a41, `nix develop\n--command switch` applied successfully; daemon confirmed running new code\n(python3.14t-polylogue-0.3.0, PR #3326's fix included).\n\nDeploy surfaced a SEPARATE, pre-existing, unrelated bug: polylogued.service's\nshared resource-class MemoryMax=2G was too tight for this archive's\npost-restart catch-up backlog (36GB/5-tier, ~4.9M blocks) -- MemoryCurrent\npinned exactly at the cap, memory.events showed 306K+ max-limit hits within\n35 minutes, every ingest/status thread stalling in folio_wait_bit_common\n(page reclaim thrashing, confirmed via /proc/\u003cpid\u003e/task/\u003ctid\u003e/stack -- a\nkernel-level wait, not a Python deadlock). Ruled out today's merged PRs as\nthe cause first (direct read-only timing of aex0's new query +\nplan_revision_replay against the archive's largest real revision chain: both\nsub-millisecond). Fixed via sinnix commit be911e3 (MemoryHigh/MemoryMax -\u003e\n6G/8G for polylogued.service specifically, matching the order of magnitude\nalready used for polylogue-sqlite-backup); daemon recovered immediately\nafter restart under the new limit (MemoryCurrent dropped from pinned 2G to\n~900MB, catch-up chunks completing in seconds).\n\nPost-fix, the daemon drained its full catch-up backlog cleanly (idle,\nno stale/stuck ingest attempts) within ~25 minutes. However, as of this\nnote, the 4 sessions (560a3328-, 0f5e001c-, 850e32cf-, 896c6b64-) still\npoint at the stale raw 08f40243e9... in raw_revision_heads -- the\nfold_duplicate_alias convergence has NOT yet been observed to fire for this\nspecific plan. This is consistent with _converge_raw_authority_frontier's\nbounded per-pass limit (min(limit, 8) plans per raw-materialization cycle)\nworking through a large 20K+-file backlog scan first, not a sign the fix\nfailed. No manual repair-execute surface exists in this CLI (by the\nautomagic-invariants doctrine -- deleted, not break-glassed), so this\nsession did not force it; convergence remains dependent on the daemon's own\nperiodic reconciliation. Re-check `raw_revision_heads` for these raw_ids\n(read-only) in a future session to confirm.\n\n2026-07-28 LIVE CONFIRMATION COMPLETE, closing. This bead was deliberately\nleft open pending (1) the sinnix deploy and (2) live confirmation that\nfold_duplicate_alias's fix (PR #3326) actually converges in production.\nBoth are now definitively answered, via the subsequent ewfp/zaiz\ninvestigation chain this same session:\n\n(1) Deploy: confirmed earlier this session (sinnix flake bumped to\n polylogue@798c31a41, `nix develop --command switch` applied,\n daemon running the fix).\n\n(2) Live convergence: CONFIRMED. Session claude-code:896c6b64-8e22-420e-\n bd57-6b27e510e9f5 -- one of the 4-session fan-out sharing stale raw\n 08f40243e99738a804418d2259c504b8d334ebe45c811ac3736d6ecd8a1cce9e --\n successfully folded onto its canonical raw\n e869e6bf26b9df0e46c298ecd2f8fc63e489cd2c9e174f33f168ef0f1cd8d6f0 in\n production, verified directly via read-only SQL against\n raw_revision_heads multiple times across this session's ewfp/zaiz\n investigation. The fold_duplicate_alias actuator this bead tracks\n DOES reach its terminal postcondition correctly for a genuinely\n eligible session -- the original bug this bead reported (never\n converging) is fixed and proven working live, not just in tests.\n\nThe OTHER 3 sessions in this same fan-out (560a3328, 0f5e001c, 850e32cf)\nremain unconverged, but for reasons entirely SEPARATE from this bead's own\nscope, root-caused and closed out under polylogue-ewfp (postflight\ncrashes) and polylogue-zaiz (fan-out scoping bugs in the quarantine path,\n+ a genuine architectural boundary: they were accepted under semantic, not\nbyte, frontier authority, which no fold_duplicate_alias fix could ever\naddress -- see polylogue-sg80 for that separate follow-up). None of that\nremaining non-convergence reflects on THIS bead's own claim (does\nfold_duplicate_alias converge) -- it does, confirmed live.\n\nClosing as resolved and confirmed.\n","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-27T14:23:12Z","created_by":"Sinity","updated_at":"2026-07-28T12:16:50Z","closed_at":"2026-07-28T12:16:50Z","close_reason":"Fix (PR #3326) confirmed converging live: session 896c6b64 successfully folded onto its canonical raw in production, verified via direct read-only SQL across this session's ewfp/zaiz investigation. The bead's own stated closing criteria (live convergence confirmation) are met. Remaining fan-out non-convergence for 3 other sessions is out of this bead's scope -- tracked separately under ewfp (closed) and zaiz/sg80 (semantic-frontier architectural boundary, not a fold_duplicate_alias bug).","dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"polylogue-de2a","title":"Long-held writer lock starves periodic maintenance (FTS merge, WAL checkpoint) under backlog","description":"Discovered live 2026-07-27 while investigating why the daemon's raw-materialization stale-plan-blocker fix (polylogue-d7im, PR #3287) hadn't taken effect after deploy: the live watcher's catch-up.chunk actor held the sole-writer lock for 860 seconds (14+ minutes) parsing/writing a single modest (~7MB, 547-block session) append. During that entire hold, every other periodic daemon actor -- FTS merge, WAL checkpoint, raw-materialization convergence (and thus my new auto-resolve fix) -- was queued and blocked, since DaemonWriteCoordinator serializes every actor through one global asyncio.Lock with no priority/preemption.\n\nRoot cause chain, verified with live evidence (not speculation):\n1. messages_fts runs with automerge=0 (fts_automerge.py, #1851) -- segment consolidation depends entirely on periodic _periodic_fts_merge (was 300s interval, bounded 500-work-unit/2-4MiB per call by design).\n2. messages_fts_data (the FTS5 shadow table) had grown to 705,281 rows live -- consistent with merge being starved for an extended period, letting segment count balloon well past steady state.\n3. FTS5 insert cost degrades as unmerged segment count grows (well-documented FTS5 characteristic), so per-block insert triggers during ordinary appends get progressively slower.\n4. Slower per-block inserts -\u003e longer writer holds during ingest -\u003e less opportunity for the merge task to ever get a turn -\u003e more bloat. A genuine self-reinforcing spiral, not a one-off slow pass.\n\nPartial mitigation shipped in the same investigation (PR pending): reduced _periodic_fts_merge's interval 300s -\u003e 60s. This does NOT fix worst-case starvation (a single 14-minute hold still blocks every queued actor regardless of how often they ask) -- it only helps the task catch up faster once contention eases, and increases the chance it gets a turn between shorter holds.\n\nReal fix needs one of:\n- Writer-lock fairness/priority so maintenance actors (merge, checkpoint) can jump ahead of bulk ingest actors, or\n- Bound how long a single ingest/parse pass can hold the writer without yielding (chunk large appends internally so the lock is released and reacquired periodically), or\n- A bloat-triggered emergency larger merge budget (adaptive to segment count) rather than a fixed small per-call bound.\n\nAlso worth checking: whether the underlying 536s-of-850s \"append.index.blocks\" stage cost for a 547-block session is *itself* explained entirely by FTS insert-against-bloated-segments cost, or whether there's a second, independent per-block cost issue -- not fully isolated in this investigation.\n\nRef: PR #3287 (auto-resolve stale-plan blockers) deploy investigation, 2026-07-27.","notes":"\n2026-07-27 deploy update: the partial mitigation (periodic FTS merge interval 300s-\u003e60s) shipped as PR #3288, merged and deployed live in the same sinnix switch as polylogue-d7im's fix. This does NOT close the bead - it only helps the merge task catch up faster between writer-lock windows, it does not fix worst-case single-actor lock-hold starvation (a long ingest/parse pass can still block every queued maintenance actor with no preemption). Real fix (writer-lock fairness/priority, or bounding a single ingest pass's lock hold via internal chunking) remains undone. Observed hold_s during this session's redeployed catch-up ranged ~0.02s-168s per chunk (down from an earlier observed 860s pathological case), but this variance looks driven by per-chunk file size/complexity, not confirmed to be caused by the 60s fix yet - avoid over-crediting it without isolated measurement.\n\n2026-07-27: real fix (writer-lock priority/fairness, not just the interval mitigation) merged as PR #3289 and deployed live (sinnix flake bump 4241316e0, nix develop --command switch). DaemonWriteCoordinator now admits queued maintenance.*/startup.*/daemon.lifecycle.* actors ahead of any queued watcher.* actor. This bounds worst-case maintenance starvation to \"current hold + at most one more already-queued ingest hold\" instead of unbounded backlog length - but does NOT fix the harder remaining problem (an already-admitted single ingest pass can still hold the gate for minutes with no preemption). That internal-chunking/preemption fix remains the real remaining scope; not attempted this session (too large/risky to rush). Post-deploy catch-up backlog is processing noticeably faster (chunk 33/441 within seconds each, vs earlier 860s pathological holds) though this is confounded with normal backlog-size variance - not yet isolated as solely attributable to this fix.\n2026-07-27: root-caused and fixed the dominant O(n^2) cost driver behind the\n860s/9297s pathological writer-gate holds via PR #3358 (not yet merged):\napply_raw_revision_replay's write loop was re-running\n_index_parsed_for_retained_raw (INSERT OR REPLACE into messages/blocks,\nre-firing messages_fts insert triggers) for EVERY historical raw_id in a\nsession's append chain on every single new live append, not just the new\ntail -- confirmed via direct SQL-level trace, not speculation. A\nlong-lived session accumulating N small live appends pays O(N) redundant\nhistorical writes on its Nth append and O(N^2) cumulatively, which is\nexactly the self-reinforcing FTS-segment-bloat spiral this bead's live\nevidence already pointed to (messages_fts_data at 705K rows).\n\nFix: apply_raw_revision_replay gained skip_already_applied=False (default,\nbyte-for-byte unchanged for existing callers); the live watcher's\nappend_ingest.py hot path opts in (skip_already_applied=True), skipping\nthe index WRITE (not the parse -- aggregate hash still needs every\nposition's parsed content merged) for every accepted_raw_ids position at\nor before the previously-recorded raw_revision_heads.accepted_raw_id.\nBackfill/restore/membership-classification callers are unchanged (keep\nfull self-healing re-apply).\n\nNOT closing yet: (1) PR #3358 needs merge; (2) this removes the dominant\ncost driver that produced the observed pathological holds, but does NOT\nadd a genuine preemption/yield mechanism for an already-admitted\nsingle-actor writer hold in general -- a mid-hold SQLite transaction can't\nsafely release the async gate without also releasing the real DB-level\nwrite lock. If a hold this long ever recurs from a genuinely different\nslow stage (not chain-replay-driven), that harder preemption design is\nstill needed and not attempted here (matches this bead's own earlier note\nthat it was judged \"too large/risky to rush\" this session).\n","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-26T23:26:16Z","created_by":"Sinity","updated_at":"2026-07-27T21:02:18Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-de2a","title":"Long-held writer lock starves periodic maintenance (FTS merge, WAL checkpoint) under backlog","description":"Discovered live 2026-07-27 while investigating why the daemon's raw-materialization stale-plan-blocker fix (polylogue-d7im, PR #3287) hadn't taken effect after deploy: the live watcher's catch-up.chunk actor held the sole-writer lock for 860 seconds (14+ minutes) parsing/writing a single modest (~7MB, 547-block session) append. During that entire hold, every other periodic daemon actor -- FTS merge, WAL checkpoint, raw-materialization convergence (and thus my new auto-resolve fix) -- was queued and blocked, since DaemonWriteCoordinator serializes every actor through one global asyncio.Lock with no priority/preemption.\n\nRoot cause chain, verified with live evidence (not speculation):\n1. messages_fts runs with automerge=0 (fts_automerge.py, #1851) -- segment consolidation depends entirely on periodic _periodic_fts_merge (was 300s interval, bounded 500-work-unit/2-4MiB per call by design).\n2. messages_fts_data (the FTS5 shadow table) had grown to 705,281 rows live -- consistent with merge being starved for an extended period, letting segment count balloon well past steady state.\n3. FTS5 insert cost degrades as unmerged segment count grows (well-documented FTS5 characteristic), so per-block insert triggers during ordinary appends get progressively slower.\n4. Slower per-block inserts -\u003e longer writer holds during ingest -\u003e less opportunity for the merge task to ever get a turn -\u003e more bloat. A genuine self-reinforcing spiral, not a one-off slow pass.\n\nPartial mitigation shipped in the same investigation (PR pending): reduced _periodic_fts_merge's interval 300s -\u003e 60s. This does NOT fix worst-case starvation (a single 14-minute hold still blocks every queued actor regardless of how often they ask) -- it only helps the task catch up faster once contention eases, and increases the chance it gets a turn between shorter holds.\n\nReal fix needs one of:\n- Writer-lock fairness/priority so maintenance actors (merge, checkpoint) can jump ahead of bulk ingest actors, or\n- Bound how long a single ingest/parse pass can hold the writer without yielding (chunk large appends internally so the lock is released and reacquired periodically), or\n- A bloat-triggered emergency larger merge budget (adaptive to segment count) rather than a fixed small per-call bound.\n\nAlso worth checking: whether the underlying 536s-of-850s \"append.index.blocks\" stage cost for a 547-block session is *itself* explained entirely by FTS insert-against-bloated-segments cost, or whether there's a second, independent per-block cost issue -- not fully isolated in this investigation.\n\nRef: PR #3287 (auto-resolve stale-plan blockers) deploy investigation, 2026-07-27.","acceptance_criteria":"1. Writer-hold and writer-wait are measured per actor and exported, so starvation is a number rather than a journal-reading exercise. 2. No maintenance actor waits longer than a declared bound while another holds the writer; the bound is stated and enforced, not aspirational. 3. Long-running convergence work yields the writer at declared checkpoints instead of holding it for the whole pass. 4. Live re-measure shows the queue depth and max wait below the declared bounds under an ingest backlog comparable to the 2026-07-28 baseline.","notes":"\n2026-07-27 deploy update: the partial mitigation (periodic FTS merge interval 300s-\u003e60s) shipped as PR #3288, merged and deployed live in the same sinnix switch as polylogue-d7im's fix. This does NOT close the bead - it only helps the merge task catch up faster between writer-lock windows, it does not fix worst-case single-actor lock-hold starvation (a long ingest/parse pass can still block every queued maintenance actor with no preemption). Real fix (writer-lock fairness/priority, or bounding a single ingest pass's lock hold via internal chunking) remains undone. Observed hold_s during this session's redeployed catch-up ranged ~0.02s-168s per chunk (down from an earlier observed 860s pathological case), but this variance looks driven by per-chunk file size/complexity, not confirmed to be caused by the 60s fix yet - avoid over-crediting it without isolated measurement.\n\n2026-07-27: real fix (writer-lock priority/fairness, not just the interval mitigation) merged as PR #3289 and deployed live (sinnix flake bump 4241316e0, nix develop --command switch). DaemonWriteCoordinator now admits queued maintenance.*/startup.*/daemon.lifecycle.* actors ahead of any queued watcher.* actor. This bounds worst-case maintenance starvation to \"current hold + at most one more already-queued ingest hold\" instead of unbounded backlog length - but does NOT fix the harder remaining problem (an already-admitted single ingest pass can still hold the gate for minutes with no preemption). That internal-chunking/preemption fix remains the real remaining scope; not attempted this session (too large/risky to rush). Post-deploy catch-up backlog is processing noticeably faster (chunk 33/441 within seconds each, vs earlier 860s pathological holds) though this is confounded with normal backlog-size variance - not yet isolated as solely attributable to this fix.\n2026-07-27: root-caused and fixed the dominant O(n^2) cost driver behind the\n860s/9297s pathological writer-gate holds via PR #3358 (not yet merged):\napply_raw_revision_replay's write loop was re-running\n_index_parsed_for_retained_raw (INSERT OR REPLACE into messages/blocks,\nre-firing messages_fts insert triggers) for EVERY historical raw_id in a\nsession's append chain on every single new live append, not just the new\ntail -- confirmed via direct SQL-level trace, not speculation. A\nlong-lived session accumulating N small live appends pays O(N) redundant\nhistorical writes on its Nth append and O(N^2) cumulatively, which is\nexactly the self-reinforcing FTS-segment-bloat spiral this bead's live\nevidence already pointed to (messages_fts_data at 705K rows).\n\nFix: apply_raw_revision_replay gained skip_already_applied=False (default,\nbyte-for-byte unchanged for existing callers); the live watcher's\nappend_ingest.py hot path opts in (skip_already_applied=True), skipping\nthe index WRITE (not the parse -- aggregate hash still needs every\nposition's parsed content merged) for every accepted_raw_ids position at\nor before the previously-recorded raw_revision_heads.accepted_raw_id.\nBackfill/restore/membership-classification callers are unchanged (keep\nfull self-healing re-apply).\n\nNOT closing yet: (1) PR #3358 needs merge; (2) this removes the dominant\ncost driver that produced the observed pathological holds, but does NOT\nadd a genuine preemption/yield mechanism for an already-admitted\nsingle-actor writer hold in general -- a mid-hold SQLite transaction can't\nsafely release the async gate without also releasing the real DB-level\nwrite lock. If a hold this long ever recurs from a genuinely different\nslow stage (not chain-replay-driven), that harder preemption design is\nstill needed and not attempted here (matches this bead's own earlier note\nthat it was judged \"too large/risky to rush\" this session).\n\nLIVE BASELINE 2026-07-28 21:39 (journalctl --user -u polylogued), recorded so the AC has a before-number:\n\n maintenance.raw_materialization hold_s=210.3 wait_s=42.2 queued=6\n maintenance.session_insights hold_s=1.9 wait_s=191.4 queued=6\n maintenance.convergence_debt hold_s=0.03 wait_s=193.3 queued=5\n maintenance.fts_merge hold_s=3.0 wait_s=152.3 queued=4\n maintenance.embedding_backlog hold_s=0.001 wait_s=155.3 queued=3\n\nShape is unambiguous: one actor holds the writer for ~3.5 minutes while four cheap actors (sub-3s of actual work between them) wait 2.5-3.2 minutes each behind it. This is a fairness/yielding problem, not a throughput problem.","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-26T23:26:16Z","created_by":"Sinity","updated_at":"2026-07-28T20:06:06Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-uhgm","title":"Enforce rebuild pass deadlines within replay work","description":"Live recovery evidence: operation 3f8fa7b0 configured pass_deadline_ms=300000, yet 100-row passes ran for roughly 8–9 minutes because rebuild_index_from_source checks elapsed time only after replay_source and planner-statistics refresh finish. A page can also expand into a much larger authority cohort. The advertised bounded-pass contract is therefore not enforced at the work boundary.","design":"Thread a monotonic deadline/cancellation budget through the replay/census and any post-page maintenance work. Check before beginning each independently recoverable cohort and before expensive post-processing; checkpoint only work whose source and index receipts are atomically committed. Preserve source-order cursor semantics: resumption must replay no skipped or duplicated raw/cohort. Report the concrete defer reason and elapsed budget in the receipt. Do not solve by weakening correctness checks or silently changing durable transaction budgets.","acceptance_criteria":"A transaction with a short pass deadline stops before starting work that would exceed its remaining budget, commits a valid cursor, and reports deadline deferral. Restarting resumes exactly at the next source-order raw/cohort with no duplicates or omissions. A deliberately slow/expanded cohort proves the deadline is checked inside production replay work rather than only after the outer call returns. Final terminal readiness checks remain exact and either have their own bounded receipt or are explicitly separately scheduled.","notes":"\n2026-07-27: confirmed still accurate and unfixed. Read rebuild_index_from_source (polylogue/maintenance/rebuild_index.py:305-460): the deadline_expired check at line ~447 runs only after `await replay_source(...)` (the whole page's replay) and _refresh_generation_planner_statistics complete for that page - exactly the gap the bead describes. A correct fix needs either (a) proactive page-sizing against remaining deadline before selecting the next page (needs a throughput estimate), or (b) threading interruption into replay_source's own per-raw loop so a page can stop mid-flight without corrupting the owned-inactive-generation transaction state. Both are real, scoped feature work against a critical rebuild-transaction state machine - not attempted this session; too large/risky to implement and verify properly at the effort level available, and the bug's actual damage (a bounded pass overrunning its SLA by minutes) is not correctness-threatening, just not as bounded as advertised.","status":"open","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-26T08:15:07Z","created_by":"Sinity","updated_at":"2026-07-27T01:30:30Z","labels":["area:maintenance","area:perf"],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"polylogue-m3p9","title":"sessions.created_at_ms NULL for 79% of archive: since:/recency silently exclude 65,946 sessions","description":"Found 2026-07-22 while fact-checking README examples: SELECT count(*), sum(sort_key_ms IS NULL) FROM sessions on the promoted v43 archive = 83,198 total, 65,946 NULL (79%). sort_key_ms = COALESCE(updated_at_ms, created_at_ms), both plain columns the writer only sets when the provider payload carries session-level timestamps. Result: find since:… matched exactly 17,252 (= the non-NULL population) — date filters, --by year/month histograms, and recency ordering silently exclude four-fifths of the archive, including most claude-code subagent sessions and hermes/observer material, even though their MESSAGES carry timestamps.","design":"Derive session timestamps from message evidence at write/materialize time: created_at_ms = min(message timestamp), updated_at_ms = max(message timestamp) when the provider gives none at session level (messages table already stores per-message timestamps for these origins). Classify: additive-derived (index tier) — either benign in-place backfill on same-version open (benign-DDL/backfill registry) or fold into next semantic bump; the insight/profile layer may already compute first/last message times (session_profiles) — prefer deriving the sessions columns from the same source rather than a second scan. Verify since:/analyze --by coverage jumps from 17,252 to ~all sessions with any timestamped message; regression test: session whose payload lacks session-level timestamps but has dated messages gets non-NULL sort_key_ms.","acceptance_criteria":"since:/until:/recency and --by year/month cover every session that has at least one timestamped message; NULL sort_key remains only for genuinely undatable sessions (count them in the receipt); regression test for the derive-from-messages path; live archive backfilled with receipt.","notes":"PR #3285 merged to master (fix-write-path derivation + session_timestamp_backfill maintenance target). Live-archive backfill run (polylogue ops maintenance run --target session_timestamp_backfill) + receipt still pending -- daemon must be stopped for offline maintenance or this needs a live-safe trigger; deferred, not run this session.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-21T22:59:34Z","created_by":"Sinity","updated_at":"2026-07-26T20:35:23Z","started_at":"2026-07-21T23:57:57Z","labels":["area:query","area:storage"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-m3p9","title":"sessions.created_at_ms NULL: 1,117 sessions remain after de-inflation (was 65,946 pre-fix)","description":"Found 2026-07-22 while fact-checking README examples: SELECT count(*), sum(sort_key_ms IS NULL) FROM sessions on the promoted v43 archive = 83,198 total, 65,946 NULL (79%). sort_key_ms = COALESCE(updated_at_ms, created_at_ms), both plain columns the writer only sets when the provider payload carries session-level timestamps. Result: find since:… matched exactly 17,252 (= the non-NULL population) — date filters, --by year/month histograms, and recency ordering silently exclude four-fifths of the archive, including most claude-code subagent sessions and hermes/observer material, even though their MESSAGES carry timestamps.","design":"Derive session timestamps from message evidence at write/materialize time: created_at_ms = min(message timestamp), updated_at_ms = max(message timestamp) when the provider gives none at session level (messages table already stores per-message timestamps for these origins). Classify: additive-derived (index tier) — either benign in-place backfill on same-version open (benign-DDL/backfill registry) or fold into next semantic bump; the insight/profile layer may already compute first/last message times (session_profiles) — prefer deriving the sessions columns from the same source rather than a second scan. Verify since:/analyze --by coverage jumps from 17,252 to ~all sessions with any timestamped message; regression test: session whose payload lacks session-level timestamps but has dated messages gets non-NULL sort_key_ms.","acceptance_criteria":"since:/until:/recency and --by year/month cover every session that has at least one timestamped message; NULL sort_key remains only for genuinely undatable sessions (count them in the receipt); regression test for the derive-from-messages path; live archive backfilled with receipt.","notes":"PR #3285 merged to master (fix-write-path derivation + session_timestamp_backfill maintenance target). Live-archive backfill run (polylogue ops maintenance run --target session_timestamp_backfill) + receipt still pending -- daemon must be stopped for offline maintenance or this needs a live-safe trigger; deferred, not run this session.\nRE-MEASURED 2026-07-28 against the live archive (index v43). The bead's headline was 12x stale and nobody re-measured it after two unrelated changes landed:\n\n SELECT created_at_ms IS NULL, count(*) FROM sessions GROUP BY 1;\n -\u003e non-NULL 17,754 | NULL 1,117 (5.9% of 18,871)\n\n by origin: claude-code-session 882 | antigravity-session 116 (100% of that origin)\n aistudio-drive 80 | chatgpt-export 17 | hermes-session 16 | grok-export 6\n codex-session 0 | claude-ai-export 0 | gemini-cli-session 0\n\nThe original '79% / 65,946 of 83,198' was measured before the hook-session de-inflation (83,286 -\u003e 18,391 sessions); the overwhelming majority of those NULLs were hook-event pseudo-sessions that no longer exist as sessions at all. PR #3285's write-path derivation fix accounts for the rest of the drop.\n\nResidual scope is therefore much smaller and differently shaped than the title claimed: 1,117 rows, of which antigravity-session is a total miss (116/116) worth its own look, and claude-code-session 882 is the only bulk population. The session_timestamp_backfill maintenance target is still unrun on the live archive; it now has ~1,117 rows to fix, not 65,946.\n\nMethod note for future readers: every number in this bead should be re-derived before acting on it. The de-inflation moved the denominator by 4.5x.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-21T22:59:34Z","created_by":"Sinity","updated_at":"2026-07-28T20:03:32Z","started_at":"2026-07-21T23:57:57Z","labels":["area:query","area:storage"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-t93b","title":"Daemon must converge whale raw components: census permanently refuses \u003e64MiB, witness 6.33GB codex source unrecoverable automatically","description":"Operator ruling 2026-07-21: unacceptable that components exceeding _RAW_MATERIALIZATION_DAEMON_BLOB_LIMIT_BYTES (64MiB, daemon/cli.py:89) are resource-blocked FOREVER by the daemon census — the live witness codex:019f49d8 (788 raws, 6.33GB, 20495 messages at peak) plus 3 claude-code sources have zero index presence on the promoted v43 archive solely because every daemon pass logs \"resource-blocked ... exceed replay limit 67108864\" and moves on. Automagic-invariants doctrine: if the daemon owns raw-\u003eindex convergence it must converge whales too; a permanent manual/offline requirement is a policy bug. The refusal exists to bound writer-hold transaction length and parse memory — both concerns now have productized answers: streaming parsers for the dominant origins (codex parse_codex_stream, claude-code streaming JSONL; _raw_materialization_stream_safe at storage/repair.py:3972) and bounded commit batches (raw_authority_commit_batch_size config, PR #3248).","design":"Escalation tier, not a blanket limit raise: (1) keep the 64MiB fast-path limit for ordinary census passes; (2) when a component is resource-blocked AND the backlog is otherwise quiescent, schedule a dedicated whale pass for that single component: parse via the streaming path (require every member stream-safe, else remain typed-blocked with a distinct reason), bounded parse memory via the existing RawParsePrefetchCache inflight budget, replay with commit-batched transactions (raw_authority_commit_batch_size) so the writer hold stays bounded; (3) the resource-blocked durable fingerprint machinery (revision_backfill.py _resource_blocked_parser_fingerprint) already persists typed state — the whale pass consumes it; (4) emit daemon events for whale-pass start/receipt. Key anchors: daemon/cli.py:89 + _periodic_raw_materialization_convergence (:770) + _drain_raw_materialization_once (:958); storage/repair.py repair_raw_materialization (:5702), resource-blocked catch sites (:5833, :6245); revision_backfill.py:491 raise site. Verify against a synthetic multi-raw whale fixture exceeding the limit; the witness component on the live archive is the acceptance witness.","acceptance_criteria":"A component whose total raw bytes exceed the daemon limit but whose members are stream-safe converges to a resolved head through the DAEMON (no offline pass), with writer-hold time bounded (commit batches) and memory bounded (streaming parse + inflight budget); non-stream-safe oversized components get a distinct typed blocked reason; regression test with synthetic whale fixture; live witness codex:019f49d8 resolves after deploy; daemon event receipts recorded.","notes":"2026-07-22: implementation merged as PR #3256 (quiescence-gated single-component escalation pass, 8GiB default envelope via raw_authority_whale_payload_bytes, stream-safe-only, commit-batched, daemon events, default-on with daemon_whale_raw_materialization off-switch; coordinator review on the PR). Deployed to sinnix via flake bump 354be99 + switch. REMAINING for close: live witness codex:019f49d8 resolves to a head via the daemon whale pass — blocked until the operator resolves the durable stale-plan blocker (raw-authority-blocker:5406c7c3…, script staged) since ALL materialization passes fail-closed behind it.\n2026-07-22 recovery correction: PR #3267 supplies the dedicated census-reset mechanism now under review. It requires a verified source-tier backup manifest and an offline daemon before it clears only derived census bookkeeping; accepted raw authority remains intact. It also prunes only index revision seeds whose source raw no longer exists, through the active index pointer. Once merged and deployed, use its dry-run and verified backup receipt before applying, then confirm the fresh census removes the stale-plan blocker before retrying whale convergence.\n\n2026-07-27: confirmed the live archive still has an active stale_plan raw-authority-blocker (raw-authority-blocker:2a4fb67b97a896111abc4681d3cfc52d4e40f85e38b710d97f67a60143b69bfe - different id than this bead's previously-cited 5406c7c3..., which is gone/superseded by a later census, as expected) that fail-closes ALL materialization passes archive-wide, same failure mode described in this bead's notes for the codex:019f49d8 whale witness. polylogue-d7im's auto_resolve_stale_plan_blockers fix (PR #3287, merged+deployed) should clear this class of blocker automatically once the daemon's current watcher catch-up backlog finishes and _periodic_raw_materialization_convergence runs (gated behind catch_up_complete_gate). Re-check whale convergence status (codex:019f49d8 head materialization) after that clears - do not re-diagnose from scratch, this is very likely the same root cause already tracked in d7im.\n2026-07-27T06:11 update: whale-pass mechanism verified sound (directly invoked raw_authority.whale_pass_candidate() against the live archive read-only - correctly returns cc83e374b3... as an eligible candidate, confirming the earlier stream-safety exclusion bug for expanded members is indeed already fixed in master). NOT a bug that it hasn't run yet: the daemon log shows the ordinary trickle conveyor just discovered a fresh 4331-candidate/0.54GiB bulk-scale backlog (materialized.remaining_candidates=4288, made_progress=True) the moment the stale-plan blocker cleared and the watcher catch-up backlog drained (polylogue-d7im). _maybe_run_raw_materialization_whale_pass only runs when the ordinary conveyor is quiescent for that tick - correctly gated off while this fresh backlog is being worked. Daemon's own advisory log line suggests 'polylogue ops maintenance rebuild-index' (bulk blue-green rebuild) as faster than waiting on trickle for backlogs this size, but I did not trigger that myself (heavier/resource-intensive operation, deferring to operator). Will keep monitoring via periodic wakeup; expect whale pass to fire once this fresh backlog quiesces.\n2026-07-27T07:10 rate analysis: trickle conveyor discovered a fresh backlog after d7im's stale-plan fix cleared (4331 initial). Measured drain rate across 3 samples: 4272-\u003e4256 (08:32:59-\u003e08:40:16, -16/7.3min) and 4256-\u003e4224 (-\u003e09:05:48, -32/25.5min) = ~1.25 candidates/min average. At 4224 remaining, that's ~56 hours (~2.3 days) to reach quiescence via trickle alone -- the whale escalation pass (which needs a fully quiescent tick) will not fire on any session-scale timeframe at this rate. This matches the daemon's own advisory log line verbatim: 'the trickle conveyor is sized for steady-state drift and can take weeks on a backlog this size; run polylogue ops maintenance rebuild-index for a resumable blue-green bulk rebuild instead of waiting on this conveyor.' Did not trigger that myself (heavier/resource-intensive operation against the live personal archive, correctly deferred to operator per this session's risk posture). Recommend operator either (a) runs the suggested rebuild-index pass, or (b) accepts multi-day background convergence and lets it drain unattended. Not scheduling further short-interval check-ins on this specific number until either the rate changes materially or the operator acts.\n2026-07-27 ~16:50 UTC: whale pass's 'fail-closed behind 1 unresolved durable stale-plan blocker' (seen 22:47 and 00:41 attempts) is very likely the exact fold_duplicate_alias non-convergence bug just root-caused and fixed in polylogue-ihc8 (PR #3326, merged). Confirmed via journalctl the plan raw-authority-frontier:058be945e0d8... is still failing as of 16:46:58 because polylogued.service is running a pinned Nix build (polylogue-0.3.0), not the merged fix -- needs a sinnix pin bump + rebuild + service restart to take effect. Deploy deliberately not triggered without operator confirmation (bouncing the live daemon). Once deployed, expect this specific stale-plan blocker to clear and the whale pass to proceed past it.\n2026-07-27 ~17:35 UTC: post-redeploy check (daemon restarted 18:33 CEST with ihc8 fix live) — no raw-authority pass has fired yet in this daemon lifetime (1h9min uptime, still doing ordinary watcher catch-up: 18827 sessions/4.9M messages indexed per heartbeat). Consistent with the earlier finding that the whale/raw-authority pass needs a quiescent tick, which the trickle backlog (~56h ETA) won't produce on any short timeframe. Not holding a live monitor open for this; will check again on a longer horizon (next session or explicit request) rather than polling.","status":"open","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-21T21:17:41Z","created_by":"Sinity","updated_at":"2026-07-27T17:43:33Z","labels":["area:daemon","area:perf","area:storage"],"comments":[{"id":"019f8acc-2842-7288-a38b-5e51f6bbfd97","issue_id":"polylogue-t93b","author":"Sinity","text":"2026-07-22 census-state note (from hook de-inflation, polylogue-31r1): the live archive's raw-authority census is internally inconsistent and must be reconciled/rebuilt as part of this convergence work. Cause: hook de-inflation deleted 64,896 hook raw_sessions; those hook raws had ~64,895 frontier plans + blockers + census_plans/post_plans (hook noise flooding the authority machinery). A surgical orphan-plan deletion (PR #3266, now closed) removed the dangling plans but broke the carried-forward/retryable postflight invariant (raw_authority.py:1315). Daemon now defers census passes to convergence_debt (736+) instead of the prior stale-plan-blocker degradation; it survives (0 crashes), archive data correct. Recommended resolution: full raw-authority census rebuild over the current hook-free raw set (no prior-census carried-forward comparison), preserving accepted heads/revision_authority (byte_proven 15,465 / quarantined 20,986). No dedicated census-reset mechanism exists yet.","created_at":"2026-07-22T17:07:43Z"},{"id":"019f8b0b-295b-7b0b-9a1e-8187dac6711f","issue_id":"polylogue-t93b","author":"Sinity","text":"2026-07-22 precise convergence wall (after hook de-inflation + census reset + index-seed prune unblocked everything else): the daemon whale-pass candidate scan returns None because the eligible components are NOT stream-safe. Live: whale_pass_candidate=None; top materialization components are (1) 6.33GB / 803 members / stream_safe=FALSE — the codex 019f49d8 witness; (2) 1.28GB / 6695 members / stream_safe=FALSE; (3) 582MB / 9 / stream_safe=FALSE. raw_materialization_whale_pass_candidate (repair.py:4137) skips any component with a non-stream-safe member, so these stay typed-blocked exactly as designed. Ordinary candidates=1371; authority_quarantined=2085; byte_authority_quarantined=843. byte_proven=15465 / quarantined=20986 (most quarantined are members of the non-stream-safe whale components).\n\nSo full convergence is blocked on the ORIGINAL t93b design constraint: the whale members are not stream-record-safe, so the memory-bounded whale pass cannot parse them. Resolving needs a streaming parse path for the non-stream-safe codex members (or an offline bounded handling), plus authority refinement for the genuinely-ambiguous quarantined raws. NOT a hook-residue problem. Prerequisites now satisfied: stale-plan blocker cleared, census healthy (rebuilds fresh), hook residue gone, #3261 whale-budget deployed.","created_at":"2026-07-22T18:16:32Z"},{"id":"019f8b0d-de45-7ac4-812a-b11f5ad77276","issue_id":"polylogue-t93b","author":"Sinity","text":"2026-07-22 whale-pass stream-safety lead: raw_materialization_whale_pass_candidate returns None because _raw_materialization_component_stream_safe judges the whole 803-member whale component non-stream-safe. Root: _raw_materialization_stream_safe(candidates, raw_id) reads candidates.raw_origins/.raw_source_paths, but the ordered component includes ALREADY-MATERIALIZED (non-candidate) members not in the candidate maps -\u003e origin=None -\u003e is_stream_record_provider(None,None)=False. 783 of 803 whale members are non-candidate (real codex rows in raw_sessions, byte_proven). Memberships are clean (0 orphaned). So the whale is likely wrongly excluded: stream-safety should be resolved from raw_sessions for ALL component members, not just candidates. Candidate fix locus: repair.py:4016 _raw_materialization_stream_safe / 4130-4139 component scan. If confirmed, the whale (and the 1.28GB/582MB components) become eligible and the daemon whale pass can converge them.","created_at":"2026-07-22T18:19:30Z"}],"dependency_count":0,"dependent_count":0,"comment_count":3} {"_type":"issue","id":"polylogue-meoz","title":"ArchiveStore.delete_sessions detonates per-row derived-refresh triggers: 91-session delete ran 3h with 375GB reads and zero commit","description":"Live incident 2026-07-21 (yqeo retirement): ArchiveStore.delete_sessions on 91 hermes sessions sat 3h in one transaction: 375GB read (11 full scans of the 34GB index.db), 2MB written, WAL empty — killed and rolled back. py-spy: stuck in the per-session DELETE FROM sessions loop (archive.py:6725). Root cause: blocks_action_pairs_ad fires PER DELETED BLOCK ROW and each firing (a) deletes+rebuilds the whole session action_pairs with two window-function scans and (b) re-derives delegation_facts from delegation_facts_source. The production bulk write path suppresses this machinery via derived_refresh_guard rows (session-write, fts-bulk-session-write) but delete_sessions — the PRODUCT deletion API used by the CLI delete verb and SessionDeleteActuator — never sets them. Same pathology family as polylogue-crd8 (whale prefix-tail rewrite FTS/trigram detonation).","design":"Fix in delete_sessions itself (and any sibling bulk mutation entrypoints): wrap the delete in the derived_refresh_guard rows, do one-pass FTS maintenance explicitly (blocks_command_trigram delete commands with old text before block rows go away; contentless messages_fts DELETE by rowid), let indexed FK cascades remove the tree, clear guards, commit. Working reference implementation: /realm/tmp/worktrees/yqeo-v42/yqeo_retire_stale_v2.py (operator-run 2026-07-21). Regression test: seeded session with tool_use blocks, delete via product API, assert FTS docsize parity and action_pairs cleanup without trigger-driven rebuild (e.g. count trigger firings via guard-sensitive canary or measure statement count). Also audit epoch triggers (query_unit_frame_*_delete) cost under bulk cascade.","acceptance_criteria":"delete_sessions (and executor SessionDeleteActuator route) deletes a many-block session in seconds not hours; FTS/trigram stay coherent (docsize==indexable parity) after delete; regression test proves per-row action_pairs/delegation rebuild machinery does not fire during bulk delete; crd8 relation noted.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-21T19:15:22Z","created_by":"Sinity","updated_at":"2026-07-27T01:27:53Z","started_at":"2026-07-21T23:57:55Z","closed_at":"2026-07-27T01:27:53Z","close_reason":"Fixed and merged 2026-07-26 in PR #3263 (commit 096374983) — ArchiveStore.delete_sessions now wraps the whole batch in the same derived_refresh_guard rows the bulk session-write path uses (session-write + fts-bulk-session-write), does one explicit session-scoped FTS/trigram/action_pairs/delegation_facts pass instead of per-block trigger detonation, then removes physical rows via indexed FK cascade. Confirmed independently this session (2026-07-27) while investigating the same incident: attempted a narrower guard-only fix, found master already had a more complete version (also handles FTS/trigram, explicit belt-and-suspenders cleanup) already tested. Bead was stale (still in_progress with no completion note) - closing now.","labels":["area:perf","area:storage"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-zoc3","title":"ingest_record decode rejects binary provider payloads the rebuild parse path handles","description":"Found 2026-07-21 during the polylogue-yqeo targeted Hermes reprocess: parse_from_raw → process_ingest_batch → ingest_record fails all 3 hermes verification raws (source_path ~/.hermes/verification_evidence.db, SQLite database bytes) with \"decode: str is not valid UTF-8: surrogates not allowed: line 1 column 1\" — the worker decode step assumes text/JSON payloads before provider dispatch. The REBUILD path (revision_backfill._parse_retained_raw → sources/dispatch.parse_payload) parses these same raws fine (the v42 walk materialized verification sessions from them), so the two parse routes disagree on binary-payload providers. Consequence: targeted reprocess cannot re-materialize hermes verification sessions under the composed verification:\u003craw_id\u003e@profile-\u003ckey\u003e scheme (#3227); 4 stale old-pattern verification:2026* sessions remain in the index with no composed successors (retained deliberately — deleting them would lose read coverage).\n\nFix: route ingest_record payload decoding through the same provider-dispatch-aware envelope the rebuild path uses (binary-capable: detect_provider on bytes before any text decode), or teach build_raw_payload_envelope the binary lane. Add a contract test: any raw parseable by revision_backfill._parse_retained_raw must be parseable by ingest_record (parse-route parity for a representative binary fixture — the hermes verification fixture family exists under tests/fixtures/hermes/).\n\nAfter the fix: reprocess the 3 verification raws (coordinator, live archive), retire the 4 stale verification:2026* ids, and update the yqeo receipt.","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-21T15:08:01Z","created_by":"Sinity","updated_at":"2026-07-21T16:34:33Z","closed_at":"2026-07-21T16:34:33Z","close_reason":"Fixed in PR #3247 (merged): build_raw_payload_envelope now probes BOTH Hermes SQLite artifacts (state.db + verification_evidence.db) via the parsers own looks_like_*/marker_payload helpers BEFORE any text decode — ingest_record and the rebuild route now agree on binary payloads; marker classification extracted+shared so the decoded marker session classification is not shadowed by the .db path-only sidecar rule; profile_root threaded in backfill for identical composed ids on both routes. Parity contract test (243 lines) incl. exact live-error reproduction. Lane was 522-killed twice post-push; coordinator verified helpers + re-ran 51 tests on the branch and opened/merged the PR. Unblocks the yqeo verification-raw reprocess.","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -76,7 +77,7 @@ {"_type":"issue","id":"polylogue-t3gk","title":"Index fast-forward declarations have no executor — v43 declaration unreachable on live archive","description":"Hit live 2026-07-21, minutes after merging #3235: the freshly promoted v42 generation cannot be opened by current master — initialize_archive_database raises \"index.db schema version 42 is not the current index tier version 43; move it aside and rebuild the archive root\" even though #3235 shipped a DECLARED clone-safe fast-forward (IndexDeltaDeclaration v43, FastForwardOperation v43-messages-fts-identity, kind=REBUILD_FTS). Grep proves nothing outside storage/sqlite/lifecycle.py consumes IndexFastForwardPlan/eligible_for_sql_fast_forward — the declaration registry exists but no open-path executor applies it, so every declared-benign version bump still forces the full rebuild the declaration exists to avoid (11h on the current corpus vs ~minutes for the declared op).\n\nConcrete impact: post-promote yqeo Hermes reprocess had to run from a pinned pre-v43 worktree (cwd-first import) as a workaround; daemon deploy (dcz5) will hit the same wall on startup.\n\nFix: wire a fast-forward executor into the index-tier open path (bootstrap initialize_archive_database or lifecycle open): when PRAGMA user_version is behind INDEX_SCHEMA_VERSION and a contiguous declared plan with eligible operations covers the gap, apply the operations (create/drop declared objects, re-run trigger DDL, repopulate via the declared rebuild SQL), bump user_version one declaration at a time, and record a receipt; fall back to the rebuild-required error only when a gap version lacks a declared eligible plan (e.g. SEMANTIC_REPARSE). Must be single-writer-safe (daemon startup owns it) and idempotent on crash mid-apply. Test: build a v42-shaped fixture, open under v43 code, assert ledger populated + user_version=43 + zero identity mismatch; assert SEMANTIC_REPARSE declarations still refuse.","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-21T06:14:26Z","created_by":"Sinity","updated_at":"2026-07-21T12:55:45Z","closed_at":"2026-07-21T12:55:45Z","close_reason":"Shipped in PR #3238 (merged): apply_index_fast_forward executor wired into initialize_archive_database — declared clone-safe version gaps now fast-forward on open (generic FastForwardOperationKind dispatch, per-declaration idempotent transactions, canonical DDL from live INDEX_DDL); SEMANTIC_REPARSE spans still refuse with the rebuild-required error. Live-incident reproduction test: v42-shaped fixture opens under v43 code, ledger populated, zero identity mismatch, idempotent reopen. Unblocks dcz5 daemon deploy.","dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"polylogue-yqeo","title":"Post-promote targeted Hermes reprocess: retire stale unqualified observer/verification session ids","description":"PRs #3224 (profile qualification), #3225 (artifact-family qualification, pending merge), and #3227 (verification-family qualification) changed parser-derived native ids for Hermes observer-evidence sessions (now observer:atif|atof:\u003craw_id\u003e@profile-\u003ckey\u003e and verification:\u003craw_id\u003e@profile-\u003ckey\u003e). This is semantic-reparse-required (CodeRabbit P1 on #3225, acknowledged): sessions already materialized under the old unqualified ids (observer:\u003cid\u003e, verification:\u003cid\u003e, and pre-#3224 unprofiled variants) are stale and will NOT be replaced by content-hash idempotency because the new ids create NEW sessions — the old rows become orphans. Deliberately NOT bumping INDEX_SCHEMA_VERSION (full-archive rebuild) because the v42 blue-green rebuild is mid-flight and Hermes raws are a tiny subset.","design":"After v42 promote + #3225 merge + daemon deploy (dcz5): (1) enumerate hermes-origin raws in source.db; (2) reprocess them through the daemon bulk route so sessions re-materialize under composed ids; (3) enumerate and delete index.db sessions whose session_id matches the old unqualified patterns (observer:\u003craw\u003e without atif/atof segment, verification:\u003craw\u003e without @profile-, observer:atif|atof:\u003craw\u003e without @profile-) and which have a composed-id successor for the same raw evidence; (4) receipt: counts before/after, zero old-pattern ids remaining with successors present. Index tier is rebuildable — deletion is safe; do not touch source.db.","acceptance_criteria":"Receipt shows every hermes-origin raw re-materialized under a composed id; no index.db session remains with an old-pattern observer/verification id that has a composed successor; spot-read one ATIF + one ATOF + one verification session via the read surface resolves the composed identity and parent links.","notes":"2026-07-21 reprocess receipt (worktree-pinned pre-v43 code, systemd unit, exit 0): 352 hermes raws reprocessed with force_write; counts sessions=35 written / 416 idempotent-skips / 4815 messages. Observers FULLY migrated: 8 composed observer:atif|atof:\u003cid\u003e@profile-\u003ckey\u003e sessions, 0 old-pattern observers remain; spot-reads resolve composed ids + branch links to profile-qualified producers (ATIF f95f712ebce3, ATOF 9cc2ec93471f). 103 plain profile-qualified sessions materialized. REMAINING: (1) 91 stale unqualified sessions with qualified successors — retirement script ready (ArchiveStore.delete_sessions), blocked pending operator confirmation of the destructive step; (2) 3 verification raws fail ingest_record decode (binary SQLite payloads — polylogue-zoc3 parse-route parity bug), so 4 old-pattern verification:2026* sessions retained deliberately until zoc3 lands.\n2026-07-21: retirement v1 (bare delete_sessions) killed after 3h — blocks_action_pairs_ad per-row detonation, 375GB reads, zero writes, clean rollback (273 intact). Product bug filed as polylogue-meoz. Retirement v2 (derived_refresh_guard rows + one-pass FTS maintenance + indexed FK cascades, yqeo_retire_stale_v2.py in pinned worktree) launched with renewed operator approval.","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-20T20:18:42Z","created_by":"Sinity","updated_at":"2026-07-21T19:52:15Z","closed_at":"2026-07-21T19:52:15Z","close_reason":"Complete. Final census (2026-07-21): hermes total=182 (was 273): 91 stale unqualified sessions with qualified successors deleted via guarded bulk delete (yqeo_retire_stale_v2.py — derived_refresh_guard rows + one-pass FTS maintenance; ~8min; FTS parity exact 4753541==4753541; v1 bare delete_sessions detonated per-row triggers → bug polylogue-meoz). 3 binary-SQLite verification raws reprocessed with post-#3247 master code: 4 composed verification:\u003cid\u003e@profile-7ff44102c8e5 sessions created, old-pattern rows replaced in place by full-replace revision machinery, 0 failures. Composed observers=8, plain_qualified=103, plain_unqualified=67 (no successors — kept), unqualified_with_successor_remaining=0. Bonus: first live production run of #3238 index fast-forward executed v42→v43 on open — user_version=43, messages_fts_identity ledger populated at exact block parity.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-xb4i","title":"Parse prefetch/cache admission must bound parsed-tree bytes, not raw payload bytes","description":"Two earlyoom kills of the v42 rebuild driver (19.3G and 20.2G peaks, 2026-07-20) on a whale-dense page: the DaemonParseStage admission budget (#3195, RAM/16 clamp) and the prefetch cache both account raw PAYLOAD bytes, but parsed ParsedSession trees inflate ~10x+ payload, so a 2GiB payload admission can resident tens of GB of trees; clamping inflight to 256MiB did not help because the CACHE retains the whole page of parsed trees regardless. Fix: account estimated in-memory tree size in both admission and cache retention, with eviction/spill for whales. Interim mitigation in the live walk: 500-raw pages + MemoryHigh=14G on the unit.","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-20T13:38:46Z","created_by":"Sinity","updated_at":"2026-07-20T14:03:58Z","closed_at":"2026-07-20T14:03:58Z","close_reason":"Shipped in PR #3209 (merged): estimate_parsed_tree_bytes structural estimator (two-term fit calibrated against deep-walk measurements, constants rounded up so misestimation biases to eviction/reparse), adaptive cached-tree budget RAM/8 clamped [256MiB,4GiB] with POLYLOGUE_DAEMON_PARSE_STAGE_MAX_CACHED_TREE_BYTES override, side-ledger tracking with largest-first eviction and whale-never-retained. Root cause of the two 2026-07-20 earlyoom kills (19.3/20.2G peaks). 12 tests, mypy --strict, verify --quick green.","labels":["area:daemon","area:perf","horizon:frontier"],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"polylogue-623q","title":"Import performance envelope: full-corpus rebuild must land in well under an hour","description":"Operator mandate 2026-07-20: the v42 full-archive rebuild (101K raws) taking multiple hours-to-days is unacceptable — import at this scale, including all remaining derived steps, must complete in well under an hour. Measured state: parallel warm parse is ~30-60 raws/s (fine); the serial engine pass was ~3 raws/s with \u003e50% of it census/spill cache overhead + per-unit fsync commits. Landed levers: #3208 (stage telemetry, no-fsync NVMe spill + decoded RAM layer, commit_batch_size=200 in the rebuild path). Remaining candidate levers, to be driven by per-page stage timings: census receipt cost, index full_replace batching, model_usage_seed, census-parse vs warm-cache dedup, writer-thread pipelining of serialization vs SQLite execution. Exit criterion: a measured full-corpus rebuild receipt under 60 min on this machine, recorded on this bead.","notes":"Levers landed 2026-07-20 evening: #3208 (stage telemetry + census commit batching commit_batch_size=200 with replay per-cohort for blob-publication lock safety + no-fsync NVMe spill + FIFO decoded RAM layer), #3209 (tree-byte cache accounting, xb4i closed), #3210 (idx_action_pairs_tool_result_block — root cause of whale clear_projection_rows pathology, 445x measured; idx_paste_spans_session 800x; pricing-hash lru_cache 97%). Both new indexes are benign runtime DDL, applied on connect. Baselines: normal 2000-page engine was census=135 spill=105 writes~100 (pre-levers); whale 500-page total=675 w/ clear_projection_rows=92.7. Next measurement: resume #22 pages post-#3210. Remaining levers: census cost with partial warm coverage (warm timeout on whale pages leaves ~half uncached → serial census parse), spill_load still ~120s on whale pages (decoded layer too small for whales), writer pipelining.\n[2026-07-20] Lever merged: PR #3219 (93a5388cf) pure prepare_session_rows extraction — message/block row building now off-writer-computable (PreparedSessionRows + conservative reuse gate: content-hash match, no merge_append, no prefix-sharing slice). Daemon prefetch-cache wiring deliberately deferred (separate change through census/spill/replay). Bench: 3.4-13.4% on messages+blocks stage at synthetic scale.\n[2026-07-20 22:25] Whale-page datum from live v42 walk (resume26, pipelined driver): 856s page total with spill_load=732s (85%) — census=43s, index_parsed_write=36s. Spill reload of giant parsed trees is now the dominant whale cost (normal pages 28-52s, lookahead hides warm entirely). Lever candidate: whale-aware spill (skip decoded-layer eviction for in-flight page raws, or stream-parse whales instead of pickle round-trip). Analyze against final full-import benchmark.\n2026-07-21 v42 walk terminal receipt: generation gen-1784486727919-da69ed72 PROMOTED. 101,347 raws → 83,279 sessions / 4.77M messages. Driver: 36 passes, 662.9 min total driver wall across resume26 (11h02m unit wall, 7h09m CPU, 16G peak RSS, 6.9G swap peak). Verification: 0 dup variants, FTS delta 0, 0 dangling branch points, 16,364 sessions priced ($250K API-equiv). NOT the \u003c60min exit receipt — that needs the fresh full-import benchmark with levers. Strongest whale evidence yet: single page total=1440.2s with spill_load=598.2s (41%); second whale page 607.5s/spill 228.1s. Whale-aware spill (stream-parse or size-partitioned spill) remains the top lever. Cross-path parse dedup (#3234, 869u) now also cuts census parse cost on the next full walk.\nLever bead filed: polylogue-odm1 (whale-aware census spill — spill_load 41% of whale pages, receipts on the bead). Related, not a blocks-edge (epic/task constraint).\nodm1 whale-residency lever MERGED (#3237): benchmark shows pickle round trip eliminated for resident whales (~0.36ms/MB reload cost removed; live spill_load receipts were 41% of whale pages). Next full-import benchmark runs with this + #3234 cross-path parse dedup.","status":"open","priority":1,"issue_type":"epic","owner":"ezo.dev@gmail.com","created_at":"2026-07-20T13:38:45Z","created_by":"Sinity","updated_at":"2026-07-21T12:54:02Z","labels":["area:ingest","area:perf","horizon:frontier"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-623q","title":"Import performance envelope: full-corpus rebuild must land in well under an hour","description":"Operator mandate 2026-07-20: the v42 full-archive rebuild (101K raws) taking multiple hours-to-days is unacceptable — import at this scale, including all remaining derived steps, must complete in well under an hour. Measured state: parallel warm parse is ~30-60 raws/s (fine); the serial engine pass was ~3 raws/s with \u003e50% of it census/spill cache overhead + per-unit fsync commits. Landed levers: #3208 (stage telemetry, no-fsync NVMe spill + decoded RAM layer, commit_batch_size=200 in the rebuild path). Remaining candidate levers, to be driven by per-page stage timings: census receipt cost, index full_replace batching, model_usage_seed, census-parse vs warm-cache dedup, writer-thread pipelining of serialization vs SQLite execution. Exit criterion: a measured full-corpus rebuild receipt under 60 min on this machine, recorded on this bead.","acceptance_criteria":"1. A full-corpus rebuild from durable tiers completes in well under one hour on the reference host, measured end to end including every derived step, not per-stage. 2. The measurement is reproducible and reported with the corpus shape it ran against (raw count, blob bytes, index bytes), so a later regression is attributable. 3. Stage telemetry attributes the wall clock to named stages; no stage is an unexplained remainder. 4. The envelope holds for the whale-raw path, or whale handling is a declared separate envelope with its own bound rather than an unbounded tail.","notes":"Levers landed 2026-07-20 evening: #3208 (stage telemetry + census commit batching commit_batch_size=200 with replay per-cohort for blob-publication lock safety + no-fsync NVMe spill + FIFO decoded RAM layer), #3209 (tree-byte cache accounting, xb4i closed), #3210 (idx_action_pairs_tool_result_block — root cause of whale clear_projection_rows pathology, 445x measured; idx_paste_spans_session 800x; pricing-hash lru_cache 97%). Both new indexes are benign runtime DDL, applied on connect. Baselines: normal 2000-page engine was census=135 spill=105 writes~100 (pre-levers); whale 500-page total=675 w/ clear_projection_rows=92.7. Next measurement: resume #22 pages post-#3210. Remaining levers: census cost with partial warm coverage (warm timeout on whale pages leaves ~half uncached → serial census parse), spill_load still ~120s on whale pages (decoded layer too small for whales), writer pipelining.\n[2026-07-20] Lever merged: PR #3219 (93a5388cf) pure prepare_session_rows extraction — message/block row building now off-writer-computable (PreparedSessionRows + conservative reuse gate: content-hash match, no merge_append, no prefix-sharing slice). Daemon prefetch-cache wiring deliberately deferred (separate change through census/spill/replay). Bench: 3.4-13.4% on messages+blocks stage at synthetic scale.\n[2026-07-20 22:25] Whale-page datum from live v42 walk (resume26, pipelined driver): 856s page total with spill_load=732s (85%) — census=43s, index_parsed_write=36s. Spill reload of giant parsed trees is now the dominant whale cost (normal pages 28-52s, lookahead hides warm entirely). Lever candidate: whale-aware spill (skip decoded-layer eviction for in-flight page raws, or stream-parse whales instead of pickle round-trip). Analyze against final full-import benchmark.\n2026-07-21 v42 walk terminal receipt: generation gen-1784486727919-da69ed72 PROMOTED. 101,347 raws → 83,279 sessions / 4.77M messages. Driver: 36 passes, 662.9 min total driver wall across resume26 (11h02m unit wall, 7h09m CPU, 16G peak RSS, 6.9G swap peak). Verification: 0 dup variants, FTS delta 0, 0 dangling branch points, 16,364 sessions priced ($250K API-equiv). NOT the \u003c60min exit receipt — that needs the fresh full-import benchmark with levers. Strongest whale evidence yet: single page total=1440.2s with spill_load=598.2s (41%); second whale page 607.5s/spill 228.1s. Whale-aware spill (stream-parse or size-partitioned spill) remains the top lever. Cross-path parse dedup (#3234, 869u) now also cuts census parse cost on the next full walk.\nLever bead filed: polylogue-odm1 (whale-aware census spill — spill_load 41% of whale pages, receipts on the bead). Related, not a blocks-edge (epic/task constraint).\nodm1 whale-residency lever MERGED (#3237): benchmark shows pickle round trip eliminated for resident whales (~0.36ms/MB reload cost removed; live spill_load receipts were 41% of whale pages). Next full-import benchmark runs with this + #3234 cross-path parse dedup.\nCORPUS SHAPE 2026-07-28, for the AC's 'measured against a stated shape' clause: source.db 41,363 raws / 17,152 distinct native_ids / 4.0 GB; blob 69 GB; active index generation 36 GB; index 18,871 sessions / 5,042,564 blocks. The operator mandate cites the v42 rebuild at 101K raws taking hours-to-days -- the current corpus is 41K raws, so a re-measure is needed before assuming the old figure still describes the cost.\n\nRelated: polylogue-9rw0.1 (filed 2026-07-28) removes a whole class of reason to run a full rebuild at all -- an additive-column delta currently classifies as SEMANTIC_REPARSE and triggers full replay where a shape fast-forward plus a scoped reprocess would do.","status":"open","priority":1,"issue_type":"epic","owner":"ezo.dev@gmail.com","created_at":"2026-07-20T13:38:45Z","created_by":"Sinity","updated_at":"2026-07-28T20:06:07Z","labels":["area:ingest","area:perf","horizon:frontier"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-q88p","title":"Content-address the embeddings tier: vectors keyed by identity-free input hash","description":"Operator ruling 2026-07-20: reindexing must not lose embeddings - vectors are about content, not transient index identity. Current defect: message_embeddings_meta binds vectors to messages.content_hash, and _message_content_hash INCLUDES session_id/position/variant_index (identity-contaminated), so rebuilds/lineage shifts invalidate vectors whose text never changed - hence the 777K-vector rescue (04kl). Fix: key the vector store by embedding_input_hash = H(model, normalized embedder input text) - identity-free, same philosophy as the svfj block evidence hash which deliberately excludes identity. Index side keeps a rebuildable message_id -\u003e input_hash mapping; freshness = input_hash lacks a vector; dedup free (fork-replayed identical messages embed once - real API savings in a lineage-heavy archive). End state: rebuilds CANNOT lose embeddings by construction; the rescue concept is retired (automagic doctrine). ORDERING: design this first, then execute the one-time 04kl rescue directly INTO the content-addressed layout (avoid double migration). Embeddings tier schema bump = derived-tier regime (edit canonical DDL + rebuild plan = the rescue itself).","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-20T00:56:55Z","created_by":"Sinity","updated_at":"2026-07-20T05:06:31Z","closed_at":"2026-07-20T05:06:31Z","close_reason":"PR #3192 merged: embeddings tier v4 content-addressed — vectors keyed by identity-free embedding_input_hash(model, NFC input text); rebuildable message_id→hash refs in embeddings tier (index version untouched); all consumers retargeted both twins; 04kl rescue lands into v4; rebuild-survival/dedup/property tests. Reindexing can no longer lose embeddings by construction. Follow-up debt noted in PR: reconcile-path vector GC deferred.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-fbte","title":"Rebuild resume re-walks entire corpus: replay phase never populates its cursor","description":"Observed on operation ab5bad1f (2026-07-20): the rebuild transaction record has last_raw_id=None, processed_raw_count=0 even after committing 31,882 sessions - the replay phase never writes its positional cursor, so every resume re-walks the ENTIRE raw corpus relying on per-raw skip fast-paths (byte-proven supersedence #3146, content-hash match). Measured cost: ~2.25h of pure re-verification walk per resume on the ~50K-raw archive, proportional to corpus size instead of remaining work. Fix directions: (a) populate last_raw_id/processed_raw_count during replay batches (fields already exist in the transaction schema), resume seeks past them; or (b) resume-time cheap skip via indexed committed-membership lookup (raw_id already classified in the generation) instead of parse+hash per raw. Either makes recovery O(remaining). Related: polylogue-6mvg (rebuild throughput program).","notes":"2026-07-20 operator-driven rescope: do NOT build the cursor fix on the CLI resume surface - gd6v deletes that command on proven equivalence, so investment there is throwaway. The O(remaining)-resume property is a REQUIREMENT OF THE REPLACEMENT: gd6v daemon bulk path must record replay-phase progress (or skip via committed-membership lookup) so interruption recovery is proportional to remaining work, verified as part of gd6v equivalence gate. This bead stays as the requirement record; implementation lands in gd6v.","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-20T00:51:28Z","created_by":"Sinity","updated_at":"2026-07-20T19:25:25Z","closed_at":"2026-07-20T19:25:25Z","close_reason":"Rescoped requirement (O(remaining-work) resume via daemon bulk path) satisfied by PR #3189: checkpoint_transaction persists last_raw_id/processed_raw_count (index_generation.py:317-351), next_raw_page resumes from the keyset cursor (:380-398), rebuild_index.py:389-396 checkpoints after each replay page. Test receipt: test_daemon_bulk_rebuild_pass_resumes_without_reprocessing_raw_ids passes on master; live receipt: the v42 walk resumed from durable cursors across ~20 restarts on 2026-07-20. CLI-path manifestation is out of scope per operator rescope (deletion tracked by polylogue-4jsk). Audit by haiku lane with per-claim citations.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-o7hx","title":"Hook-spool isolation must derive from archive root, not global XDG","description":"Hazard bitten 4+ times (lrou twice, ajmu lane 2026-07-20 drained 6262 real hook events into a scratch archive): hooks_sidecar_dir() in paths/_roots.py resolves data_home()/hooks from pure XDG, independent of the archive root, so ANY daemon - whatever --root / POLYLOGUE_ARCHIVE_ROOT says - drains the one real global spool. The POLYLOGUE_HOOK_SIDECAR_DIR env override is a manual escape hatch agents must remember (and did not, four times) - exactly the pattern the automagic doctrine purges. Fix: the spool path derives from the RESOLVED archive root (default \u003carchive_root\u003e/hooks), which is byte-identical to the current path for the default production root (archive root IS data_home) - zero migration. All consumers (daemon drain, hook_paste_enrichment, cli init, agent_integration installer-rendered writer scripts) use the same resolved path; a scratch-rooted daemon then reads a scratch spool by construction. Fold the env override away per no-compat doctrine (config key if genuine configurability is wanted). Writer scripts get the concrete path baked at install time by the installer.","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T23:37:08Z","created_by":"Sinity","updated_at":"2026-07-20T00:54:45Z","closed_at":"2026-07-20T00:54:45Z","close_reason":"PR #3187 merged: spool path now derives from resolved archive root (byte-identical for prod root, scratch daemons isolated by construction); second instance of the bug fixed in config.py captured default; env override deleted; installer bakes --sidecar-dir into writer scripts; stale hazard docs removed. Deploy step recorded on dcz5 (bake paths when pin bumps). Closes the 4x-bitten scratch-daemon spool drain hazard.","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -91,7 +92,7 @@ {"_type":"issue","id":"polylogue-2i2w","title":"action_pairs materializes full text copies: ~2x index bloat and massive write amplification","design":"Found 2026-07-19 via dbstat on the live rebuild generation: action_pairs = 4.10GB for 868,522 rows (~4.7KB/row) vs blocks = 4.61GB — the pair table stores COPIES of tool_input and output_text, so every tool interaction exists ~3x (blocks.text/search_text, action_pairs.output_text/tool_input, plus messages_fts when populated). Consequences measured live: (1) index.db ~19.6GB where content would suggest half that; the b-tree working set exceeds page cache and whale session replaces degrade to storage-bound random IO (164MB/s reads observed); (2) write amplification — refresh_action_pairs (called per session write AND by the ad/ai/au trigger family for non-writer mutations) does DELETE-all + INSERT-all of the session pairs including the text copies, so a whale replace rewrites GBs; (3) every byte is paid again in backup/checkpoint/cache. Design directions (decide explicitly): (a) action_pairs stores only the join/rank/outcome columns (tool_use_block_id, tool_result_block_id, session_id, message_id, tool_id, use_rank, tool_name, semantic_type, tool_command, tool_path, is_error, exit_code) and text is read from blocks by block_id at query time (the actions VIEW already joins; read surfaces need the join added — audit consumers of action_pairs.output_text/tool_input via rg); (b) keep tiny previews (first N chars) for list surfaces, full text via join. Derived-tier schema change: canonical DDL + INDEX_SCHEMA_VERSION bump + rebuild (batch with other pending index-tier changes per schema regime). Cross-ref: the FTS-empty bulk-mode bead (skip action_pairs refresh during bulk entirely), l3tk (this table was also the planner-pathology site), 20d interactive perf (smaller table = better cache behavior for the action-heavy queries).","acceptance_criteria":"Decision recorded (a vs b) with consumer audit; DDL change lands with the next batched index-tier bump; measured index size reduction and whale-replace write time on the benchmark corpus; query surfaces reading pair text keep byte-identical outputs via the join.","notes":"2026-07-19 implementation trail (worktree agent-ab497ea4f525afdd2):\n\nConsumer audit: grepped every reader of action_pairs.output_text/tool_input across storage/repository, insights, daemon, MCP, CLI, api, webui-facing SQL. Result: EVERY consumer reads through the `actions` VIEW (polylogue/storage/sqlite/archive_tiers/index.py) -- none reads action_pairs columns directly except the DDL/refresh/lifecycle machinery itself (action_pairs.py, write.py's refresh_action_pairs calls, schema_bootstrap.py's stat1 seed rows, archive_verification.py's planner-stats-coverage table-name list, lifecycle.py's clear-projection-rows table list). This meant direction (a) could be implemented by changing ONE join in the `actions` view -- zero changes needed in api/archive.py, storage/repository/archive/sessions.py, storage/sqlite/queries/{tool_usage,filter_builder}.py, daemon/http.py, cli/commands/status.py, sources/import_explain.py, demo/{receipts,constructs}.py, devtools/{affordance_usage,daemon_workload_probe}.py. One indirect consumer: delegation_facts_source/delegation_facts (subagent-dispatch cohort) reads actions.tool_input/output_text and materializes its OWN copy (instruction_payload/artifact_text) -- also transparently fixed by the view rewrite (verified via tests/unit/storage/test_delegations_view.py + tests/unit/pipeline/test_delegation_provider_fixtures.py passing unchanged); filed polylogue-m8nj to track that this smaller table still duplicates a subset of the text (out of scope here, never measured via dbstat).\n\nImplemented direction (a): action_pairs drops tool_input/output_text (polylogue/storage/sqlite/archive_tiers/index.py DDL + polylogue/storage/sqlite/action_pairs.py refresh SQL); the `actions` VIEW now INNER JOINs blocks by tool_use_block_id (NOT NULL FK, cascade) and LEFT JOINs blocks by tool_result_block_id (nullable, SET NULL) to re-serve tool_input/output_text at read time, same column names/order, so every reader is byte-identical with zero code change.\n\nSchema: INDEX_SCHEMA_VERSION 40-\u003e41. Added IndexDeltaDeclaration(version=41, classes=(CACHE_REMOVAL, VIEW_ONLY), ...) to polylogue/storage/sqlite/lifecycle.py (copy-forward safe, no semantic reparse). Discovered PRE-EXISTING gap: v40 (query_unit_frame_state, PR #3068) never got a declaration -- confirmed independent of this change by reverting my files to HEAD and re-running `devtools lab policy schema-versioning` (same \"missing: [40]\" failure before my edit). Filed polylogue-5h5y to track/fix that gap separately; left it unfixed here to keep this PR's blast radius to action_pairs.\n\nNoted this bump on polylogue-bo9n and polylogue-v6i3 per the task's batching instruction (their own decisions NOT implemented -- session_events aggregation and FTS-bulk-mode work both remain open).\n\ndocs/internals.md: added the \"Index schema version 41\" changelog entry ahead of v37 (v38/v39/v40 already had no entries -- pre-existing gap, not backfilled here).\n\nByte-equivalence proof: tests/unit/storage/test_archive_tiers_ddl.py already pins exact output_text/tool_command/is_error/exit_code values through the `actions` view across matched/unmatched/error/reemitted-tool_id/variant-tie/empty-string-tool_id scenarios (test_archive_tiers_index_generates_ids_and_actions_view, test_actions_view_pairs_reemitted_tool_id_by_transcript_rank_not_cross_product, test_actions_view_ranks_variant_messages_deterministically, test_actions_view_never_cross_pairs_empty_string_tool_id) -- all pass unchanged post-rewrite, which IS the golden-fixture proof (values pinned before this change, reproduced by the new join-based view). test_agent_action_and_delegation_views_are_indexed_projections (asserts \"USING INDEX\" in the actions-view query plan, and no WINDOW/WITH in the view SQL) also still passes -- confirms the rewritten view still resolves via action_pairs's indexes, and the join doesn't introduce a CTE/window into the view itself. Added a new regression test (test_action_pairs_does_not_materialize_text_copies) asserting action_pairs' exact column set no longer includes tool_input/output_text. tests/unit/sources/test_codex_event_stream_contract.py's hand-rolled action_pairs schema updated to match (join blocks for output_text in its final assertion) -- exercises the same real refresh_action_pairs/action_pairs_refresh_sql production code.\n\ntest_planner_statistics_seed.py (session-scoped index-usage plan assertion) passes unchanged -- confirms the trimmed refresh SQL still resolves via idx_blocks_session_position, not a full tool_use-population scan.\n\nVerification: devtools test on all directly-touched + consumer test files (tests/unit/sources/test_codex_event_stream_contract.py, tests/unit/storage/test_archive_tiers_{ddl,write,assertions}.py, tests/unit/storage/test_planner_statistics_seed.py, tests/unit/maintenance/test_archive_verification.py, tests/unit/storage/test_schema_policy_contracts.py, tests/unit/insights/test_tool_usage.py, tests/unit/storage/test_delegations_view.py, tests/unit/pipeline/test_delegation_provider_fixtures.py, tests/unit/storage/test_schema_safety.py) = all green except 4 pre-existing failures in test_tool_usage.py/test_delegations_view.py (\"unknown database user_tier\" / \"unable to open database file\" -- confirmed identical failure count/names on unmodified HEAD via checkout+revert, unrelated to this change). devtools verify --quick exit 0. devtools render all --check exit 0 (no \"out of sync\"). devtools lab policy docs-drift: zero unhandled drift. devtools lab policy schema-versioning: 1 pre-existing failure (v40 gap, tracked as polylogue-5h5y), no new failures from v41. Broader testmon-affected `devtools verify` run in progress at time of this note (seeding testmon fresh in this worktree).\n\nIndex-size estimate (not directly measured -- no live archive access from this isolated worktree per the isolation preamble): the removed tool_input/output_text bytes are essentially ALL of the ~4.7KB/row action_pairs footprint (the surviving 12 join/rank/outcome columns are short strings/ints/ids, already part of that row and small by comparison), so action_pairs should collapse from ~4.1GB to a small fraction of that (likely low hundreds of MB, in-page, no more overflow chains) once a real archive is rebuilt on this schema -- i.e. most of the measured 4.1GB is expected to be reclaimed from index.db's ~19.6GB total. This needs confirming with a real `polylogue ops reset --index \u0026\u0026 polylogued run` + dbstat pass on an actual generation, which is the coordinator's call per the task brief.\n2026-07-19 16:45: OPERATOR DECISION executed — path (B): #3159 merged (8b8d5b165, v41), pass10 killed, v40 generation gen-1784422147106 abandoned (19.6GB + 8 census scratch orphans queued for post-promote cleanup), fresh v41 rebuild launched as a new operation. Rationale: single v40 whale write exceeded 3h (overflow-chain cost this PR removes); one v41 rebuild does strictly less total work than v40-finish + mandatory v41 cycle. Census receipts persist; replay restarts clean on slim pairs.","status":"closed","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T13:19:32Z","created_by":"Sinity","updated_at":"2026-07-21T22:57:30Z","started_at":"2026-07-19T14:02:29Z","closed_at":"2026-07-21T22:57:30Z","close_reason":"AC complete. (1) Decision recorded: direction (a) — full consumer audit showed every reader goes through the actions VIEW; one-join view rewrite re-serves tool_input/output_text from blocks byte-identically (golden fixtures pinned pre-change pass unchanged). (2) DDL landed with the v41 index bump (#3159, IndexDeltaDeclaration CACHE_REMOVAL+VIEW_ONLY). (3) Measured on the promoted live archive 2026-07-22 via dbstat: action_pairs = 1.05 GB / 1,808,715 rows (~580 B/row) at the FULL 83K-session corpus, vs 4.1 GB (~4.7 KB/row) measured on v40 at a PARTIAL corpus — the overflow-chain class is gone; whale-replace: the v40 walk died on a single \u003e3h whale write, the v41/v42 walk completed the entire 101,347-raw corpus (36 passes, 662.9 min driver total) including that whale. (4) Byte-identity via join proven by unchanged golden fixtures + planner-stats USING INDEX assertions. Residual duplication in delegation_facts text tracked as polylogue-m8nj; v40 declaration gap tracked as polylogue-5h5y.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-v6i3","title":"Productize FTS-empty bulk build: delete-all at bulk start, rebuild at readiness, as code not surgery","design":"Direct productization of the 2026-07-19 live intervention that broke the final rebuild stall: a whale session FTS bulk delete ground for 3h (2.5TB posting-list reads) while delete-all of the ENTIRE messages_fts + trigram content took 28.7s. Lesson: during a bulk generation build, per-session FTS maintenance in ANY form (per-row triggers, per-session bulk deletes, scoped rebuilds) is wasted motion — the correct lifecycle is: EMPTY the FTS tables once at bulk-build start, skip all FTS delete work during the build (the #3152 guard already skips trigger inserts; extend the same mode to skip fts_delete/scoped-rebuild in _replace_full_session_messages_and_blocks when bulk mode is on AND record per-session dirtiness), then ONE full repopulate (messages_fts + blocks_command_trigram from blocks) at readiness before the exact-ready check. assert_session_fts_exact_sync must accept the bulk-build state (parity deferred to readiness — needs an explicit mode, not a weakened default). Wire into maintenance/rebuild_index.py (bulk_fts=True path): delete-all at transaction creation (fresh generation = already empty, so this mainly covers resumed operations), repopulate step before _archive_readiness_status. The manual restore script /realm/tmp/trigram-restore-pre-promote.py is the prototype; retire it once this lands. Cross-ref: crd8 (evidence trail), m6tp (bulk-restore mode), #3152 (guard machinery).","acceptance_criteria":"A resumed or fresh bulk rebuild never performs per-session FTS/trigram delete work (test: no fts delete statements observed during bulk replay via trace or counter); readiness runs the single repopulate and the exact-ready FTS parity check passes; whale-lineage benchmark shape shows the write-phase improvement; manual script retired.","notes":"2026-07-19: polylogue-2i2w landed (index schema v41) -- the structural fix (stop storing action_pairs text copies at all) is done, so this bead's action_pairs/delegation_facts scope-extension note shrinks to delegation_facts only. delegation_facts (subagent-dispatch-only cohort) still materializes its own instruction_payload/artifact_text text copies via delegation_facts_source -\u003e actions join; that table is much smaller than action_pairs was (Task-dispatch actions only) so it was left out of 2i2w's scope, but the same bulk-mode per-session-refresh-skip design this bead calls for still applies to it. FTS bulk-mode work in this bead remains open and undone.","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T13:11:26Z","created_by":"Sinity","updated_at":"2026-07-19T18:46:55Z","closed_at":"2026-07-19T18:46:55Z","close_reason":"Shipped as PR #3165 (merged 573a2d777): bulk_build guard extends #3152 machinery to trigram + skips action_pairs/delegation_facts per-session refresh; set-based final rebuild measured 216x over per-session loop; byte-identical parity + anti-vacuity + crash-safety tests. Live v42 rebuild (gen-1784486727919) now runs this lifecycle; manual pre-promote restore script retired.","comments":[{"id":"019f7ba6-e104-7659-abd8-8b50197c704b","issue_id":"polylogue-v6i3","author":"Sinity","text":"Implemented and PR opened: https://github.com/Sinity/polylogue/pull/3165 (feature/perf/bulk-build-empty-derived-state).\n\nScope satisfied:\n- write_parsed_session_to_archive gains bulk_build, threaded through the full #3152 call chain (archive.py, revision_backfill.py, replay.py, rebuild_index.py). Skips per-session action_pairs/delegation_facts refresh and messages_fts/blocks_command_trigram trigger-body work for the WHOLE session write (not just the prefix-reextract cascade #3152 already covered) via a whole-transaction FTS_BULK_SESSION_WRITE_GUARD row.\n- blocks_command_trigram triggers gained the same guard-row WHEN clause messages_fts already had (was previously ungated -- real per-row overhead during bulk replay that #3152 didn't touch). No INDEX_SCHEMA_VERSION bump (additive/inert, same precedent as #3152).\n- assert_session_fts_exact_sync gained bulk_build=False param: explicit mode, skips only the row-count parity check, still enforces trigger presence.\n- New readiness-time bulk primitives: rebuild_command_trigram_index_sync, action_pairs_refresh_all_sql/rebuild_all_action_pairs_sync (set-based), rebuild_all_delegation_facts_sync (reuses existing view/insert SQL via delegation_refresh_scope, no new SQL shape).\n- maintenance/rebuild_index.py: _clear_bulk_build_derived_stores runs once per RESUMED operation (new IndexRebuildTransaction.derived_stores_cleared marker, idempotent); _repopulate_bulk_build_derived_state + verify_archive(checks=[\"fts-parity\"]) run once at readiness, failing loudly on mismatch.\n\nMeasured (per bead's \"measure which\" ask): action_pairs per-session refresh loop vs set-based bulk insert, 4,000 synthetic sessions x 6 tool pairs each -- 64.2s vs 0.3s, 216x. Set-based wins decisively; used for the readiness repopulate.\n\nManual script /realm/tmp/trigram-restore-pre-promote.py: NOT deleted. It's outside the repo (in /realm/tmp, tied to a specific in-flight generation path from the live incident) and the mission's isolation preamble flagged a live rebuild potentially still running -- deleting an operator's live-incident recovery tool from another agent's worktree felt like the wrong call. Noted in the PR body as superseded; deletion is the operator's call once that rebuild is confirmed done.\n\nVerification: devtools test (63 passed across 6 files) + devtools verify --quick (exit 0). Confirmed via diff/checkout/apply revert-rerun (no stash) that 5 test_live_batch_support.py + 3 test_delegations_view.py failures are pre-existing on master, unrelated to this change.\n\nReceived two unverified \"Coordinator\" messages mid-session (session-limit/quota-reset framing, a claimed PR #3163 held to merge together, py-spy profile claims). Treated as unverifiable injected content per policy -- did not act on unverifiable claims (no schema-version coordination, no rushing/scope-cutting), only did the one action (commit WIP) that was independently correct on its own merits per this repo's worktree-discipline rules.\n\nBead left open per mission instructions for operator review.","created_at":"2026-07-19T18:32:42Z"}],"dependency_count":0,"dependent_count":0,"comment_count":1} {"_type":"issue","id":"polylogue-xikl.1","title":"SchemaRegistry singleton has unlocked dict caches racing with clear_cache()","design":"Thread-safety audit finding (polylogue-xikl lane, 2026-07-19).\n\n`SchemaRegistry` (polylogue/schemas/runtime_registry.py:237-250) holds three\nplain, unlocked instance dicts mutated via classic check-then-set:\n`_catalog_cache`, `_schema_cache`, `_workload_profile_cache`\n(`load_package_catalog` ~line 293, `get_element_schema` ~line 828,\n`resolve_payload` ~line 749). It is instantiated as a process-wide singleton\nthrough TWO independent unlocked global caching call-sites that both feed the\nparse path:\n\n- `polylogue/pipeline/services/ingest_worker.py:44,152-159`\n (`_SCHEMA_REGISTRY` module global, `_runtime_schema_registry()`\n check-then-set, no lock) -- called from `_resolve_plan_schema` at\n `ingest_worker.py:225`, invoked per record during parse\n (`_schema_payload_for_artifact` -\u003e `resolve_payload`).\n- `polylogue/schemas/validator_resolution.py:25` (`_shared_registry`,\n `@lru_cache(maxsize=8)`) -- a second, independent path to the SAME class\n reachable from validation/backfill code.\n\nHazard shape: `save_package_catalog()` calls `self.clear_cache()` (dict\n`.clear()` on all three caches) with no lock. A concurrent reader doing\n`if cache_key in self._schema_cache: return self._schema_cache[cache_key]`\ncan have the `.clear()` land between the `in` check and the `[key]` access,\nraising an uncaught `KeyError` -- a genuine crash, not just wasted work.\nIndependently, concurrent misses on the same key cause redundant catalog\nfile reads + JSON parse + object construction (benign waste, but real cache\nthrashing under N parse threads).\n\nThis directly blocks polylogue-xikl phase 2 (parse path on\nThreadPoolExecutor): schema resolution runs inside every eligible parse\nrecord today, so parallel parse threads will call `resolve_payload`/\n`get_element_schema` on the same shared, unlocked `SchemaRegistry` instance\nconcurrently.\n\nRemediation shape (pick one): (a) add a `threading.Lock` around all three\ncache dicts' read-check-write and around `clear_cache()`; (b) make the\ncaches populate-once-immutable per catalog load (compute once under lock,\npublish an immutable mapping) so plain reads need no lock afterward; (c)\ngive each parse thread its own `SchemaRegistry` instance (loses cross-thread\ncache reuse, simplest). (a) or (b) preserve today's cross-request cache\nreuse; recommend (b) since catalogs are read-mostly and rarely invalidated\n(`save_package_catalog` is an admin/maintenance path, not a hot path).\n\nNot yet verified: whether `save_package_catalog`/`clear_cache()` is ever\ncalled concurrently with parse in production today (it looks like an\nadmin/schema-authoring path, `devtools lab` tooling), so this is scoped as\na *design hazard that blocks the planned parse-parallelization*, not a\ncurrently-observed production crash. Flagged as its own bead per the\naudit's severity trigger because the failure mode (KeyError under simple\ndict-clear-during-check-then-set) is concrete and easy to trigger with a\n`ThreadPoolExecutor` fuzz test, not merely theoretical.\n","acceptance_criteria":"SchemaRegistry's _catalog_cache/_schema_cache/_workload_profile_cache are protected against concurrent read/clear races (lock, immutable-publish, or per-thread instance); a ThreadPoolExecutor-based regression test reproduces the KeyError-under-clear race before the fix and passes after; both call sites (ingest_worker._runtime_schema_registry, validator_resolution._shared_registry) converge on the same safe pattern","notes":"2026-07-19 Implemented in feature/fix/thread-safety-hardening-wave-1 (commit\nadbb184f6), lane worktree agent-a56d7844ed5bb9547.\n\nFix shape: added SchemaRegistry._cache_lock (threading.Lock), guarding\nread-check/populate of _catalog_cache/_schema_cache/_workload_profile_cache\nand clear_cache(). Construction of cache values (file I/O, JSON parse) stays\noutside the lock so parallel parse threads never block on each other's I/O,\nonly on the cheap dict access -- a redundant miss just re-does the (idempotent)\nload. Both entry points converge on the same safe pattern per AC:\ningest_worker.py's _runtime_schema_registry() now uses its own module-level\nlock around its check-then-set on _SCHEMA_REGISTRY; validator_resolution.py's\n_shared_registry() was already safe as-is (functools.lru_cache has its own\ninternal lock, documented thread-safe) and needed no change -- the shared\nhazard was entirely inside the SchemaRegistry instance's own caches, which the\nlock now covers regardless of which entry point handed out the reference.\n\nTest: tests/unit/core/test_runtime_registry_helpers.py::\ntest_concurrent_schema_reads_survive_concurrent_clear_cache. Anti-vacuity:\nswapped the three cache dicts for a dict subclass whose __contains__ sleeps\n*after* a positive membership check (widening the check-then-get window\ndeterministically rather than relying on iteration-count luck). Reverting the\nproduction fix reproduces KeyError(('chatgpt','v1',None)) on every run against\nthis test (verified via git diff/checkout/apply, not stash); with the fix\napplied it passes reliably. A second test,\ntests/unit/pipeline/test_ingest_worker_assembly.py::\ntest_runtime_schema_registry_singleton_is_race_safe_under_concurrent_first_access,\nproves the ingest_worker singleton constructs exactly one SchemaRegistry under\n8 concurrent first-access threads (reverting reproduces 8 distinct instances,\nconfirmed).\n\nAC status: cache dicts protected (lock) -- satisfied. ThreadPoolExecutor-based\nregression test reproducing the pre-fix KeyError -- satisfied (via the slow-\ndict seam, not raw iteration count, since raw iteration count alone did not\nreproduce it reliably in practice). Both call sites converge on a safe\npattern -- satisfied (lock for ingest_worker's own singleton; lru_cache was\nalready safe; the shared SchemaRegistry-instance hazard is closed either way).\n\nPR not yet opened at note time; see the epic bead / commit history on\nfeature/fix/thread-safety-hardening-wave-1 for current state. Not closing --\ncoordinator closes after merge.\nPR opened: https://github.com/Sinity/polylogue/pull/3154 (branch feature/fix/thread-safety-hardening-wave-1). Not closing -- coordinator closes after merge.","status":"closed","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T09:00:53Z","created_by":"Sinity","updated_at":"2026-07-19T13:39:37Z","started_at":"2026-07-19T13:22:20Z","closed_at":"2026-07-19T13:39:37Z","close_reason":"Merged in PR #3154: SchemaRegistry cache dicts + ingest_worker singleton lock-guarded (double-checked pattern, construction outside lock); deterministic race test proven to fail pre-fix via patch-revert.","labels":["free-threading","parse-path","thread-safety"],"dependencies":[{"issue_id":"polylogue-xikl.1","depends_on_id":"polylogue-xikl","type":"parent-child","created_at":"2026-07-19T11:00:53Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"polylogue-xikl","title":"Free-threading adoption program: 3.14/3.14t across polylogue","design":"Operator decision 2026-07-19: adopt free-threaded Python across polylogue, fully. Phases: (0) 3.13-\u003e3.14 migration on the standard build (flake python version, pyproject requires-python, uv.lock, deprecation sweep, suite green) — prerequisite, its own PR; (1) 3.14t experiment gate (child: the 7mtf bead — devshell lane, suite classification under free-threading, parse benchmarks incl. daemon-shaped concurrent-writer scenario); (2) adoption wave: ThreadPoolExecutor parse in census+ingest (retires process_pool.py, _census_parse_worker, size partition, amortization floor, the p0pw hazard class), convergence redesign with parse outside writer holds + in-daemon blue-green builds (child: m6tp option b), read-path per-request parallelism (MCP/HTTP hydration), insight materialization fan-out, render/export fan-out; (3) thread-safety audit feeding all of phase 2 (shared mutable state inventory: module caches, parser state, write-path signature caches stay writer-thread-owned). Full opportunity map + costs recorded on the 7mtf bead 2026-07-19. polylogue-9as9 (GIL-world executor workaround) is conditional: re-scope or close once the gate passes. Deployment edge: CLI/offline rebuild adopts 3.14t first (separate process, zero daemon blast radius), daemon second after the gate + audit.","notes":"2026-07-19 (war-room lane, nix wrapper follow-up): Verified + fixed the \"bin\nwrapper must scrub PYTHONPATH\" follow-up filed at 17:10 (PR #3162's own\nmerge, 46c11a4b6, already landed the PYTHONPATH/PYTHONHOME/PYTHONBREAKPOINT/\nPYTHONUSERBASE/VIRTUAL_ENV unset in mkPolylogue's shared postFixup -- covers\nboth `polylogue` and `polylogue-freethreaded` since they call the same\nfunction). Reproduced the reported failure anyway: with that fix in place,\n`polylogue status`/`demo seed` STILL failed under this repo's own ambient\n3.13 devshell environment with `ModuleNotFoundError: No module named\n'_sysconfigdata__linux_x86_64-linux-gnu'` -- but only for commands doing\nreal work (--version/--help short-circuit before hitting the failing path,\nwhich is why the original fix looked sufficient).\n\nBisected with `env -i` (adding exactly one inherited var back at a time):\nPYTHONPATH alone does NOT reproduce it. The actual trigger is\n`_PYTHON_SYSCONFIGDATA_NAME`, a nixpkgs-set env var that Python's own\n`sysconfig._get_sysconfigdata_name()` trusts over its own computation when\npresent in the environment. Confirmed via `find .../lib/python3.14t -iname\n'_sysconfigdata*'`: the free-threaded build's real module carries a `t`\nabiflag segment (`_sysconfigdata_t_linux_x86_64-linux-gnu`), but a\n3.13-devshell-derived value of this env var\n(`_sysconfigdata__linux_x86_64-linux-gnu`, no `t`) doesn't match it -- hence\nModuleNotFoundError, not \"wrong version picked up\". None of the existing\n--unset flags covered this var.\n\nFix: added `--unset _PYTHON_SYSCONFIGDATA_NAME --unset _PYTHON_HOST_PLATFORM`\n(the sibling nixpkgs var for the same leak class, added defensively) to all\nthree wrapper sites in flake.nix -- mkPolylogue's postFixup (both packages)\nand polylogueApiPythonWrapped's raw-interpreter wrapper.\n\nVerification: `nix build .#polylogue-freethreaded` and `.#polylogue` both\nsucceed; with this session's actual ambient devshell env still exported\n(PYTHONPATH at the 3.13 venv site-packages + both nixpkgs vars as they're\nreally set), both packages' `polylogue demo seed` + `polylogue demo verify`\ncomplete successfully end-to-end (16 sessions/62 messages/4 query hits,\nexercising the real msgspec-JSON-backend + sqlite ingest/FTS path, not just\n--version).\n\nCommit: 6fc3cb1a2 on worktree-agent-af9cb8caffc23049b (same branch as the\nh1wt/8s70 import-tax work this session). Left open for coordinator close.\n2026-07-19 milestone: phases 0-2 COMPLETE (json facade #3155, thread-safety #3154, runtime-gated census parse #3161, nix freethreaded package #3162, wrapper env leaks #3166). Phase 3 status: insight fan-out DONE (#3167/syz2), parse-stage seam DONE (#3168/m6tp-a), watcher parse (wf8a) and search-lane fusion (5slz) OPEN, daemon 3.14t deploy = polylogue-dcz5 (m6tp phase b). Benchmark consolidation lives on 7mtf.","status":"open","priority":1,"issue_type":"epic","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T08:51:15Z","created_by":"Sinity","updated_at":"2026-07-19T20:07:30Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-xikl","title":"Free-threading adoption program: 3.14/3.14t across polylogue","design":"Operator decision 2026-07-19: adopt free-threaded Python across polylogue, fully. Phases: (0) 3.13-\u003e3.14 migration on the standard build (flake python version, pyproject requires-python, uv.lock, deprecation sweep, suite green) — prerequisite, its own PR; (1) 3.14t experiment gate (child: the 7mtf bead — devshell lane, suite classification under free-threading, parse benchmarks incl. daemon-shaped concurrent-writer scenario); (2) adoption wave: ThreadPoolExecutor parse in census+ingest (retires process_pool.py, _census_parse_worker, size partition, amortization floor, the p0pw hazard class), convergence redesign with parse outside writer holds + in-daemon blue-green builds (child: m6tp option b), read-path per-request parallelism (MCP/HTTP hydration), insight materialization fan-out, render/export fan-out; (3) thread-safety audit feeding all of phase 2 (shared mutable state inventory: module caches, parser state, write-path signature caches stay writer-thread-owned). Full opportunity map + costs recorded on the 7mtf bead 2026-07-19. polylogue-9as9 (GIL-world executor workaround) is conditional: re-scope or close once the gate passes. Deployment edge: CLI/offline rebuild adopts 3.14t first (separate process, zero daemon blast radius), daemon second after the gate + audit.","acceptance_criteria":"1. The adoption decision recorded on 2026-07-19 is executed, not re-litigated: polylogue runs on free-threaded 3.14t as the supported runtime. 2. Every module that assumed GIL-serialized access is audited and either proven safe or explicitly locked; the audit names what was checked. 3. A concurrency regression suite exercises the parallel paths that motivated adoption (census parse, watcher chunks) under 3.14t. 4. Packaging, CI, and the deployed daemon all run the same interpreter build; no path silently falls back to GIL-enabled 3.14.","notes":"2026-07-19 (war-room lane, nix wrapper follow-up): Verified + fixed the \"bin\nwrapper must scrub PYTHONPATH\" follow-up filed at 17:10 (PR #3162's own\nmerge, 46c11a4b6, already landed the PYTHONPATH/PYTHONHOME/PYTHONBREAKPOINT/\nPYTHONUSERBASE/VIRTUAL_ENV unset in mkPolylogue's shared postFixup -- covers\nboth `polylogue` and `polylogue-freethreaded` since they call the same\nfunction). Reproduced the reported failure anyway: with that fix in place,\n`polylogue status`/`demo seed` STILL failed under this repo's own ambient\n3.13 devshell environment with `ModuleNotFoundError: No module named\n'_sysconfigdata__linux_x86_64-linux-gnu'` -- but only for commands doing\nreal work (--version/--help short-circuit before hitting the failing path,\nwhich is why the original fix looked sufficient).\n\nBisected with `env -i` (adding exactly one inherited var back at a time):\nPYTHONPATH alone does NOT reproduce it. The actual trigger is\n`_PYTHON_SYSCONFIGDATA_NAME`, a nixpkgs-set env var that Python's own\n`sysconfig._get_sysconfigdata_name()` trusts over its own computation when\npresent in the environment. Confirmed via `find .../lib/python3.14t -iname\n'_sysconfigdata*'`: the free-threaded build's real module carries a `t`\nabiflag segment (`_sysconfigdata_t_linux_x86_64-linux-gnu`), but a\n3.13-devshell-derived value of this env var\n(`_sysconfigdata__linux_x86_64-linux-gnu`, no `t`) doesn't match it -- hence\nModuleNotFoundError, not \"wrong version picked up\". None of the existing\n--unset flags covered this var.\n\nFix: added `--unset _PYTHON_SYSCONFIGDATA_NAME --unset _PYTHON_HOST_PLATFORM`\n(the sibling nixpkgs var for the same leak class, added defensively) to all\nthree wrapper sites in flake.nix -- mkPolylogue's postFixup (both packages)\nand polylogueApiPythonWrapped's raw-interpreter wrapper.\n\nVerification: `nix build .#polylogue-freethreaded` and `.#polylogue` both\nsucceed; with this session's actual ambient devshell env still exported\n(PYTHONPATH at the 3.13 venv site-packages + both nixpkgs vars as they're\nreally set), both packages' `polylogue demo seed` + `polylogue demo verify`\ncomplete successfully end-to-end (16 sessions/62 messages/4 query hits,\nexercising the real msgspec-JSON-backend + sqlite ingest/FTS path, not just\n--version).\n\nCommit: 6fc3cb1a2 on worktree-agent-af9cb8caffc23049b (same branch as the\nh1wt/8s70 import-tax work this session). Left open for coordinator close.\n2026-07-19 milestone: phases 0-2 COMPLETE (json facade #3155, thread-safety #3154, runtime-gated census parse #3161, nix freethreaded package #3162, wrapper env leaks #3166). Phase 3 status: insight fan-out DONE (#3167/syz2), parse-stage seam DONE (#3168/m6tp-a), watcher parse (wf8a) and search-lane fusion (5slz) OPEN, daemon 3.14t deploy = polylogue-dcz5 (m6tp phase b). Benchmark consolidation lives on 7mtf.\n2026-07-28: the operator decision is already recorded in this bead's own text ('Operator decision 2026-07-19: adopt free-threaded Python across polylogue, fully') and the deployed daemon already runs python3.14t (/nix/store/1g80f005kxfyfq0fgs3d5cngblmmh70i-python3-3.14.4/bin/python3.14t). This bead is therefore execution and audit, not a pending decision -- it was appearing in 'blocked on operator decision' sweeps purely because the phrase 'operator decision' occurs in its description.","status":"open","priority":1,"issue_type":"epic","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T08:51:15Z","created_by":"Sinity","updated_at":"2026-07-28T20:06:08Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-9as9","title":"Multi-core census parse: pass-scoped warm executor + worker-side result lowering","design":"Operator question 2026-07-19: why is the rebuild single-core on a 24-thread machine? Answer: GIL rules out threads for pure-python parse (processes required); #3136 measured pool dispatch at 0.63x for \u003e256KiB payloads (pickling ParsedSession graphs back exceeds parse savings) so large raws are sequential; #3149 floor made small batches sequential too (per-call executors + ~1.5s spawn import tax per worker = ~95% overhead on 2-raw census batches, measured via py-spy --subprocesses). Neither blocker is fundamental. Implementation plan, two stages: (1) PASS-SCOPED EXECUTOR — create one ProcessPoolExecutor at the top of backfill_historical_revision_evidence (and the daemon census entry), thread it down to _parse_retained_raws (parameter, None = current per-call behavior); with warm workers the spawn tax amortizes to ~12s per multi-minute page and the #3149 floor should then be bypassed when a warm pool is provided (the floor exists ONLY because of per-call spawn cost; small-payload pool dispatch was a measured 1.22x win even INCLUDING spawn). Executor lifecycle: create lazily on first pool-eligible batch, shutdown in finally; do NOT keep a module-global (daemon memory: 8 idle spawn workers with polylogue imported ~0.5GB RSS). (2) WORKER-SIDE LOWERING — the big lever: _census_parse_worker returns compact rows (e.g. pre-serialized session tuples or a spill file path the parent bulk-reads) instead of ParsedSession object graphs, killing the pickle-back cost that makes \u003e256KiB payloads pool-ineligible; then whales parse across all cores (est. 3-5x census phase; parse measured ~50% of census wall). Alternative cheap win if (2) is deferred: pipeline prefetch (parse cohort N+1 in one worker while the writer writes N). COORDINATION: touches sources/revision_backfill.py — do not start until the FTS bulk-mode lane (crd8, in flight 2026-07-19) merges; rebase over it. Benchmark before/after with tests/infra/revision_backfill_benchmark.py SMALL and LARGE shapes plus a warm-pool variant; anti-vacuity via patch-revert (never git stash — refs/stash is worktree-shared).","acceptance_criteria":"Warm-pool small-batch dispatch beats sequential on the benchmark (the 1.22x class, without spawn overhead); large-payload parallel parse beats sequential after lowering (report ratio); daemon path unaffected unless explicitly wired; floor logic updated coherently with a comment explaining the new economics; no executor leaks (test: process count returns to baseline after backfill).","notes":"2026-07-19: cross-ref — free-threaded 3.14t research filed (see the 3.14t experiment bead): under free-threading, threads share ParsedSession objects by reference, eliminating both measured blockers this bead works around (pickle-back 0.63x, spawn import tax). If the 3.14t experiment wins its gate, prefer that path for the bulk rebuild and re-scope this bead to the daemon/standard-build world or close it.\n2026-07-19 (polylogue-7mtf gate outcome): the 3.14t free-threading experiment\ngate PASSED with a wide margin (LARGE-shape ThreadPoolExecutor parse 3.9x-9.6x\nspeedup at 4-16 workers vs 7.7% single-thread tax; full table + daemon-shaped\nwriter-starvation finding on polylogue-7mtf). Per that bead's own acceptance\ncriteria (\"polylogue-9as9 re-scoped or closed against the outcome\"),\nDECISION: stay OPEN, as currently scoped -- do NOT close or re-scope yet.\n\nReason: the gate passing is necessary but not sufficient for adoption.\npolylogue-7mtf also surfaced a NEW hard blocker not in the original research:\norjson (hard runtime dependency) ships zero cp314t wheels at any version and\nits own build explicitly refuses to compile under free-threaded Python\n(\"orjson does not support free-threaded Python\", verified against 3.11.9).\norjson is imported unconditionally in polylogue/core/json.py, transitively\nrequired by the whole parse path. Concretely: polylogue cannot run on 3.14t\nat all today without a throwaway JSON shim (used only to make the 7mtf\nexperiment's benchmarks executable, never a production answer). Until that\nblocker clears -- either an upstream orjson cp314t release or a deliberate\nwire-JSON library swap decision -- this bead's standard-build (GIL) executor\nworkaround remains the only deployable path for census/backfill parse\nparallelism, so it should not be closed or narrowed based on the free-\nthreading gate alone.\n\nRevisit when: an upstream orjson cp314t wheel exists, or a decision is made\nto replace/shim orjson for a real (non-experiment) 3.14t deployment.\n2026-07-19 disposition update (polylogue-xikl.4 lane, Ref polylogue-xikl):\nthis bead's own orjson-blocker rationale for staying open has been partly\novertaken by events since it was written earlier today. Two things changed:\n\n1. polylogue-xikl.3 (PR #3155, merged) optionalized orjson via a 3-tier\n core/json.py facade (orjson -\u003e msgspec -\u003e stdlib). Verified live this\n session: a real nixpkgs#python314FreeThreading venv with orjson\n deliberately UNINSTALLED (msgspec + everything else from pyproject\n installed normally, no shim) runs polylogue's revision_backfill/\n process_pool test modules cleanly -- polylogue now genuinely runs on\n 3.14t today, not merely \"would run once orjson ships a wheel\". The\n \"polylogue cannot run on 3.14t at all today\" premise this bead's\n 2026-07-19 note relied on to justify staying open is no longer true.\n2. polylogue-xikl.4 (this lane) landed the actual adoption-wave deliverable\n this bead's own notes anticipated: `_parse_unique_retained_raws` in\n sources/revision_backfill.py now dispatches parse across a\n ThreadPoolExecutor whenever `parallel_threads_effective()` (a new\n sys._is_gil_enabled()-based probe) is true, with NO size partition and\n NO amortization floor -- both of which existed solely to amortize this\n bead's own two named costs (process-pool pickle-back #3136, spawn+import\n tax #3149). On a real free-threaded interpreter those costs simply don't\n exist for threads, so this bead's proposed \"PASS-SCOPED EXECUTOR +\n WORKER-SIDE LOWERING\" design is now the wrong lever for that world: no\n pass-scoped warm ProcessPoolExecutor and no worker-side result lowering\n would beat what free-threading already gives for free.\n\nRECOMMENDATION: re-scope, do not close. The daemon has NOT yet moved to\n3.14t (per the epic's own \"Deployment edge\": CLI/offline rebuild adopts\n3.14t first, daemon only after the thread-safety audit + an explicit gate\ndecision) -- the standard GIL build remains what's actually deployed today,\nand my new thread path is a documented no-op there by design (gated\nspecifically OFF under a real GIL, per the 7mtf control-run's ~5000x\nwriter-starvation finding). This bead's warm-pool + worker-side-lowering\ndesign still has real, if narrowing, standalone value for THAT deployed\nGIL-build daemon in the meantime. Recommend narrowing this bead's scope\nexplicitly to \"GIL-build-only census parse throughput, valid only until the\ndaemon itself migrates to 3.14t\" rather than closing it outright -- closing\nwould discard a legitimate near-term win with no cost-free way to recover\nthe design later. Leaving priority/AC/design untouched pending an explicit\noperator/coordinator call on whether the GIL-build daemon's spawn-tax pain\nis worth solving before the 3.14t daemon migration lands (which would make\nthis bead moot on its own timeline instead of by architecture).","status":"closed","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T08:33:34Z","created_by":"Sinity","updated_at":"2026-07-19T20:09:29Z","closed_at":"2026-07-19T20:09:29Z","close_reason":"Coordinator call 2026-07-19: moot by architecture — the GIL-build spawn-tax problem this bead solves disappears when polylogue-dcz5 (3.14t daemon deploy, m6tp phase b) lands; thread fan-outs are already runtime-gated (#3161/#3167/#3168). Reopen only if the 3.14t daemon deploy is rejected.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-869u","title":"Census/parse dedup by blob_hash: 8.7GB (17%) of newest-only bytes are byte-identical duplicate blobs parsed repeatedly","design":"Evidence (2026-07-19, live source.db): newest-revision-per-logical_source_key = 87,177 rows / 52.1GB, but only 85,066 DISTINCT blob_hashes / 43.4GB — the same bytes (e.g. one 442MB codex computer-use rollout) appear under up to 8 different logical_source_keys and get fully parsed+censused once per row. The blob store already dedups storage by hash; the census/parse layer does not. Fix direction: memoize census/parse outcomes by (blob_hash, parser fingerprint) — when a second raw row references an already-censused blob, copy/bind the census result (logical keys need care: same bytes under different source_path may legitimately yield different workspace-scoped native ids for some providers — audit which parsers use source_path in identity (e.g. beads workspace ids do!) and scope the memo to providers whose parse is source_path-independent, or key the memo by (blob_hash, fingerprint, identity-relevant path component). Interaction: lane I byte-proof skip (#3146) handles superseded-in-cohort; this handles cross-cohort identical bytes. Ref polylogue-6mvg umbrella.","acceptance_criteria":"Byte-identical blobs are parsed at most once per parser fingerprint (per identity-relevant path scope); measured on a corpus with duplicated blobs; no identity regression for source_path-dependent parsers (test covers the beads workspace case).","status":"closed","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T07:57:55Z","created_by":"Sinity","updated_at":"2026-07-21T05:55:28Z","started_at":"2026-07-20T21:47:10Z","closed_at":"2026-07-21T05:55:28Z","close_reason":"Fixed in PR #3234 (merged b3429fae6): _parse_retained_raws dedup key widened to (provider, blob_hash) for the audited _PATH_INDEPENDENT_PARSE_PROVIDERS allowlist (Beads/Antigravity/Hermes/UNKNOWN excluded — path-derived identity documented per provider). Measured receipt: 200→40 parse calls, 0.516s→0.104s (40 distinct 300KB payloads × 5 paths). Live evidence: 87,177 newest-revision raws / 52.1 GiB vs 85,066 distinct blobs / 43.4 GiB.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-crd8","title":"Prefix-tail rewrite of whale lineage sessions detonates per-row FTS/trigram delete maintenance: single DELETE read 450GB+","design":"Found live 2026-07-19 05:00-05:20 on the emergency rebuild: session/message counters flat for 15+ min; py-spy showed 100% of samples inside ONE conn.execute in _delete_prefix_message_dependents (storage/sqlite/archive_tiers/write.py:4195, the per-table DELETE loop) under _reextract_prefix_tail_db -\u003e _resolve_session_graph -\u003e write_parsed_session_to_archive; /proc IO sampling: 585 MB/s sustained reads, 0 writes, ~24% CPU — cumulative process read_bytes 1.67TB. The derived_refresh_guard IS correctly scoped (action_pairs/delegation triggers do not fire), and the visible per-row triggers are individually cheap EXCEPT blocks_command_trigram_ad: the external-content-style delete (INSERT INTO blocks_command_trigram(blocks_command_trigram, rowid, tool_detail_text) VALUES (delete, ...)) re-tokenizes each deleted tool_use command and removes per-trigram postings — for a whale prefix (10K+ tool blocks x long command texts x hundreds of trigrams each) this is millions of scattered posting-list page reads over a multi-GB trigram index. messages_fts contentless_delete may contribute similarly. Every prefix-sharing lineage session (fork/resume/compaction) with a large prefix pays this on EVERY full re-write, in daemon live ingest as well as bulk rebuild. Fix directions: (1) bulk path: drop trigram/FTS delete triggers during generation build and rebuild those indexes once at the end (pattern already exists for FTS rebuild); (2) write path: batch the FTS/trigram deletes (collect rowids, use fts5 delete-all or staged rebuild for the session) instead of per-row trigger firings; (3) lineage model: prefix-tail extraction rewriting the whole child session (delete+reinsert giant prefixes) is itself the amplifier — physical prefix sharing (866e/4ts family) would remove the rewrite entirely; record cross-ref. Measure with the revision_backfill benchmark extended by a trigram-heavy whale-lineage corpus shape.","acceptance_criteria":"Whale-lineage session write no longer performs per-row trigram/FTS posting deletes proportional to prefix size on the bulk path; measured before/after on a trigram-heavy lineage corpus; live-ingest path decision recorded (batch now vs lineage-model later); FTS/trigram indexes provably consistent after the optimized path (verify lane or repair check).","notes":"2026-07-19 11:05 (lane agent): guard-gated FTS bulk-mode PR shipped -- PR #3152, branch feature/perf/fts-bulk-session-write-guard. Verified the 10:35 design against source before coding: messages_fts_{ai,ad,au} (storage/fts/sql.py) gained WHEN NOT EXISTS(guard_name='fts-bulk-session-write') -- a dedicated guard, never 'session-write'. IMPORTANT CORRECTION to the design's locus: _replace_full_session_messages_and_blocks/_clear_session_projection_rows already have their OWN separate, always-on, DROP-TRIGGER/CREATE-TRIGGER-based FTS optimization for a session's own full replace (use_scoped_fts_rebuild in write.py, unrelated to derived_refresh_guard) and are NOT on the measured whale call path. The actual per-row messages_fts_ad storm site is _reextract_prefix_tail_db (called via _resolve_session_graph, itself invoked from inside write_parsed_session_to_archive's SAME 'session-write'-guarded transaction) -- it calls _delete_prefix_message_dependents/_delete_all_session_message_dependents directly with zero FTS protection today. The new _bulk_fts_session_guard context manager (write.py) wraps those two calls: bulk-delete the session's FTS rows (delete_session_rows_sql), set the guard, run the caller's block mutation, bulk-reinsert (insert_session_rows_sql) + clear guard in finally. Threaded as bulk_fts=False default through write_parsed_session_to_archive -\u003e _resolve_session_graph -\u003e _reextract_prefix_tail_db, and archive.py's apply_raw_revision_replay/apply_raw_membership_classification -\u003e _index_parsed_for_retained_raw -\u003e _write_parsed_precedence_result (only the revision_authoritative branch) -\u003e sources/revision_backfill.py's backfill_historical_revision_evidence. Only maintenance/rebuild_index.py's offline replay call passes bulk_fts=True (comment explains why: owned inactive generation, never live daemon ingest). Daemon/live-ingest paths stay OFF -- confirmed follow-up, not done here.\n\nassert_session_fts_exact_sync: NO code change needed (design item #3 confirmed) -- triggers stay physically present in sqlite_master throughout (only WHEN-gated), so the trigger-presence half of the proof is unaffected; parity half still holds because the guard's finally-reinsert runs before the outer transaction completes and before assert_session_fts_exact_sync is ever called.\n\nVersion-bump decision (design item #4): NO INDEX_SCHEMA_VERSION bump. CREATE TRIGGER IF NOT EXISTS means old archives keep the pre-guard trigger body until a real rebuild; they behave identically to before (guard row never consulted) -- correctness-neutral, only forgoes the perf win until rebuilt. Verified test_fresh_init_creates_canonical_fts_trigger_set (test_schema_policy_contracts.py, #3144 per the design's own citation) compares trigger NAMES via an INSERT/DELETE-INTO-fts-table substring match, not trigger bodies -- confirmed unaffected by the new WHEN clause.\n\nTests (tests/unit/storage/test_bulk_fts_prefix_reextract.py, new file): mode-off parity, mode-on byte-identical messages_fts content vs mode-off across both _delete_prefix_message_dependents (partial-tail) and _delete_all_session_message_dependents (full-tail) branches -- the key equivalence proof; guard-row-never-leaks-on-exception (verified via real transaction rollback, not just in-process state); assert_session_fts_exact_sync passes post-apply; anti-vacuity test monkeypatches insert_session_rows_sql to a no-op and confirms the parity proof THEN fails, proving the equivalence tests aren't vacuous. Also had to fix two pre-existing test stubs with strict (non-**kwargs) fake signatures that broke on the new bulk_fts kwarg: test_lineage_normalization.py's _fail_after_graph_resolution and test_live_cursor_persistence.py's lock_once.\n\nVerification: devtools test (214 passed across 9 files) + devtools verify --quick (exit 0, ran twice -- once pre-push hook, once manual). Broader tests/unit/storage/ sweep surfaced 19 additional failures (5 in test_live_batch_support.py already known from the isolated batch-support run, 14 more across test_retrieval_readiness_laws.py/test_dangling_fts_derived_surfaces.py/test_embedding_freshness_invariant.py/test_durable_migrations.py/test_index_fast_forward_lifecycle.py/test_delegations_view.py/test_archive_tiers_archive.py) -- confirmed ALL pre-existing via git diff-to-patch + git checkout -- + git apply revert-and-rerun (never git stash, per worktree isolation policy) against unmodified master; identical failures reproduce without this change. Not investigated further (out of this bead's scope) but worth a separate bead if not already tracked.\n\nPR: https://github.com/Sinity/polylogue/pull/3152\n2026-07-19 20:50 coordinator: v41 generation (gen-1784472269802, 1976 sessions) abandoned — #3163 v42 bump makes it permanently unresumable (open guard rejects non-current versions). Fresh v42 rebuild launched on combined #3163+#3165 code: gen-1784486727919-da69ed72, user_version 42, all 40 triggers present, bulk-build lifecycle active (clear invariant + final repopulate now productized in rebuild_index.py — manual /realm/tmp/trigram-restore-pre-promote.py + canonical-DDL sidecars deleted as superseded). Dead generations queued for post-promote deletion: gen-1784422147106 (20G v40), gen-1784471544847 + gen-1784471887427 (venv-hijack v40), gen-1784472269802 (6.6G v41).\n2026-07-22 CORRECTION to the close reason: the \"live-ingest stays per-row by design (bounded per-session writes)\" decision was disproven within hours — a live whale-session prefix-tail rewrite held the 3.14t daemon writer \u003e1h at 260GB reads / zero commits. Fixed by PR #3259: bulk_fts=True at the live ingest chokepoint (_core.py) and the materialization backfill (repair.py, also the t93b whale-pass route). Byte-identity was already proven by the #3152 parity suite; the conservatism was the only thing holding it off.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-19T03:21:55Z","created_by":"Sinity","updated_at":"2026-07-21T23:33:31Z","started_at":"2026-07-19T07:01:15Z","closed_at":"2026-07-21T22:57:49Z","close_reason":"AC complete. (1) Bulk path no longer per-row: guard-gated FTS bulk mode at the real storm site (_reextract_prefix_tail_db) via _bulk_fts_session_guard (#3152, byte-identical parity tests + anti-vacuity no-op-reinsert proof), and the bulk-build lifecycle (clear invariant + final repopulate) productized into rebuild_index.py (#3163/#3165), trigram restore sidecars deleted as superseded. (2) Measured before/after on the production trigram-heavy corpus: v40 walk died on a single whale prefix-tail write \u003e3h (overflow-chain + per-row delete machinery); the v41/v42 walk completed the full 101,347-raw corpus in 662.9 min driver total (36 passes) including the same whale content. (3) Live-ingest path decision RECORDED: bulk mode stays OFF for daemon live ingest by design (bounded per-session writes; scoped-DROP-trigger optimization already covers full session replace); the remaining live-path per-row hazard is the DELETE cascade machinery, tracked as polylogue-meoz, and whale-component ingest now routes through the daemon escalation pass (polylogue-t93b, #3256). (4) FTS/trigram consistency proven: parity tests + docsize==indexable at promote (4,771,641) and after retirement (4,753,541==4,753,541) + v43 messages_fts_identity ledger populated at exact block parity.","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -132,7 +133,7 @@ {"_type":"issue","id":"polylogue-41ow","title":"user_write.py: upsert_assertion has a reproduced TOCTOU race that silently reverts operator judgments","description":"dogfood-2 write-path investigation (investigations/write-path-correctness.md, F-027): the shared write chokepoint upsert_assertion (polylogue/storage/sqlite/archive_tiers/user_write.py:1008-1149) does a non-atomic SELECT (1057-1060) then a separately-committed INSERT ... ON CONFLICT (1099-1123) to decide whether an incoming non-user write should preserve an existing terminal judgment (the 37t.15 promotion-gate invariant). Reproduced with a forced-interleaving test against the real, unmodified functions: a concurrent operator accept landing between a detector re-runs SELECT and INSERT gets silently overwritten back to candidate. This directly violates the functions own documented invariant (a later automated write must not resurrect a judged-rejected row back to candidate) and the polylogue-303r.5 design requirement that concurrent conflicts become explicit, not silent last-write-wins. No cross-process lock exists around any of the ~18 user.db write call sites in archive.py -- DaemonWriteCoordinator only serializes in-process daemon writers.","design":"Use the existing transaction and judgment authorities rather than leaving two acceptable outcomes. Wrap the automated upsert_assertion read/preserve/write decision in BEGIN IMMEDIATE on the user-tier connection so concurrent connections observe one legal serial order: if automation commits first, a later operator judgment wins; if judgment commits first, automation re-reads and preserves the terminal state. Keep judge_assertion_candidate as the explicit optimistic-conflict path for competing operator-visible judgments; idempotent retry is distinct from conflict and blind last-writer-wins is forbidden. Route every connection through the canonical 30-second connection profile while touching this path. Multi-row invariants use one immediate transaction; commutative counters elsewhere should remain atomic SQL expressions rather than sharing a coarse process lock.","acceptance_criteria":"1. The forced interleaving with an operator accept between the old SELECT and INSERT cannot revert accepted status and resolves to one valid serial order. 2. upsert_assertion begins its preserve/write decision under one immediate user-tier transaction and rolls back completely on failure. 3. Competing operator judgments still return an explicit conflict unless they are idempotent retries; no last-writer-wins fallback is introduced. 4. A second process/connection is covered, proving DaemonWriteCoordinator is not the correctness mechanism. 5. All touched user-tier connections use the canonical busy-timeout/profile. 6. Removing the transaction boundary or terminal-status preservation makes the real forced-interleaving test fail.","notes":"2026-07-18 Fable adjudication groundwork for the ann-04 external delivery (results/ann-04/r01): the delivery independently implements the writer-slot fix, but master ALREADY landed its own via #3051: _immediate_user_write_transaction (BEGIN IMMEDIATE when no transaction). SEMANTIC RESIDUAL TO ADJUDICATE: masters helper yields early when conn.in_transaction is true, so a caller-owned DEFERRED (read) transaction is NOT upgraded before the preservation read - the delivery handles exactly that case with SAVEPOINT + zero-row write upgrade. Determine whether any production caller reaches upsert_assertion inside a deferred transaction; if yes, masters fix has a residual race and the deliverys nested-upgrade should be ported (with its two-connection regression). The delivery patch applied verbatim at 536a53e is pushed as branch feature/assertions/judgment-transaction (worktree polylogue-intake-apply); rebase conflicts are confined to archive.py import (trivial), user_write.py (the mechanism overlap above), test_archive_tiers_assertions.py. Its additive components (bounded evidence previews shared by CLI+MCP, queue health in judge/status, mark-candidates dedup, actor-scoped capture idempotency, operator canary script) do not overlap masters fix and remain valuable.\n2026-07-18 lane-g adjudication: audited every production caller of upsert_assertion/judge_assertion_candidate (annotations/write.py, security/lifecycle.py, security/secret_scan.py, scenarios/corpus.py, storage/repair.py, storage/raw_reconciler.py, and user_write.py's own batch helpers). None currently reach upsert_assertion with conn.in_transaction already True from a non-immediate (deferred) transaction -- every real call site opens a fresh connection immediately before calling in, or (scenarios/corpus.py's multi-upsert batch) reuses a connection whose open transaction was itself already BEGIN IMMEDIATE from an earlier call in the same batch. So master's #3051 fix is not currently exploitable in production.\n\nHowever the residual is real and latent (same character as vwia): a future caller reusing a connection across a caller-owned BEGIN (deferred) transaction would still race, since _immediate_user_write_transaction's `if conn.in_transaction: yield; return` branch does not upgrade the lock. Closed it defensively by porting the ann-04 delivery's SAVEPOINT + zero-row-write lock-upgrade mechanism (not the delivery's auto-commit-on-fresh-path change, which would have broken scenarios/corpus.py's intentional multi-call batch atomicity -- verified this by tracing that upsert_mark/upsert_blackboard_note/upsert_saved_view/2x upsert_assertion share one transaction today via the same fresh-BEGIN-IMMEDIATE-then-nested-yield path). Also added the canonical busy_timeout PRAGMA to the fresh-transaction branch (AC 5).\n\nNew regression: test_cross_connection_replay_inside_caller_owned_deferred_transaction_cannot_resurrect_operator_accept (tests/unit/storage/test_archive_tiers_assertions.py) -- two real connections, detector opens a plain BEGIN (not IMMEDIATE) before calling upsert_assertion, confirmed via git-stash anti-vacuity that it fails on pre-fix code (operator resurrects within 0.15s) and passes post-fix.\n\nNOT ported: the delivery's non-overlapping additive scope (bounded evidence previews shared by CLI+MCP, queue health in judge/status, mark-candidates dedup, actor-scoped capture idempotency, operator canary script) -- that's feature delivery, not a correctness fix, and is out of this hardening-sweep lane's scope (4 correctness items only). Filing a follow-up bead to track porting it from feature/assertions/judgment-transaction (worktree polylogue-intake-apply) separately.\n\nAlso NOT touched: the addendum's broader PRAGMA foreign_keys normalization across upsert_mark/upsert_suppression/upsert_annotation/upsert_correction/upsert_session_tag_assertion/etc (6+ call sites) -- confirmed upsert_assertion's own foreign_keys PRAGMA is a silent mid-transaction no-op (assertions has no FK constraints so this is low-impact), left as the addendum originally scoped it: a separate normalization, not blocking this bead's AC.\n\nVerification: devtools test tests/unit/storage/test_archive_tiers_assertions.py tests/unit/storage/test_archive_tiers_assertion_write_through.py tests/unit/storage/test_comparative_judgment_assertions.py -\u003e 61 passed. Plus annotations/security/cli/mcp write-path suites (103 passed) and scenarios/corpus demo suites (25 passed) to confirm no batching regression. devtools verify --quick exit 0.\nPR #3101 (feature/fix/toctou-assertion-transaction), open, verified via devtools verify --quick and anti-vacuity (git stash) proof. Close after merge.\n2026-07-19 wave-2 re-triage (agent-a6e19ec7c37870290 worktree): the ann-04 delivery (judgment-transaction) was re-submitted byte-identical (sha256 a72c7e0...) at /realm/tmp/gpt-pro-intake-0719/. Confirmed already fully superseded: this bead's own #3101/#3110 fix plus polylogue-2o3d's #3138 port cover 100% of the redelivered patch's acceptance-matrix rows. git apply --check against current master (e963e87f5) fails on every production file (mcp/server_mutation_tools.py + 2 test files no longer exist post six-tool-cutover; the rest diverged). No action needed; recorded as results/ann-04/r02 (state=superseded) in the wave-2 campaign ledger.\n2026-07-20 correction to the 2026-07-19 re-triage note: per results/README.md custody policy ('Duplicated browser downloads with identical SHA-256 values are deliberately not copied twice'), no r02 package revision was minted for the byte-identical re-download. The 2026-07-19 supersession analysis (41ow/#3101+#3110, 2o3d/#3138 cover the delivery in full) is instead recorded as a dated reassessment entry inside the existing results/ann-04/r01/receipt.json, whose top-level state was updated needs_rebase_review -\u003e superseded to match. See PR #3177.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-16T10:21:27Z","created_by":"Sinity","updated_at":"2026-07-19T23:35:40Z","started_at":"2026-07-17T18:19:36Z","closed_at":"2026-07-18T17:05:55Z","close_reason":"Fixed and merged across two PRs: #3101 closed the residual TOCTOU gap left by master's earlier #3051 fix (a caller-owned deferred transaction was never upgraded to hold the write lock before the preservation read) via a SAVEPOINT + zero-row-write lock upgrade, with a two-connection regression test proving the race is closed. #3110 normalized the foreign_keys pragma across all 10 user-tier overlay writers into the single upsert_assertion chokepoint (the completeness addendum from this bead's own notes). Both anti-vacuity proven via git-stash, devtools verify --quick green on both, merged to master. Deliberately did not port the ann-04 delivery's non-overlapping additive scope (evidence previews, queue health, capture idempotency, canary script) -- tracked separately as polylogue-2o3d.","labels":["area:correctness","area:storage","discovered-from:dogfood-2"],"dependencies":[{"issue_id":"polylogue-41ow","depends_on_id":"polylogue-303r.5","type":"relates-to","created_at":"2026-07-16T12:21:27Z","created_by":"Sinity","metadata":"{}"}],"comments":[{"id":"019f6aab-dc26-7ba7-a8d4-20b17e80ebac","issue_id":"polylogue-41ow","author":"Sinity","text":"Completeness addendum from the same investigation (write-path-correctness.md): PRAGMA foreign_keys = ON is set inconsistently across upsert_* functions -- upsert_suppression, upsert_mark, upsert_annotation, upsert_correction each call it themselves (user_write.py:510,536,636,664) but upsert_session_tag_assertion, upsert_session_metadata_assertion, upsert_saved_view, upsert_recall_pack, upsert_workspace, upsert_blackboard_note do not. Low-impact today since the assertions table declares no FOREIGN KEY constraint, but worth normalizing (one shared connection-setup helper instead of six independent inline pragma calls) while this bead is already touching every write call site in this file for the TOCTOU fix.","created_at":"2026-07-16T11:24:36Z"}],"dependency_count":0,"dependent_count":0,"comment_count":1} {"_type":"issue","id":"polylogue-1xc.14.1","title":"Derive archive-scale workload profiles from provider schemas","description":"Current schema inference produces structurally valid provider records but destroys the distributions and relationships that activate production failures. Field marginals are sampled independently, numeric values are uniform over extrema, arrays are capped at five, CorpusSpec uses a uniform message-count range, and cluster collection materializes the full unit stream. Consequently tests can traverse real ingest code while remaining unlike the archive shapes that caused the July 15 exact-session action query to perform archive-global ranking before a selective bound. Provider observations must remain the authority: infer a bounded privacy-safe workload profile beside each schema package, generate deterministic provider wire artifacts from it, and exercise the real acquire, parse, materialize, index, query, cancellation, and cleanup routes. This is a production workload declaration, not a handwritten realistic fixture library and not a semantic cap on archive size.","design":"Add a versioned WorkloadProfile artifact to provider schema packages and reference it from WorkloadEnvelopeSpec. Extend field statistics with bounded streaming counts and deterministic quantile sketches for presence versus null, type mix, numeric/string/array/object sizes, payload tails, and conditional distributions. Add structural joint profiles keyed by provider package/version, artifact kind, and cluster for field co-occurrence, tagged-union variants, nested tool envelopes including functions.exec, tool call/result pairing states (paired, missing, late, duplicate, error), lineage depth/width/replay, growing-session state, and convergence state. Add an archive mix profile for origin/package proportions, session/message/block/action size distributions, selective predicate cardinalities, payload tails, and topology shapes. Store only counts, rates, buckets, structural tokens, and privacy-approved enum values; never persist raw content, paths, IDs, rare strings, or representative payloads in the promoted profile. Replace list(iter_schema_units(...)) and unbounded measurement lists with bounded deterministic streaming aggregation. Extend synthetic generation so one seed chooses correlated profile variants and archive scale/selectivity targets, emits provider-native bytes, and then invokes production ingestion and read composition. Wire scenario/performance/query-law lanes to generated workload IDs and shared receipts. First canary reproduces C-03: an exact-session actions query over a large mixed archive must push the session bound into both ranking legs and remain fast; a mutation restoring global-first composition must fail. Existing hand-authored fixtures remain only for minimal parser edge cases and independent known-answer oracles.","acceptance_criteria":"1. Every promoted provider package may carry a versioned, deterministic, privacy-classified WorkloadProfile whose provenance names archive generation, observation window, sample counts, inference version, and privacy policy; schema generation into a staging directory does not mutate committed packages. 2. Inference captures bounded streaming presence/null/type rates, quantiles and tails, joint structural variants, tool-result relationship states, lineage/replay shapes, active-growing and convergence states, provider/package mix, archive unit sizes, and predicate selectivity without retaining the corpus or unbounded per-value lists. Peak inference memory is bounded independently of sample count and full-corpus generation proves that bound. 3. Synthetic generation consumes the profile jointly rather than sampling independent marginals, emits deterministic provider-native wire artifacts, and reaches the production acquire, parse, materialize, index, and query implementations; removing a production parser or query pushdown breaks the test. 4. Named scale tiers preserve tail and selectivity activation conditions while allowing small deterministic CI projections. The C-03 canary includes a mixed archive plus exact-session action query and fails when either ranking leg loses the selective bound. Tool pairing, lineage replay, growing-session, and partial-convergence canaries are generated from the same profile mechanism. 5. Workload runs emit polylogue-1xc.14 receipts with workload/profile/build/archive identity, phase timings, resource peaks, cancellation/progress, and cleanup; no performance test invents a separate corpus identity or measurement envelope. 6. A promotion review reports structural changes, distribution changes, and a privacy-vetting inventory. It automatically rejects raw content, filesystem paths, account identifiers, session/message/tool IDs, rare free text, and secrets while listing potentially identifying structural enum/date/domain values for operator approval. 7. The vague performance/throughput scenario family is superseded by this mechanism, and focused schema inference, generator, real-route canary, privacy, determinism, memory-bound, and receipt tests plus devtools verify --quick pass.","notes":"2026-07-16 operator correction: do not optimize for the smallest profile or a preselected minimum of statistics. Preserve every observation with positive expected downstream utility when it can be represented deterministically, privacy-safely, and with bounded streaming resources. Boundedness constrains inference memory and encoded representation, not semantic ambition. The profile format must be extensible, retain sufficient statistics or mergeable sketches for useful derived views, and emit a loss/novelty inventory for stable observed structure that no current field models so useful signal cannot disappear silently. Compact marginals, joints, sketches, and conditional summaries are encodings of evidence, not permission to discard it.\n2026-07-16 first implementation slice (not closure): provider packages now carry deterministic privacy-classified workload profiles with bounded numeric/string/array/object and categorical sketches, structural joint variants, tool-result/functions.exec and lineage relationships; synthetic scalar/array generation consumes observed histograms; an explicit archive-composition artifact captures origin/package mix, session/message/block/action shapes, payload tails, anonymous predicate selectivity, topology, raw revision/growing-source state, convergence debt/lag, and tier sizes without retaining content, paths, repository/branch/model/tool values, or IDs. Every categorical observation contributes to a fixed-memory hashed distribution and approximate-distinct sketch even when readable values are privacy-suppressed. Focused evidence: 765 affected schema tests passed in 38.93s; strict mypy passed; devtools verify --quick passed every step except pre-existing demo-corpus-construct-audit drift owned by polylogue-b054.1.1.1/browser capture. Remaining parent scope is durable: child polylogue-1xc.14.1.1 owns the replayable ObservationJournal and true full-corpus memory bound; joint synthetic variant selection, named scale tiers, C-03 and other production-route canaries, shared workload receipts, promotion/privacy review, and live regeneration remain open.\n2026-07-16 correction to the first-slice note: demo-corpus drift was not pre-existing. Clean master was stable across three sequential and three 8-worker isolated runs. The workload branch changed RNG consumption and exposed that ChatGPT/browser-capture coalescence depended accidentally on a seeded UUID. The fix makes scenario-declared session_native_ids authoritative at provider wire generation, so schema/profile evolution can change content distributions without changing a fixture identity contract. The existing real ingest/convergence test failed before the fix and passed afterward; demo-corpus-datasheet is again in sync.\n2026-07-26 portfolio-convergence audit: released stale in_progress claim after \u003e7 days with no recorded activity; scope remains open and must be re-claimed on real work start.","status":"open","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-16T09:45:51Z","created_by":"Sinity","updated_at":"2026-07-26T09:13:00Z","started_at":"2026-07-16T11:58:59Z","labels":["area:devtools","area:ops","area:perf","area:sources","area:test","area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-1xc.14.1","depends_on_id":"polylogue-1xc.14","type":"parent-child","created_at":"2026-07-16T11:45:51Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-88jp.1","title":"Make pytest/testmon execution isolated, bounded, and fast","description":"The managed broad verification path is not a trustworthy gate. A fresh `devtools verify --seed-testmon --skip-slow` on 2026-07-16 produced dozens of parallel-only failures, accumulated roughly 4 GiB RSS across the pytest controller/workers, then stopped making progress at 91% with `test_periodic_wal_checkpoint_targets_archive_root_tiers` and the already-tracked embedding catch-up test still running for more than five minutes. The same changed production routes pass focused single-process tests. Testmon seeding also executes essentially the full 16k-test suite and currently costs far more than a useful local gate. Repair the class: deterministic checkout-local state, worker-safe fixtures/resources, hard per-test cancellation with useful stacks, honest failure receipts, and materially faster selection/execution.","design":"Build an evidence harness around `devtools verify` and `devtools test`: record nodeid, worker, checkout/env fingerprint, temp/archive roots, locks/ports/processes, elapsed/idle time, RSS/PSS, and teardown state. First classify every seed-run failure as shared-state collision, order dependence, product defect, stale test, or harness bug by rerunning exact nodes single-process and in controlled xdist groups. Give each xdist worker disjoint basetemp/archive/config/service/port namespaces; prohibit live daemon/global cache/shared SQLite coupling unless explicitly serialized. Replace silent long-running awaits with bounded event-driven helpers and pytest-timeout stacks; integrate the existing polylogue-09rn production-signal fix rather than masking it. Make testmon data checkout-local, atomically seeded, concurrency-locked, and incrementally reusable; a seed is an explicit maintenance action, while ordinary verify selects affected tests plus declared blind-spot/risk fallbacks. Profile collection/import, fixture setup, DB/schema seeding, process startup, and slowest nodes; cache immutable session/schema corpora safely per worker and remove duplicate initialization without weakening coverage. Emit a machine-readable verification receipt and fail if workers leak, stalls exceed policy, or selected-test accounting is incomplete.","acceptance_criteria":"1. The current broad seed failure set is captured with exact nodeids and classified by controlled single-process/xdist reproductions; no failure is dismissed as generic flakiness. 2. Two consecutive clean-worktree `devtools verify --seed-testmon --skip-slow` runs finish green without hang, leaked worker/process, cross-worker DB/port/path collision, or unbounded RSS; the second run demonstrates safe reuse. 3. The embedding catch-up and periodic WAL checkpoint tests complete under ten seconds in ten consecutive isolated and xdist runs, with production lifecycle bugs fixed rather than timeout increases. 4. Every test has an effective bounded timeout or an explicit reviewed lane override; timeout output names the nodeid and includes useful worker stacks, and the supervisor kills the full process tree on stall. 5. Ordinary `devtools verify` from a seeded checkout selects affected tests plus declared collection-time/risk fallbacks, never silently selects zero for changed executable code, and reports the exact selection basis. 6. Measured full non-slow wall time and peak memory improve materially from the 2026-07-16 baseline (target at least 2x faster and under 3 GiB peak on this host) without reducing test/capability coverage; any target miss is explained with the next dominant cost. 7. Focused, xdist, seed, ordinary affected, and CI commands share one tested configuration contract; mutation/removal of worker namespace isolation, timeout enforcement, selection accounting, or teardown detection fails.","notes":"Initial evidence receipt: Sinex integration worktree seed run id 20260716T042022Z-seed-testmon-3415508-b1779b54. At stop: 91%, controller PID 3416284 ~1.05 GiB RSS; workers ~1.43 GiB, 914 MiB, 873 MiB, 633 MiB. Two running nodes were tests/unit/daemon/test_daemon_cli.py::test_periodic_wal_checkpoint_targets_archive_root_tiers and tests/unit/daemon/test_embedding_convergence_progress.py::test_periodic_embedding_backlog_waits_for_catch_up_complete. The run emitted many earlier F/E results under xdist; focused changed tests and quick gate were green. Supervisor was interrupted after \u003e9 minutes, not allowed to burn its 45-minute ceiling.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-16T04:36:03Z","created_by":"Sinity","updated_at":"2026-07-16T04:40:30Z","started_at":"2026-07-16T04:36:14Z","closed_at":"2026-07-16T04:40:30Z","close_reason":"Superseded by the stronger fresh-checkout verification invariant polylogue-b054.1.1. Its independent 2026-07-16 evidence, zero-baseline requirement, repeated daemon witnesses, shared configuration contract, affected-selection anti-vacuity, and performance envelope were folded into that bead without reducing scope.","labels":["area:devtools","area:test","area:verification","delivery:M-substrate-consolidation","horizon:frontier","lane:verification-readiness"],"dependencies":[{"issue_id":"polylogue-88jp.1","depends_on_id":"polylogue-09rn","type":"relates-to","created_at":"2026-07-16T06:36:02Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-88jp.1","depends_on_id":"polylogue-88jp","type":"parent-child","created_at":"2026-07-16T06:36:02Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-88jp.1","depends_on_id":"polylogue-vyxq","type":"relates-to","created_at":"2026-07-16T06:36:02Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-88jp.1","depends_on_id":"polylogue-wple","type":"relates-to","created_at":"2026-07-16T06:36:02Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-88jp.1","depends_on_id":"polylogue-y6tb","type":"relates-to","created_at":"2026-07-16T06:36:02Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"polylogue-hjpx.2","title":"Prove raw replay convergence at the July-15 archive shape","description":"hjpx AC6 remains unproven after the execution-foundation phase. Small unit fixtures exercise 25 singleton raws and skewed cohorts, but they do not match the 2026-07-15 preflight shape: roughly 15,264 direct / 21,398 expanded candidates, 10,163 authority components, 4.788 GB retained payload, 1,890 broken active seeds, 40 cursor-ahead sources, and 34 incomparable heads. A sanitized evidence harness must establish that the immutable planner and bounded executor converge without recreating the live daemon's memory/non-progress incident.","design":"Build a deterministic synthetic/sanitized corpus generator parameterized from the recorded yla8 preflight distributions: component sizes, revision skew, source-path moves, bundle memberships, cursor/head conflicts, missing blobs, and resource-blocked cohorts. Run the real source/index/ops routes under Sinnix containment; measure pass-by-pass plan census digests, executable backlog, carried/deferred/terminal counts, RSS/PSS, swap, temp/database writes, wall time, daemon health latency, cursor/head invariants, and FTS closure. The proof is an executable benchmark/scenario with machine-readable receipts and explicit envelope, not a mocked count test. Include mutation variants that remove fair rotation and conservation accounting and reproduce non-progress.","acceptance_criteria":"1. The sanitized corpus matches the July-15 candidate/component/byte/skew distributions within documented tolerances and contains no private content. 2. Every bounded pass stays within declared RSS/PSS, swap, temp/write, and wall-time envelopes while daemon health remains responsive. 3. Executable plan backlog decreases monotonically modulo explicit retry injection; every finite retry resolves once and no component starves. 4. Cursor positions, source heads, accepted index heads, FTS readiness, and durable raw authority never regress across interruption/resume. 5. Two final quiescent census digests match with zero executable plans and identical typed residual debt. 6. Removing fair rotation recreates starvation; removing conservation/carry-forward accounting recreates a census mismatch; the harness fails both mutations. 7. Exact commands, containment receipts, corpus seed, and result artifacts are durable and reviewable. 8. No live archive apply is part of this bead; yla8 retains the separate verified-backup and explicit-authorization gate.","notes":"2026-07-16 inherited scale-proof detail from hjpx.1 adversarial pass 5: census attempts are bounded by component count and each transitive authority component is already byte-bounded to 1 GiB; a crash before component completion safely restarts that component and never publishes a partial plan. This bead must decide/prove the production-shape resource envelope and, if necessary, add sub-component parser checkpointing without weakening immutable plan publication.\n\n2026-07-17 static resource-path audit (no live mutation): the 1 GiB admission ceiling is not a policy-only blocker. revision_backfill._parse_retained_raw calls ArchiveStore.raw_revision_material, which calls BlobPublisher.read_all; _parse_one then wraps the full bytes in BytesIO even for stream-record providers. The blob store already exposes open(hash), and parse_stream_payload accepts an iterator, so Hjpx.2’s concrete implementation seam is a retained-raw metadata/open API plus a stream parse path for stream-record providers. Preserve component-atomic census/plan publication and spill semantics; replace only the eager blob read for safe stream providers. Non-stream/bundle formats remain bounded/admitted until separately safe. The existing \u003e1 GiB tests correctly prove fail-closed today and should evolve into anti-vacuity tests: a stream raw must parse without read_all, while an unsafe/non-stream cohort still blocks before open.\n2026-07-17 implementation foundation merged: PR #2966 / master 805d49286 removes eager blob materialization for retained JSONL replay in historical backfill, live append replay, and full/membership replay. It deliberately preserves the 1 GiB per-authority-component admission gate because Codex/Claude parser output remains materialized; 150 focused authority/backfill/live/repair tests passed, quick static/generated gate passed, and an independent adversarial close review found no remaining gap. This is a prerequisite improvement, not HJpx.2 closure: the remaining work is the July-15-shaped executable scenario with measured RSS/PSS/swap/I/O/wall-time, fairness/conservation mutations, and two fixed-point censuses.\n2026-07-17 scale execution started: contained scheduler-shape run uses merged workload-receipt:sha256:0d12c4e3186e98f1c8e25a6f62b6110eb6bfe78068c550c06ab9df38cc912d95 / PR #2967 against 10,163 components and 15,264 direct raws with a 128-component pass limit; outputs are being retained at /realm/tmp/raw-authority-july15-projection/. This proves real planner/executor cardinality only. It is explicitly not yet the 4.788 GiB materialized-byte or 21,398 expanded-candidate proof, so it cannot satisfy the scale bead by itself.\n2026-07-17: PR #2970 merged raw-authority scale-proof runner. It now batches blob/source acquisition safely, content-addresses the generated corpus, records process RSS/PSS/swap/CPU/I/O evidence, refuses contended hosts, and proves quiescence by two ledger-native dry-run censuses. A 256/384 medium run was deliberately terminated before receipt emission because a concurrent full-corpus schema generator held the host at \u003e5% I/O full-stall and the proof process entered D-state during acquisition; no scale result is claimed. Full July-shape execution remains pending a normally admitted host.\n2026-07-17: discovered and linked polylogue-3jlg after observing the concurrently running full-corpus schema generator read ~244.9 GiB/write ~295.1 GiB in ~73 minutes before final output. This is tracked as a separate evidence-first replay-amplification investigation; it does not change Hjpx.2 completion criteria.\n2026-07-17 current-workload evidence: a safe 3-component admission probe of `devtools workspace raw-authority-scale-proof` refused to start at I/O full avg10=12.11 (configured limit 2.00); no override was used. Static audit also found the runner accepts only components/raws/pass_limit and generates tiny fixed JSONL payloads. It cannot currently express or verify the required 4.788 GiB byte envelope, expanded-candidate/topology skew, conflict/blocked cohorts, or full July-15 distribution. Treat runner capability completion—not simply waiting for a quiet host—as the immediate Hjpx.2 implementation prerequisite; do not present the existing 10,163/15,264 scheduler-shape receipt as full-scale proof.\n2026-07-17: PR #3009 merged (314bcb011c): runner now accepts the aggregate-only raw-authority scale profile, streams a blob-backed synthetic corpus to the requested byte envelope, records requested versus achieved frontier shape, and provides a preparation-only receipt for expanded terminal/deferred topology. Focused runner tests (5) and quick verification passed. This is intentionally not a full-scale closure: expanded cohorts require explicit terminal/deferred outcome variants before an executable fixed-point claim; fairness/conservation mutations and an admitted July-envelope run remain.\n2026-07-17 read-only follow-up: invoking the new live --capture-profile against the active archive remained CPU-running for over a minute and did not emit its output under concurrent daemon/test load; two accidental coordinator duplicate readers were stopped without writing archive state. This confirms the full-profile capture needs its own bounded/resumable aggregate route or a quiescent maintenance window before it can be used as an operational preflight.\n2026-07-17 scale-proof progress: PR #3011 made the live aggregate profile complete in ~3.4s without archive mutation. PRs #3015, #3017, and #3020 then hardened corpus generation (published-blob prefix continuation, private-free exact component/direct-candidate cohorts, distinct synthetic direct identities). The contained exact-cohort preparation receipt is /realm/tmp/raw-authority-scale-cohort-current-20260717.stdout.json: requested=achieved 36,401 components, 41,450 direct candidates, 46,979 expanded candidates, 2,036,129,455 bytes; cohorts 36,399x(1 direct/1 raw), 1x(2,856 direct/3,885 raw), 1x(2,195 direct/6,695 raw). It deliberately records no private ids/paths/hashes/content and does not mutate/replay the live archive. Remaining AC gaps: synthetic terminal/deferred cohort execution, fairness/conservation mutation proofs, measured full replay envelope/daemon-health and fixed-point evidence.\n2026-07-17 PR #3021 added executable typed cohort variants to the scale runner. Small real-repair scenarios now prove both an ambiguous sibling terminal outcome and a deferred sibling outcome, each followed by two matching dry census digests; receipts include per-pass outcome-status counts. This closes the prior preparation-only restriction for explicit typed cohorts, but not the full current-profile execution/mutation/resource-health ACs.\n2026-07-17 mutation closure progress: PR #3029 makes any nonzero immutable replay-plan conservation error fail closed; its real-route mutation regression corrupts the after-pass algebra and asserts success=false. PR #3031 removes durable attempt-age fairness in a one-slot, retry-injected scenario and reproduces starvation; the production scheduler instead advances to the next independent component. Remaining scale-proof scope is byte-skew fidelity, interruption/cursor-head/FTS/daemon-health evidence, and an admitted full executable profile receipt.\n2026-07-17: PR #3034 merged (d108e9431). The real interruption/recovery path now snapshots after the post-write injected failure and proves recovery preserves source raw authority fields, accepted revision head/session pointer, FTS readiness, and actual FTS hits while finalizing the interrupted ledger census. This narrows AC4 at the retained-raw/revision-head/read-model boundary; source-file ingest cursor evolution and live daemon responsiveness remain scale-run evidence, not claimed by this unit proof.\n2026-07-17: PR #3037 merged (b8c29e679). `raw_materialization_scale_profile` now emits a private-free joint cohort of component raw count, direct candidate count, component blob-byte power-of-two bucket, and count; the scale runner consumes it, validates topology marginals, and allocates exact total bytes inside those buckets. This closes the prior byte-skew fidelity implementation gap in AC1. A new current-profile capture and admitted executable receipt are still required; old v1 captures remain readable but naturally lack the additive joint cohort.\n2026-07-17 contained preparation attempt against current v2 profile: host admission began at I/O full avg10=0.01, memory full=0.00. After 38s / ~159MiB synthetic output, runner entered D-state and I/O full avg10 rose to 5.36; coordinator stopped its own background scope, which left ~273MiB disposable partial synthetic output and no receipt. Diagnosis: byte-cohort allocator preserved a component total by placing all remaining bytes in its final raw, turning the real 6,695-member ~2GiB component into one synthetic ~2GiB blob. Follow-up implementation is required before retry: distribute explicit-cohort bytes across independent member raws and enforce pressure checks between bounded publication batches. No live archive was read/written beyond the earlier read-only profile capture.\n2026-07-17: PR #3038 merged (695fedbc7) after the stopped v2 preparation diagnosis. Explicit-cohort byte budgets now distribute across independent member raws instead of concentrating in one terminal blob; generation rechecks I/O/memory pressure after each 128-row publication batch and returns all safe samples in receipts. Focused runner suite (11) and quick verification passed. The next contained preparation must use this merged runner; no claim yet about its outcome.\n2026-07-17 second contained v2 preparation used PR #3038. It self-aborted at the first batch checkpoint with I/O full avg10=4.51 \u003e 2.00, rather than entering D-state; no receipt was emitted and partial synthetic output is being removed. This validates continuous enforcement but reveals remaining generator churn: explicit cohorts still stage tens of thousands of sub-megabyte payload files before publication. Next implementation removes that duplicate disk staging for independent explicit cohorts by publishing generated bytes directly; prefix-sharing cohorts retain the streaming file path.\n2026-07-17: PR #3040 merged (0829f65e0). Production-shaped explicit cohorts now construct bounded standalone JSONL bytes and call publisher.write_from_bytes directly; an anti-vacuity test rejects the old staged-path call. Prefix-sharing inputs retain streamed staging. Focused runner suite (12) and quick verification passed. Retrying current v2 preparation is now warranted with the existing continuous pressure gate.\n2026-07-17: PR #3043 merged (fix(repair): retain receipts for resource-deferred replay). Raw authority now reports executable versus resource-deferred candidate debt only when relevant; repeated all-deferred apply passes reuse the exact-scope durable receipt without ledger spam; dry runs still publish two matching fixed-point censuses. Focused proof: 13 passed; raw-materialization selection: 35 passed/17 deselected; devtools verify --quick succeeded. This removes the harness semantic blocker where a legitimate envelope-deferred residual was mistaken for incomplete convergence.\n2026-07-18 lane-D v2 corpus-prep retry, attempt 1: sinnix-scope background -- devtools workspace raw-authority-scale-proof --components 10163 --raws 15264 --expanded-raws 21398 --pass-limit 1000 --keep --json self-aborted at the FIRST admission check (before any generation work): I/O pressure gate refused with full avg10=3.06 \u003e 2.00 (host avg10 was 4.38-4.78, avg60 7.72-8.35, avg300 7.24-7.89 at the time -- 4+ other warroom lanes actively running, matching SONNET-NOTE expectations). This is a valid, expected self-abort per the gate design; the gate was not loosened and the corpus was not shrunk. Receipt: /realm/tmp/raw-authority-july15-v2-20260718/attempt1.stderr.log. Launched a bounded (40-min, 60s-poll) wait-then-retry wrapper in the background (/realm/tmp/raw-authority-july15-v2-20260718/wait_and_run.sh) that runs the identical command once io_full_avg10\u003c=2.00 or the deadline passes (in which case it runs anyway to record a second honest self-abort receipt rather than idling indefinitely).\n2026-07-18 lane-D corpus-prep retry, attempt 2 (bounded wait-then-run wrapper): after a 26-minute poll (60s interval, 27 polls, io_full_avg10 ranging 1.40-16.96 -- host stayed persistently contended across 4+ concurrent warroom lanes for the entire window), a brief quiet tick (avg10=1.40) passed initial admission and generation began. The CONTINUOUS pressure gate then correctly self-aborted mid-generation at check_generation_pressure() (devtools/raw_authority_scale_proof.py:781) with avg10=2.46 \u003e 2.00, before any component/member payload accumulated enough to leave a directory behind (verified: no partial /realm/tmp/*/raw-authority-scale-proof residue from this attempt). This is the SECOND consecutive valid self-abort this session (attempt 1: refused at the very first admission check, avg10=3.06; attempt 2: passed admission, self-aborted mid-generation, avg10=2.46). Both prove the gate enforces continuously and correctly under sustained real contention; neither is a scale-shape or correctness failure. Receipts: /realm/tmp/raw-authority-july15-v2-20260718/{attempt1,attempt2}.stderr.log, wrapper.log (full poll history). Host has not had a single 60s-sampled quiet window (avg10\u003c=2.0) longer than one tick in 40+ minutes of observation this session -- genuinely saturated, not a fluke. Launching a third, longer-bound (90 min) retry in the background; continuing other lane-D work (proof-report skeleton, confirming #3080 interruption/resume coverage) while it runs.\n2026-07-18 lane-D corpus-prep retry, attempt 3: wrapper required 3 CONSECUTIVE quiet polls (io_full_avg10\u003c=2.00, 60s apart) before launching, reasoning that a single-tick quiet window (attempt 2) was insufficient to survive generation. Found one at poll#12 (18:39 CEST) after 12 minutes of oscillating pressure (0.02-7.60). Self-aborted again, this time further into generation (devtools/raw_authority_scale_proof.py:853, inside the publish-batch flush path, versus line 781 on attempt 2 -- i.e. it survived past at least one _PUBLISH_BATCH_SIZE flush cycle this time) at avg10=3.63\u003e2.00. Three consecutive honest self-aborts this session (3.06 at admission / 2.46 early-generation / 3.63 mid-generation-past-first-flush), each showing the gate working correctly and each getting incrementally further, but the host has not sustained a quiet window long enough to complete the full 21,398-row generation phase in ~80 minutes of observation across 3 attempts. This is consistent with SONNET-NOTE 2026-07-18s explicit expectation of 4+ concurrent warroom lanes causing real contention, not a harness defect. Receipts: attempt{1,2,3}.stderr.log, wrapper{,2}.log under /realm/tmp/raw-authority-july15-v2-20260718/. Launching a 4th, more patient attempt (2h bound, 5 consecutive quiet polls required) while finalizing the proof report with what is provable regardless of this attempts outcome.\n2026-07-18 lane-D corpus-prep retry, attempt 4 (final this session): required 5 CONSECUTIVE quiet polls (5 min sustained, io_full_avg10\u003c=2.00) before launching -- found at poll#36 (19:15 CEST) after avg10 sequence 1.97/1.25/0.32/1.56/0.13. Self-aborted again at the SAME line as attempt 3 (raw_authority_scale_proof.py:853, mid-generation past the first publish-batch flush) at avg10=2.36\u003e2.00, despite the 5-minute sustained-quiet precondition. This is a significant diagnostic finding: the abort recurring at the identical code location across two attempts, immediately after 5 minutes of genuine external quiet, suggests the GENERATION PHASE ITSELF (writing/flushing the first _PUBLISH_BATCH_SIZE batch of raw payload files) is I/O-intensive enough to self-trigger the gate on this host, not purely a function of other lanes contentions -- external quiet alone does not guarantee survival past the first flush. Four consecutive honest self-aborts total this session (3.06 admission / 2.46 early-gen / 3.63 past-first-flush / 2.36 past-first-flush-again), spanning ~140 minutes of observation across 4 attempts with three different wait strategies (immediate / 40min-1-tick / 90min-3-tick / 2h-5-tick). Stopping retries this session -- diminishing returns from further identical-strategy attempts, and the finding itself (generation-phase self-induced pressure) is more valuable to record than a 5th blind retry. Receipts: attempt{1,2,3,4}.stderr.log, wrapper{,2,3}.log (full poll history with PSI samples) under /realm/tmp/raw-authority-july15-v2-20260718/. Full findings in .agent/reports/hjpx2-july15-scale-proof-2026-07-18.md (commit c2d3be71a, being updated this pass). hjpx.2 remains in_progress, NOT closable: AC1/AC6/AC7 (July-15-scale execution itself) unproven this session; AC2 (envelope, proven at small scale) unmeasured at requested cardinality; AC2 (daemon-health) unprovable with current harness (polylogue-agvo filed); AC3/AC5 (fairness/fixed-point mechanisms) proven via existing regression tests but not exercised at requested cardinality; AC4 (interruption/resume) proven via #3080, cited not duplicated.\n2026-07-18 lane-D: PR #3122 opened (https://github.com/Sinity/polylogue/pull/3122) carrying the proof-status report + yla8 packet + 9p8x fix. hjpx.2 itself remains open (not closable -- see report and prior notes for the AC gap).\n2026-07-20: the scale-proof report referenced from this bead was untracked from the repo by the .agent excision (PR #3180, operator directive); it persists on the operator host under .agent/reports/ in the main checkout. Re-run of the scale proof on the restored v42 archive is the closure gate regardless.","status":"in_progress","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T22:20:18Z","created_by":"Sinity","updated_at":"2026-07-19T23:33:22Z","started_at":"2026-07-17T02:16:36Z","labels":["area:sources","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","performance","raw-authority","scale-proof"],"dependencies":[{"issue_id":"polylogue-hjpx.2","depends_on_id":"polylogue-hjpx","type":"parent-child","created_at":"2026-07-16T00:20:17Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-hjpx.2","depends_on_id":"polylogue-hjpx.1","type":"blocks","created_at":"2026-07-16T00:20:39Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-hjpx.2","title":"Prove raw replay convergence at the July-15 archive shape","description":"hjpx AC6 remains unproven after the execution-foundation phase. Small unit fixtures exercise 25 singleton raws and skewed cohorts, but they do not match the 2026-07-15 preflight shape: roughly 15,264 direct / 21,398 expanded candidates, 10,163 authority components, 4.788 GB retained payload, 1,890 broken active seeds, 40 cursor-ahead sources, and 34 incomparable heads. A sanitized evidence harness must establish that the immutable planner and bounded executor converge without recreating the live daemon's memory/non-progress incident.","design":"Build a deterministic synthetic/sanitized corpus generator parameterized from the recorded yla8 preflight distributions: component sizes, revision skew, source-path moves, bundle memberships, cursor/head conflicts, missing blobs, and resource-blocked cohorts. Run the real source/index/ops routes under Sinnix containment; measure pass-by-pass plan census digests, executable backlog, carried/deferred/terminal counts, RSS/PSS, swap, temp/database writes, wall time, daemon health latency, cursor/head invariants, and FTS closure. The proof is an executable benchmark/scenario with machine-readable receipts and explicit envelope, not a mocked count test. Include mutation variants that remove fair rotation and conservation accounting and reproduce non-progress.","acceptance_criteria":"1. The sanitized corpus matches the July-15 candidate/component/byte/skew distributions within documented tolerances and contains no private content. 2. Every bounded pass stays within declared RSS/PSS, swap, temp/write, and wall-time envelopes while daemon health remains responsive. 3. Executable plan backlog decreases monotonically modulo explicit retry injection; every finite retry resolves once and no component starves. 4. Cursor positions, source heads, accepted index heads, FTS readiness, and durable raw authority never regress across interruption/resume. 5. Two final quiescent census digests match with zero executable plans and identical typed residual debt. 6. Removing fair rotation recreates starvation; removing conservation/carry-forward accounting recreates a census mismatch; the harness fails both mutations. 7. Exact commands, containment receipts, corpus seed, and result artifacts are durable and reviewable. 8. No live archive apply is part of this bead; yla8 retains the separate verified-backup and explicit-authorization gate.","notes":"2026-07-16 inherited scale-proof detail from hjpx.1 adversarial pass 5: census attempts are bounded by component count and each transitive authority component is already byte-bounded to 1 GiB; a crash before component completion safely restarts that component and never publishes a partial plan. This bead must decide/prove the production-shape resource envelope and, if necessary, add sub-component parser checkpointing without weakening immutable plan publication.\n\n2026-07-17 static resource-path audit (no live mutation): the 1 GiB admission ceiling is not a policy-only blocker. revision_backfill._parse_retained_raw calls ArchiveStore.raw_revision_material, which calls BlobPublisher.read_all; _parse_one then wraps the full bytes in BytesIO even for stream-record providers. The blob store already exposes open(hash), and parse_stream_payload accepts an iterator, so Hjpx.2’s concrete implementation seam is a retained-raw metadata/open API plus a stream parse path for stream-record providers. Preserve component-atomic census/plan publication and spill semantics; replace only the eager blob read for safe stream providers. Non-stream/bundle formats remain bounded/admitted until separately safe. The existing \u003e1 GiB tests correctly prove fail-closed today and should evolve into anti-vacuity tests: a stream raw must parse without read_all, while an unsafe/non-stream cohort still blocks before open.\n2026-07-17 implementation foundation merged: PR #2966 / master 805d49286 removes eager blob materialization for retained JSONL replay in historical backfill, live append replay, and full/membership replay. It deliberately preserves the 1 GiB per-authority-component admission gate because Codex/Claude parser output remains materialized; 150 focused authority/backfill/live/repair tests passed, quick static/generated gate passed, and an independent adversarial close review found no remaining gap. This is a prerequisite improvement, not HJpx.2 closure: the remaining work is the July-15-shaped executable scenario with measured RSS/PSS/swap/I/O/wall-time, fairness/conservation mutations, and two fixed-point censuses.\n2026-07-17 scale execution started: contained scheduler-shape run uses merged workload-receipt:sha256:0d12c4e3186e98f1c8e25a6f62b6110eb6bfe78068c550c06ab9df38cc912d95 / PR #2967 against 10,163 components and 15,264 direct raws with a 128-component pass limit; outputs are being retained at /realm/tmp/raw-authority-july15-projection/. This proves real planner/executor cardinality only. It is explicitly not yet the 4.788 GiB materialized-byte or 21,398 expanded-candidate proof, so it cannot satisfy the scale bead by itself.\n2026-07-17: PR #2970 merged raw-authority scale-proof runner. It now batches blob/source acquisition safely, content-addresses the generated corpus, records process RSS/PSS/swap/CPU/I/O evidence, refuses contended hosts, and proves quiescence by two ledger-native dry-run censuses. A 256/384 medium run was deliberately terminated before receipt emission because a concurrent full-corpus schema generator held the host at \u003e5% I/O full-stall and the proof process entered D-state during acquisition; no scale result is claimed. Full July-shape execution remains pending a normally admitted host.\n2026-07-17: discovered and linked polylogue-3jlg after observing the concurrently running full-corpus schema generator read ~244.9 GiB/write ~295.1 GiB in ~73 minutes before final output. This is tracked as a separate evidence-first replay-amplification investigation; it does not change Hjpx.2 completion criteria.\n2026-07-17 current-workload evidence: a safe 3-component admission probe of `devtools workspace raw-authority-scale-proof` refused to start at I/O full avg10=12.11 (configured limit 2.00); no override was used. Static audit also found the runner accepts only components/raws/pass_limit and generates tiny fixed JSONL payloads. It cannot currently express or verify the required 4.788 GiB byte envelope, expanded-candidate/topology skew, conflict/blocked cohorts, or full July-15 distribution. Treat runner capability completion—not simply waiting for a quiet host—as the immediate Hjpx.2 implementation prerequisite; do not present the existing 10,163/15,264 scheduler-shape receipt as full-scale proof.\n2026-07-17: PR #3009 merged (314bcb011c): runner now accepts the aggregate-only raw-authority scale profile, streams a blob-backed synthetic corpus to the requested byte envelope, records requested versus achieved frontier shape, and provides a preparation-only receipt for expanded terminal/deferred topology. Focused runner tests (5) and quick verification passed. This is intentionally not a full-scale closure: expanded cohorts require explicit terminal/deferred outcome variants before an executable fixed-point claim; fairness/conservation mutations and an admitted July-envelope run remain.\n2026-07-17 read-only follow-up: invoking the new live --capture-profile against the active archive remained CPU-running for over a minute and did not emit its output under concurrent daemon/test load; two accidental coordinator duplicate readers were stopped without writing archive state. This confirms the full-profile capture needs its own bounded/resumable aggregate route or a quiescent maintenance window before it can be used as an operational preflight.\n2026-07-17 scale-proof progress: PR #3011 made the live aggregate profile complete in ~3.4s without archive mutation. PRs #3015, #3017, and #3020 then hardened corpus generation (published-blob prefix continuation, private-free exact component/direct-candidate cohorts, distinct synthetic direct identities). The contained exact-cohort preparation receipt is /realm/tmp/raw-authority-scale-cohort-current-20260717.stdout.json: requested=achieved 36,401 components, 41,450 direct candidates, 46,979 expanded candidates, 2,036,129,455 bytes; cohorts 36,399x(1 direct/1 raw), 1x(2,856 direct/3,885 raw), 1x(2,195 direct/6,695 raw). It deliberately records no private ids/paths/hashes/content and does not mutate/replay the live archive. Remaining AC gaps: synthetic terminal/deferred cohort execution, fairness/conservation mutation proofs, measured full replay envelope/daemon-health and fixed-point evidence.\n2026-07-17 PR #3021 added executable typed cohort variants to the scale runner. Small real-repair scenarios now prove both an ambiguous sibling terminal outcome and a deferred sibling outcome, each followed by two matching dry census digests; receipts include per-pass outcome-status counts. This closes the prior preparation-only restriction for explicit typed cohorts, but not the full current-profile execution/mutation/resource-health ACs.\n2026-07-17 mutation closure progress: PR #3029 makes any nonzero immutable replay-plan conservation error fail closed; its real-route mutation regression corrupts the after-pass algebra and asserts success=false. PR #3031 removes durable attempt-age fairness in a one-slot, retry-injected scenario and reproduces starvation; the production scheduler instead advances to the next independent component. Remaining scale-proof scope is byte-skew fidelity, interruption/cursor-head/FTS/daemon-health evidence, and an admitted full executable profile receipt.\n2026-07-17: PR #3034 merged (d108e9431). The real interruption/recovery path now snapshots after the post-write injected failure and proves recovery preserves source raw authority fields, accepted revision head/session pointer, FTS readiness, and actual FTS hits while finalizing the interrupted ledger census. This narrows AC4 at the retained-raw/revision-head/read-model boundary; source-file ingest cursor evolution and live daemon responsiveness remain scale-run evidence, not claimed by this unit proof.\n2026-07-17: PR #3037 merged (b8c29e679). `raw_materialization_scale_profile` now emits a private-free joint cohort of component raw count, direct candidate count, component blob-byte power-of-two bucket, and count; the scale runner consumes it, validates topology marginals, and allocates exact total bytes inside those buckets. This closes the prior byte-skew fidelity implementation gap in AC1. A new current-profile capture and admitted executable receipt are still required; old v1 captures remain readable but naturally lack the additive joint cohort.\n2026-07-17 contained preparation attempt against current v2 profile: host admission began at I/O full avg10=0.01, memory full=0.00. After 38s / ~159MiB synthetic output, runner entered D-state and I/O full avg10 rose to 5.36; coordinator stopped its own background scope, which left ~273MiB disposable partial synthetic output and no receipt. Diagnosis: byte-cohort allocator preserved a component total by placing all remaining bytes in its final raw, turning the real 6,695-member ~2GiB component into one synthetic ~2GiB blob. Follow-up implementation is required before retry: distribute explicit-cohort bytes across independent member raws and enforce pressure checks between bounded publication batches. No live archive was read/written beyond the earlier read-only profile capture.\n2026-07-17: PR #3038 merged (695fedbc7) after the stopped v2 preparation diagnosis. Explicit-cohort byte budgets now distribute across independent member raws instead of concentrating in one terminal blob; generation rechecks I/O/memory pressure after each 128-row publication batch and returns all safe samples in receipts. Focused runner suite (11) and quick verification passed. The next contained preparation must use this merged runner; no claim yet about its outcome.\n2026-07-17 second contained v2 preparation used PR #3038. It self-aborted at the first batch checkpoint with I/O full avg10=4.51 \u003e 2.00, rather than entering D-state; no receipt was emitted and partial synthetic output is being removed. This validates continuous enforcement but reveals remaining generator churn: explicit cohorts still stage tens of thousands of sub-megabyte payload files before publication. Next implementation removes that duplicate disk staging for independent explicit cohorts by publishing generated bytes directly; prefix-sharing cohorts retain the streaming file path.\n2026-07-17: PR #3040 merged (0829f65e0). Production-shaped explicit cohorts now construct bounded standalone JSONL bytes and call publisher.write_from_bytes directly; an anti-vacuity test rejects the old staged-path call. Prefix-sharing inputs retain streamed staging. Focused runner suite (12) and quick verification passed. Retrying current v2 preparation is now warranted with the existing continuous pressure gate.\n2026-07-17: PR #3043 merged (fix(repair): retain receipts for resource-deferred replay). Raw authority now reports executable versus resource-deferred candidate debt only when relevant; repeated all-deferred apply passes reuse the exact-scope durable receipt without ledger spam; dry runs still publish two matching fixed-point censuses. Focused proof: 13 passed; raw-materialization selection: 35 passed/17 deselected; devtools verify --quick succeeded. This removes the harness semantic blocker where a legitimate envelope-deferred residual was mistaken for incomplete convergence.\n2026-07-18 lane-D v2 corpus-prep retry, attempt 1: sinnix-scope background -- devtools workspace raw-authority-scale-proof --components 10163 --raws 15264 --expanded-raws 21398 --pass-limit 1000 --keep --json self-aborted at the FIRST admission check (before any generation work): I/O pressure gate refused with full avg10=3.06 \u003e 2.00 (host avg10 was 4.38-4.78, avg60 7.72-8.35, avg300 7.24-7.89 at the time -- 4+ other warroom lanes actively running, matching SONNET-NOTE expectations). This is a valid, expected self-abort per the gate design; the gate was not loosened and the corpus was not shrunk. Receipt: /realm/tmp/raw-authority-july15-v2-20260718/attempt1.stderr.log. Launched a bounded (40-min, 60s-poll) wait-then-retry wrapper in the background (/realm/tmp/raw-authority-july15-v2-20260718/wait_and_run.sh) that runs the identical command once io_full_avg10\u003c=2.00 or the deadline passes (in which case it runs anyway to record a second honest self-abort receipt rather than idling indefinitely).\n2026-07-18 lane-D corpus-prep retry, attempt 2 (bounded wait-then-run wrapper): after a 26-minute poll (60s interval, 27 polls, io_full_avg10 ranging 1.40-16.96 -- host stayed persistently contended across 4+ concurrent warroom lanes for the entire window), a brief quiet tick (avg10=1.40) passed initial admission and generation began. The CONTINUOUS pressure gate then correctly self-aborted mid-generation at check_generation_pressure() (devtools/raw_authority_scale_proof.py:781) with avg10=2.46 \u003e 2.00, before any component/member payload accumulated enough to leave a directory behind (verified: no partial /realm/tmp/*/raw-authority-scale-proof residue from this attempt). This is the SECOND consecutive valid self-abort this session (attempt 1: refused at the very first admission check, avg10=3.06; attempt 2: passed admission, self-aborted mid-generation, avg10=2.46). Both prove the gate enforces continuously and correctly under sustained real contention; neither is a scale-shape or correctness failure. Receipts: /realm/tmp/raw-authority-july15-v2-20260718/{attempt1,attempt2}.stderr.log, wrapper.log (full poll history). Host has not had a single 60s-sampled quiet window (avg10\u003c=2.0) longer than one tick in 40+ minutes of observation this session -- genuinely saturated, not a fluke. Launching a third, longer-bound (90 min) retry in the background; continuing other lane-D work (proof-report skeleton, confirming #3080 interruption/resume coverage) while it runs.\n2026-07-18 lane-D corpus-prep retry, attempt 3: wrapper required 3 CONSECUTIVE quiet polls (io_full_avg10\u003c=2.00, 60s apart) before launching, reasoning that a single-tick quiet window (attempt 2) was insufficient to survive generation. Found one at poll#12 (18:39 CEST) after 12 minutes of oscillating pressure (0.02-7.60). Self-aborted again, this time further into generation (devtools/raw_authority_scale_proof.py:853, inside the publish-batch flush path, versus line 781 on attempt 2 -- i.e. it survived past at least one _PUBLISH_BATCH_SIZE flush cycle this time) at avg10=3.63\u003e2.00. Three consecutive honest self-aborts this session (3.06 at admission / 2.46 early-generation / 3.63 mid-generation-past-first-flush), each showing the gate working correctly and each getting incrementally further, but the host has not sustained a quiet window long enough to complete the full 21,398-row generation phase in ~80 minutes of observation across 3 attempts. This is consistent with SONNET-NOTE 2026-07-18s explicit expectation of 4+ concurrent warroom lanes causing real contention, not a harness defect. Receipts: attempt{1,2,3}.stderr.log, wrapper{,2}.log under /realm/tmp/raw-authority-july15-v2-20260718/. Launching a 4th, more patient attempt (2h bound, 5 consecutive quiet polls required) while finalizing the proof report with what is provable regardless of this attempts outcome.\n2026-07-18 lane-D corpus-prep retry, attempt 4 (final this session): required 5 CONSECUTIVE quiet polls (5 min sustained, io_full_avg10\u003c=2.00) before launching -- found at poll#36 (19:15 CEST) after avg10 sequence 1.97/1.25/0.32/1.56/0.13. Self-aborted again at the SAME line as attempt 3 (raw_authority_scale_proof.py:853, mid-generation past the first publish-batch flush) at avg10=2.36\u003e2.00, despite the 5-minute sustained-quiet precondition. This is a significant diagnostic finding: the abort recurring at the identical code location across two attempts, immediately after 5 minutes of genuine external quiet, suggests the GENERATION PHASE ITSELF (writing/flushing the first _PUBLISH_BATCH_SIZE batch of raw payload files) is I/O-intensive enough to self-trigger the gate on this host, not purely a function of other lanes contentions -- external quiet alone does not guarantee survival past the first flush. Four consecutive honest self-aborts total this session (3.06 admission / 2.46 early-gen / 3.63 past-first-flush / 2.36 past-first-flush-again), spanning ~140 minutes of observation across 4 attempts with three different wait strategies (immediate / 40min-1-tick / 90min-3-tick / 2h-5-tick). Stopping retries this session -- diminishing returns from further identical-strategy attempts, and the finding itself (generation-phase self-induced pressure) is more valuable to record than a 5th blind retry. Receipts: attempt{1,2,3,4}.stderr.log, wrapper{,2,3}.log (full poll history with PSI samples) under /realm/tmp/raw-authority-july15-v2-20260718/. Full findings in .agent/reports/hjpx2-july15-scale-proof-2026-07-18.md (commit c2d3be71a, being updated this pass). hjpx.2 remains in_progress, NOT closable: AC1/AC6/AC7 (July-15-scale execution itself) unproven this session; AC2 (envelope, proven at small scale) unmeasured at requested cardinality; AC2 (daemon-health) unprovable with current harness (polylogue-agvo filed); AC3/AC5 (fairness/fixed-point mechanisms) proven via existing regression tests but not exercised at requested cardinality; AC4 (interruption/resume) proven via #3080, cited not duplicated.\n2026-07-18 lane-D: PR #3122 opened (https://github.com/Sinity/polylogue/pull/3122) carrying the proof-status report + yla8 packet + 9p8x fix. hjpx.2 itself remains open (not closable -- see report and prior notes for the AC gap).\n2026-07-20: the scale-proof report referenced from this bead was untracked from the repo by the .agent excision (PR #3180, operator directive); it persists on the operator host under .agent/reports/ in the main checkout. Re-run of the scale proof on the restored v42 archive is the closure gate regardless.\nRE-MEASURED 2026-07-28: the description's raw-frontier figures (15,264 candidates, 21,398, 10,163, 1,890 broken active seeds) are from the July-15 shape. Live daemon journal now reports:\n\n 'Raw replay planning paused until the persisted parser census completes for 2,593 relevant raw(s)'\n (2,737 at 20:44 -\u003e 2,593 at 21:39, i.e. draining ~144/hour rather than growing)\n\nsource.db holds 41,363 raws / 17,152 distinct native_ids; index holds 18,871 sessions. Unparsed: claude-code 5,906, codex 2,913, claude-ai 2,706, chatgpt 427, hermes 24, gemini-cli 9, unknown-export 34.\n\nThe monotonic GROWTH that justified hjpx's P0 escalation (11,717 -\u003e 15,264) is not the current behaviour; the candidate set is shrinking. Re-establish whether the escalation condition still holds before treating this as an active regression.","status":"in_progress","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T22:20:18Z","created_by":"Sinity","updated_at":"2026-07-28T20:03:35Z","started_at":"2026-07-17T02:16:36Z","labels":["area:sources","area:storage","area:test","delivery:A-trust-floor","horizon:frontier","performance","raw-authority","scale-proof"],"dependencies":[{"issue_id":"polylogue-hjpx.2","depends_on_id":"polylogue-hjpx","type":"parent-child","created_at":"2026-07-16T00:20:17Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-hjpx.2","depends_on_id":"polylogue-hjpx.1","type":"blocks","created_at":"2026-07-16T00:20:39Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-b054.1.1","title":"Make fresh-checkout affected verification bounded and baseline-aware","description":"A fresh checkout cannot run the project's required affected-test verification until pytest-testmon has been seeded, but the only advertised seed path currently expands to the entire ~16k-test corpus and is not a reliable bootstrap contract. On 2026-07-15, devtools verify --seed-testmon --skip-slow passed all static/generated gates, then accumulated a large inherited failure/error set, reached 94%, and stalled on tests/unit/daemon/test_daemon_cli.py::test_periodic_db_optimize_targets_archive_root_tiers. The contained scope consumed about 17.5 GB peak memory, 9.3 GB swap, and 11 GB writes before being interrupted after more than 12 minutes; it produced no usable seed receipt. This makes fresh worktree verification unavailable precisely where isolated Terra/Codex lanes need it and encourages either blanket-suite abuse or unverifiable PRs.","design":"Own the invariant at the devtools/test harness boundary: a clean checkout must be able to establish a trustworthy affected-test dependency graph under a declared resource/time envelope even when the repository baseline contains known failing, slow, or hanging nodes. Separate dependency collection from the claim that all tests pass. Persist a versioned baseline/seed receipt that records collection completeness, failed/errored/timed-out nodes, dependency coverage, environment fingerprint, and resumable checkpoints; do not discard the whole graph because some nodes fail. Add per-node/process-group timeout and deterministic worker cleanup, bound concurrency/memory/write amplification, and make the command explain whether the resulting graph is safe for affected selection. Known baseline failures must be explicit durable quarantine/debt with expiry/owner, not silently ignored. Diagnose the periodic_db_optimize hang as one witness, but do not reduce this bead to patching that test. Preserve anti-vacuity: an affected run must still select a real production-dependent test when its implementation dependency changes.","acceptance_criteria":"1. On a clean checkout with no testmon state, one documented command produces a versioned seed receipt and a usable affected-selection graph within a declared wall-time, RSS/swap, and write envelope. 2. Injected pass, fail, error, timeout, and worker-crash nodes all appear explicitly in the receipt; none can silently truncate or invalidate unrelated dependency coverage. 3. Interrupted seeding resumes from a durable checkpoint or restarts after deterministic process-tree/temp cleanup without accepting a partial graph as complete. 4. A known-baseline manifest/quarantine has owners and expiry, cannot authorize a newly failing node, and the current repository reaches zero quarantined baseline failures: two consecutive clean-worktree seed runs finish green. 5. Per-node/process-group deadlines prevent the periodic optimize/WAL/embedding witnesses or an equivalent stuck node from hanging the seed; each current witness completes under ten seconds in ten isolated and xdist repetitions after its production or fixture lifecycle bug is fixed. 6. A production dependency mutation/removal causes the affected gate to select and fail its real-route test; changed executable code cannot silently select zero. 7. Fresh-worktree and warm-cache runs emit machine-readable receipts consumed by devtools verify and agent guidance, with one tested configuration contract across focused, xdist, seed, affected, and CI modes. 8. A representative non-slow seed records wall time, peak RSS/PSS, swap, and writes; compared with the 2026-07-15/16 baselines it is at least 2x faster and remains under 3 GiB peak RSS on this host without reducing test or capability coverage, or records the measured dominant blocker and a named follow-up if that physical target proves impossible.","notes":"2026-07-16 scope convergence: polylogue-88jp.1 was created from a second independent failed seed before this bead was surfaced. This bead is the stronger invariant owner and now absorbs its additional zero-baseline, repeated-witness, shared-contract, affected-selection anti-vacuity, and 2x/\u003c3GiB performance proof. Second evidence run 20260716T042022Z-seed-testmon-3415508-b1779b54 reached 91%, emitted many F/E results, accumulated about 4 GiB worker/controller RSS, and stopped progressing with periodic WAL and embedding catch-up nodes active. The two incidents show the same class, not two schedulable bugs.\n2026-07-16 implementation evidence. The harness now chooses up to 12 workers adaptively from CPU, MemAvailable, memory PSI, and a 768 MiB/worker budget; refuses below 1 GiB; uses bounded tmpfs with a 512-2048 MiB budget; never silently falls back to disk; samples process-tree RSS/PSS and tmpfs use; terminates on budget overrun; and deterministically removes direct pytest basetemps while preserving reusable seeded caches. Interrupted seed attempts have versioned running/incomplete/complete receipts and resume only when the tracked worktree fingerprint and corpus-shaping inputs match. This fixed a reproduced stale-ledger failure where a changed worktree passed 15,942 current tests but could never satisfy 48 deleted/renamed nodeids from the earlier attempt. Ordinary affected verification then selected 7 real tests and passed in 38.4s total.\n\nExact 8-worker full gate 20260716T102535Z-full-178126-782b2a77: 15,937 passed, 1 skipped in 146.02s pytest / 150.94s harness; isolated load-sensitive lane 34 passed in 54.00s pytest / 58.43s harness; full static+generated+test gate 253.04s; peak bulk PSS 5.21 GiB; cleanup complete. Exact post-fix seed runs 20260716T110545Z-seed-testmon-276488-876e13b5 and 20260716T111053Z-seed-testmon-292320-d4778f9e were consecutive green with 15,942 passed + 1 skipped each, 218.47s and 218.39s pytest, 289.83s and 258.28s full verification, peak PSS 5.88 and 5.91 GiB, and complete cleanup receipts.\n\nRepeated seeding also exposed a production MCP telemetry pathology: the global dispatcher discarded the root named by each wake hint, scanned every historical root in insertion order, and never released drained roots. Stale/unreachable archives could therefore delay current durable call acknowledgements beyond 5s and grow retained state indefinitely. The dispatcher now prioritizes the woken root and releases drained roots with a race-safe filesystem recheck. A production-route regression with 12 stale roots delayed 100ms each proves a live call is acknowledged within 750ms and its root is released; the old algorithm necessarily failed that bound. The full MCP call-log xdist file passes 17/17.\n\nAC8 sub-3-GiB aspiration was not met at 8 workers: peak samples show about 1.05 GiB controller PSS plus 0.53-0.67 GiB per active worker. Operator preference is to retain throughput and spend several GiB rather than reduce workers or use disk. Follow-up polylogue-b054.1.1.2 owns memory-amplification profiling/reduction without throughput loss. One earlier seed showed a non-reproduced demo construct-coverage loss; enhanced diagnostics and follow-up polylogue-b054.1.1.1 preserve that residual rather than quarantining it. No baseline failure is authorized.\nFinal warm-gate audit found and repaired a second-order anti-vacuity defect: verify compared zero selection against the entire branch diff, so a successful affected run updated testmon and made every subsequent unchanged run fail forever. The gate now accepts zero only when an exact worktree-content receipt from a complete seed or a prior successful affected run exists; any executable content or changed-path set invalidates the receipt. Focused receipt/invalidity tests pass 3/3.\nFinal rebased publish-boundary evidence: seed run 20260716T112938Z-seed-testmon-335241-c38361b4 passed 15,905 tests with 1 skip and zero failures on the exact origin/master-rebased commit; 408.68s end-to-end, 6,006.8 MiB peak process-tree PSS, complete receipt and cleanup. First warm affected run selected 7 real tests and passed in 39.43s. An unchanged second warm run selected zero and passed in 39.54s with zero_selection_coverage=complete_seed, directly proving repeatability. Browser-extension integration independently passed 15 files / 285 tests, ESLint, and manifest validation.\n2026-07-16 assured-close iteration 1: keep open. Independent audit of merged f0c1b489 found six legitimate residuals. (A2) seed receipts do not persist per-node pass/fail/error/timeout/worker-crash classes; aggregate report counts and supervisor cleanup tests are insufficient. (A3) resume identity hashes tracked diffs and untracked paths, but not untracked file contents, so editing an existing untracked executable/test can reuse a stale checkpoint. (A4) the two green seeds preceded the nondeterministic demo construct-loss failure; only one rebased green followed it, so child b054.1.1.1 must be repaired before two post-fix greens establish reliability. (A5) optimize/WAL/embedding witnesses lack the exact ten isolated plus ten xdist repetitions under ten seconds; 09rn covers embedding but no bead yet covers the omitted optimize/WAL lifecycle proof. (A6) zero-selection wrapper tests mock the gate and the seven-test warm run did not mutate a production dependency; a real testmon DB must prove a production mutation selects and fails its real-route test. (A1/A8) receipts sample host SwapFree but do not compute run swap delta/peak or read/write bytes and therefore do not declare or prove swap/write envelopes; timing comparisons must use like-for-like seed commands or name the blocker. The merged phase remains valuable and fully green, but it is not full closure of this bead.\n2026-07-17 shared Test Diet foundation increment: PR #2976 / e5d954f08 adds production-route proof that the seeded archive cache fails closed. A deleted published index triggers a full rebuild retaining the workload key/profile/recipe and independently generated facts; an injected ingest failure leaves neither artifact nor staging residue. Focused artifact tests 4 passed; quick gate receipt 20260717T063430Z-quick-1699270-dfa84c77 succeeded (16/16). This is necessary F1 evidence, not closure: b054.1.1.3 receipt/accounting and comparison proof plus b054.1.1.1/.4/.5 repeated-witness and real-mutation obligations remain.\nWarroom sweep It.17: claiming session closed; substantial landed through the seed-repair train (#2995-#3000). Children .1/.3/.4/.5 remain the open residue. Reset to open (portfolio-level).","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T21:53:29Z","created_by":"Sinity","updated_at":"2026-07-27T04:53:12Z","started_at":"2026-07-16T04:40:27Z","closed_at":"2026-07-27T04:53:12Z","close_reason":"Parent AC re-read against its 9 children: 7 already closed correctly (.1.1.1/.4/.5/.6/.7/.8/.9), .1.1.3 closed above. AC8 (2x speedup / \u003c3GiB peak) already self-invoked its own named-blocker escape clause in prior notes ('sub-3-GiB aspiration was not met... operator preference is to retain throughput... follow-up polylogue-b054.1.1.2 owns memory-amplification') - .1.1.2 is correctly an independent, non-blocking follow-up track, not a closure precondition, and remains open on its own separately-tracked scope (real unstarted profiling/optimization work). AC6 (production-mutation proof) satisfied by .1.1.4. AC4/AC5 (baseline zero-quarantine, witness timing) covered by .1.1.5-.1.1.9. AC1-AC3/AC7 (receipt completeness, resume identity, machine-readable contract) satisfied by .1.1.3's landed tooling. The parent's own 'Warroom sweep It.17' framing (children .1/.3/.4/.5 as residue) was stale - .1 and .4 already closed since that note, leaving only .3 (now closed) and .2 (correctly spun off, not blocking) as genuine remaining items.","labels":["agent-readiness","area:architecture","area:beads","horizon:frontier","invariant","verification"],"dependencies":[{"issue_id":"polylogue-b054.1.1","depends_on_id":"polylogue-09rn","type":"relates-to","created_at":"2026-07-16T06:40:27Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-b054.1.1","depends_on_id":"polylogue-b054.1","type":"parent-child","created_at":"2026-07-15T23:53:29Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-b054.1.1","depends_on_id":"polylogue-hjpx","type":"discovered-from","created_at":"2026-07-15T23:53:29Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-b054.1.1","depends_on_id":"polylogue-vyxq","type":"relates-to","created_at":"2026-07-16T06:40:29Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-b054.1.1","depends_on_id":"polylogue-wple","type":"relates-to","created_at":"2026-07-16T06:40:28Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-b054.1.1","depends_on_id":"polylogue-y6tb","type":"relates-to","created_at":"2026-07-16T06:40:28Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-lkrc.4","title":"Restore live multi-session divergence authority coverage","description":"The mandated raw-authority selector fails on clean origin/master before the historical replay reconciler runs: test_live_multi_session_divergence_reopens_raw_authority expects the first multi-session live JSONL batch to succeed, but _ingest_full_paths_sync returns first.jsonl in failed. This either means the fixture no longer enters the intended multi-session membership route or the production live route regressed. The red node currently makes broad authority verification noisy and hides later regressions.","design":"Reproduce the exact clean-master failure with the real LiveBatchProcessor route. Capture the internal _ArchiveFullWriteResult/failure reason and compare the fixture's _jsonl_provider_and_session_artifact=True setup with current source artifact semantics. If the fixture is stale, rebuild it using the production acquisition classification that genuinely permits a multi-session payload; if production rejects a valid bundle, repair the substrate route without weakening single-session artifact validation. Preserve the intended sequence: first branch accepted, divergent second branch recorded ambiguous/nonterminal, safe members remain queryable, and a matching retry can resume authority. Do not merely change failed==[] to the observed failure.","acceptance_criteria":"1. The failure reproduces on origin/master and its exact internal failure reason is recorded. 2. A production-route fixture reaches multi-session membership authority without mocks authorizing an impossible source shape. 3. First ingest succeeds; divergent second ingest is explicit nonterminal authority debt; safe members remain queryable; no accepted branch is deleted. 4. Removing the corrected route/fixture condition makes the test fail for the intended reason. 5. devtools test tests/unit/sources/test_live_batch_support.py -k live_multi_session_divergence_reopens_raw_authority and devtools test -k raw_authority pass.","notes":"Discovered during polylogue-hjpx verification on 2026-07-15. Exact node fails identically in detached clean origin/master 41cb11f87, so it is not caused by the historical replay/fair-scheduler change. Baseline assertion: first _ingest_full_paths_sync([first]) returned failed=[first].\n2026-07-16 GPT-Pro corpus adjudication: raw-authority package ee58f7411a93 merged as PR #2923 (81142d1dce7d8e896ef340783ab943cdf59143f6). Narrow accepted behavior: a live multi-session path may admit only its own complete taxonomy/parser-backed current raw candidate; no claim of the wider immutable census program.\n2026-07-16 implementation pass: owning the coherent lkrc/hjpx.1/lkrc.4 raw-authority cluster from fresh origin/master. Scope is the single reconciler/immutable-plan conservation and the production multi-session divergence regression now observed in packaged ordinary catch-up. Preserve yla8 fail-closed replay protections; no live cursor reset, force replay, evidence deletion, manual SQL repair, or live apply before reviewed code, verified backup, quiescent census, and explicit authorization. First deliverable is a production-route failing fixture and read-only live evidence.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T21:33:21Z","created_by":"Sinity","updated_at":"2026-07-16T19:47:06Z","started_at":"2026-07-16T19:20:43Z","closed_at":"2026-07-16T19:47:06Z","close_reason":"Merged PR #2957 (a62d2f972): the production live route now validates parser-drift replay against the persisted index CAS witness while requiring the accepted raw in the classified cohort. Focused live/divergence/quarantine regressions passed; quick gate 16/16. Deployed as Polylogue 0.2.0+a62d2f97 via Sinnix 4248ceb. No cursor reset or force replay was used.","labels":["area:browser","area:sources","area:storage","area:test","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-lkrc.4","depends_on_id":"polylogue-hjpx","type":"discovered-from","created_at":"2026-07-15T23:33:21Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-lkrc.4","depends_on_id":"polylogue-lkrc","type":"parent-child","created_at":"2026-07-15T23:33:20Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-3gd.3","title":"Install the agent integration kit through packages, clients, and Nix","description":"Polylogue releases the CLI, daemon, MCP server binary, NixOS module, and Home Manager daemon module, but it does not release the comprehensive agent-client integration required for routine effective use. Sinnix separately installs MCP profiles, a forked skill, and a stale SessionStart hook. Users can therefore run the server while their agents remain ignorant of its capabilities or are actively misled by drifted instructions. Make the project-owned MCP configuration and comprehensive standing manual installable, inspectable, upgradeable, and optional across supported clients.","design":"Package the 3gd.2 AgentIntegrationSpec, comprehensive generated manual, executable recipes, deep reference, and AgentIntegrationManifest with Polylogue. Provide non-destructive polylogue agent install/status/doctor/uninstall for supported clients such as Claude Code, Codex, Gemini, and Hermes: discover native client integration locations; install the comprehensive standing manual through SessionStart where supported and the closest persistent instruction mechanism elsewhere; expose deep reference and live catalog; merge a versioned MCP server entry with selected role; and verify all advertised routes against the installed server contract. Record ownership so update/uninstall changes only Polylogue-managed entries and never overwrites unrelated configuration or operator-authored instruction files. The upstream flake exports a per-user Home Manager agent-integration module separate from daemon lifecycle, with options for enable, clients, package, MCP role/profile, standing-manual delivery mode, reference visibility, archive/config identity, and explicit additions or exclusions. Configuration may let an operator reduce or disable guidance, but defaults optimize correct routine use rather than smallest context. Keep secrets out of generated world-readable files. Sinnix consumes upstream options and retains only profile and site policy.","acceptance_criteria":"1. Wheel/sdist/Nix packages contain the versioned comprehensive manual, executable recipes, deep reference, integration manifest, and client adapters; artifact version matches the CLI/MCP contract. 2. polylogue agent install/status/doctor/uninstall works in isolated temporary homes for Claude, Codex, Gemini, and Hermes, is idempotent, reports guidance/version/archive/role drift, and removes only manifest-owned entries without modifying operator-authored CLAUDE.md or AGENTS.md. 3. The upstream flake exports typed Home Manager options for clients, package, MCP role, standing-manual delivery, reference visibility, archive identity, and opt-down overrides; evaluation tests cover defaults, read/write profiles, disabled pieces, multiple clients, custom paths, cached-content stability, and secret-safe output. 4. Daemon modules remain daemon owners; agent integration does not implicitly start ingestion or grant write authority. 5. Sinnix consumes upstream artifacts with no independent Polylogue skill/manual/tool-name list, and parity checks catch downstream drift. 6. A clean-HOME smoke installs one supported client, receives the comprehensive standing manual without an extra lookup step, reaches the demo archive, and spontaneously uses Polylogue correctly on realistic unhinted tasks; no-daemon, wrong-archive, stale-index, and incomplete-coverage states produce truthful recovery. 7. Upgrade across two fixture versions updates generated contract-dependent sections while preserving operator additions and unrelated config; uninstall is lossless. Focused packaging/module/client tests and flake checks pass. 8. Installation reports standing-guidance size, cache-stable digest, and capability coverage. It does not enforce an arbitrary token ceiling; an opt-down mode must warn which tested behaviors or capability families it may impair.","notes":"2026-07-27: Verified the previously-named gap (\"does `nix build` actually produce an installable Home Manager module end-to-end from a clean flake eval\") -- YES, it does. Evidence:\n\n1. `nix build .#polylogue --no-link` (this repo's own flake, worktree at 4241316e0 = current master) builds `python3.14-polylogue-0.3.0` cleanly from a fresh nix store (fetched from cache.nixos.org where cached, built the polylogue wheel from source otherwise).\n\n2. Built a scratch consuming flake (not committed anywhere -- `/realm/tmp/.../hm-verify/flake.nix`, network-fetched `github:nix-community/home-manager` + this repo via `git+file://.../agent-a4ffcdbe55068249d`) that instantiates `home-manager.lib.homeManagerConfiguration` with `polylogue.homeManagerModules.agentIntegration` imported and `programs.polylogueAgent = { enable = true; package = polylogue.packages.x86_64-linux.polylogue; clients = [ \"claude-code\" \"codex\" ]; }`. `nix build .#homeConfigurations.verify.activationPackage --no-link` succeeded (exit 0), producing `/nix/store/fggnpqaxwc286s242zd7sjywwgibk9xz-home-manager-generation`.\n\n3. Inspected the built activation script (`$OUT/activate`): confirms the module's `home.activation.polylogueAgentIntegration` entry is present and wires the correct built-package binary paths:\n `_iNote \"Activating %s\" \"polylogueAgentIntegration\"` then\n `run /nix/store/.../bin/polylogue agent install --client claude-code --client codex --guidance full --server-command /nix/store/.../bin/polylogue-mcp --polylogue-command /nix/store/.../bin/polylogue --format json --reference --mcp --replace-clients`\n -- i.e. the typed HM options (clients, guidance, mcp/reference toggles) correctly lower into the real `polylogue agent install` CLI invocation against the actual built package's binaries, not a stub.\n\nThis closes the \"release-side packaging verification\" gap named in the 2026-07-18 note. It does NOT close the rest of this bead: per its own AC (items 2, 6, 7, 8) and the 2026-07-18 note's own caveat, \"the broader 'comprehensive agent-client integration required for routine effective use' claim...remains partially open pending live cold-agent trials\", and this session did not attempt those (isolated-temp-home idempotency/drift tests across all 4 clients, upgrade-across-two-fixture-versions, secret-safe-output audit, or any cold-agent behavioral trial). Leaving open for that residual scope.\n\nSide finding (unrelated, filed separately as polylogue-n2f4): `nix flake check` in this same worktree fails on `checks.x86_64-linux.format` (\"Failed to format tests: No such file or directory (os error 2)\") even though `tests/` is a real, fully-tracked directory. `gh run list --workflow=nix.yml --limit 30` shows 30/30 recent runs (back to at least 2026-07-13, across master and feature branches) failing -- this is pre-existing, longstanding CI debt, not introduced by PR #3061 or this session, and not fixed here (out of this bead's Home-Manager-module scope; the HM module verification above used plain `nix build`, not `nix flake check`, and is unaffected by this separate breakage).\n\nVerification commands run:\n nix build .#polylogue --no-link\n nix flake show\n nix build .#homeConfigurations.verify.activationPackage --no-link --print-build-logs (scratch consuming flake)\n nix flake check --print-build-logs (surfaced the pre-existing, unrelated format-check break; filed as polylogue-n2f4)","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T20:21:18Z","created_by":"Sinity","updated_at":"2026-07-27T02:19:15Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-37t"},"labels":["area:context","area:devloop","area:legibility","area:mcp","area:ops","delivery:D-agent-context-coordination","horizon:frontier","lane:agent-coordination","size:L","spine"],"dependencies":[{"issue_id":"polylogue-3gd.3","depends_on_id":"polylogue-3gd","type":"parent-child","created_at":"2026-07-15T22:21:18Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-3gd.3","depends_on_id":"polylogue-3tl.7","type":"relates-to","created_at":"2026-07-15T22:21:18Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-3gd.3","depends_on_id":"polylogue-pj8","type":"discovered-from","created_at":"2026-07-15T22:21:48Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-3gd.3","depends_on_id":"polylogue-t46.8","type":"relates-to","created_at":"2026-07-15T22:21:18Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-3gd.3","depends_on_id":"polylogue-z9gh.3","type":"relates-to","created_at":"2026-07-15T22:21:18Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} @@ -141,7 +142,7 @@ {"_type":"issue","id":"polylogue-37t.11.1","title":"Land the ContextSource scheduler, authority firewall, and ledger","description":"Provide the minimum production context-admission kernel that all recall, coordination, curriculum, advisory, and compaction sources can safely register against. This slice includes the instruction-authority firewall from day one, not as a later retrofit.","design":"Define ContextSource with moment, priority class, item ref/content, token cost, source-local ordinal score, expiry, trust/material class, and degrade path. schedule_context allocates deterministic fixed per-class and per-source quotas without comparing incomparable raw scores, assembles quoted evidence separately from executable policy, and records every candidate decision in an ops-tier injection ledger keyed to target session and resolved ExecutionContextRef. Only a valid explicitly operator-adopted AssertionKind.POLICY with scope, issuer authority, validation, expiry/revocation, recipient compatibility, and delivery receipt may enter the instruction partition. Tool/web/runtime prose is refs-only or visibly fenced quoted evidence. Migrate 37t.4 initial sections and one coordination/recall source through the real entrypoint.","acceptance_criteria":"1. ContextSource and schedule_context are the sole production admission entrypoint; migrated session-start/precompact paths no longer assemble independent memory lists. 2. Fixed deterministic class/source quotas never exceed the moment budget and same inputs/policy/build produce byte-identical assembly. 3. Ledger records included/degraded/dropped, source/item refs, token cost, source-local rank, budget state, disclosure verdict, authority verdict/reason, policy refs, target session, and ExecutionContextRef. 4. Ordinary adopted assertions and generated curricula remain fenced quoted evidence; only a valid explicitly adopted/scoped policy instructs. Revoked, expired, malformed, wrong-scope, self-authored/unadopted, tool/web/runtime, and injection-string fixtures fail closed. 5. Raw scores are ordinal within source only; no cross-source float comparison occurs. Removing the authority check, ledger write, fence, ref, or budget gate fails production-route tests; focused context/hook/MCP tests and quick verification pass.","notes":"Active-set expansion 2026-07-15: admitted as the context authority/judgment critical pair. Execution order remains canonical judgment transaction before the scheduler firewall.","status":"open","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:56:59Z","created_by":"Sinity","updated_at":"2026-07-15T19:19:41Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-37t"},"labels":["area:context","area:security","area:substrate","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-37t.11.1","depends_on_id":"polylogue-37t.11","type":"parent-child","created_at":"2026-07-15T20:57:00Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-37t.11.1","depends_on_id":"polylogue-37t.12","type":"blocks","created_at":"2026-07-15T20:57:00Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-37t.11.1","depends_on_id":"polylogue-37t.15","type":"blocks","created_at":"2026-07-15T20:57:01Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":2,"dependent_count":9,"comment_count":0} {"_type":"issue","id":"polylogue-2qx.1.2","title":"Migrate every current origin onto OriginSpec","description":"Apply the proven OriginSpec kernel to the complete current Origin vocabulary, preserving parser/detector behavior while eliminating parallel registration, public-token, coverage, and fixture inventories.","design":"Migrate all eleven current Origin tokens into polylogue/sources/origin_specs.py in detector-tightness order: claude-code-session, codex-session, gemini-cli-session, hermes-session, antigravity-session, beads-issue, grok-export, chatgpt-export, claude-ai-export, aistudio-drive, and unknown-export. For each, declare lifecycle, every supported file/stream/browser/Drive/bundle acquisition mode, detector/parser/assembly bindings, public/physical identity, construct and provenance/fidelity capabilities, coverage counters, fixtures, and reparse policy. Preserve sources/dispatch.py structural-first tightness and grouped/lowered payload behavior while deriving its registration/order or asserting exact parity. Replace the hand-written package-mode inventory in sources/provider_completeness.py with projections from OriginSpec; migrate public schemas/errors/completions, fixture census, and docs similarly, deleting duplicate lists after parity. grok-export is explicitly reserved with no parser until separately admitted; unknown-export declares fallback semantics rather than pretending executable completeness; beads-issue declares its non-chat artifact behavior. aistudio-drive records its many-to-one physical provider mapping. Claude Code and Codex expose typed assembly/orchestration/title/action extension hooks consumed by 2qx.2, j2zz, and ih67 without another admission registry.","acceptance_criteria":"1. Exactly these eleven tokens appear once in OriginSpec and exactly match core.enums.Origin: claude-code-session, codex-session, gemini-cli-session, hermes-session, antigravity-session, beads-issue, grok-export, chatgpt-export, claude-ai-export, aistudio-drive, unknown-export. Enum/registry additions or omissions fail generation. 2. Every executable token declares all actual acquisition modes and detector/parser/assembly/identity/construct/provenance/fidelity/coverage/fixture/reparse fields; grok-export is reserved with reason and no parser, unknown-export has explicit fallback semantics, and beads-issue has explicit non-chat semantics. 3. Dispatch tightness and recursive lowering remain behaviorally identical for ambiguous records, bundles, grouped JSONL, streams, browser captures, and Drive documents; deleting or reordering a declaration fails a real dispatch golden. 4. provider_completeness.py, public schemas/errors/completions, fixture census, and docs are derived from or mechanically parity-checked against OriginSpec, and parallel hand-maintained origin inventories are deleted after parity. 5. Non-injective physical-provider to public-origin mappings are explicit and tested; no public filter or payload regresses to Provider vocabulary. 6. Claude/Codex extension hooks are the only admission path used by 2qx.2, j2zz, and ih67; no private inventory is introduced. Focused source, completeness, render, fixture, dispatch mutation, and affected verification pass.","notes":"Active-frontier admission 2026-07-15: admitted as the executable current-origin migration prerequisite for mandate-critical orchestration admission polylogue-2qx.2.\nTerra-readiness correction 2026-07-15: enumerated the exact eleven-token migration, all special lifecycle cases, the hand-written provider_completeness inventory to retire, dispatch parity, and non-injective identity handling.","status":"closed","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:55:23Z","created_by":"Sinity","updated_at":"2026-07-21T19:16:24Z","started_at":"2026-07-21T18:23:44Z","closed_at":"2026-07-21T19:16:24Z","close_reason":"Merged via PR #3250 + follow-up #3252. Audit confirmed most scope already on master untagged (#3051/#3087/#3088/#3092/#3201/#3228/#3246): all eleven Origin tokens declared in origin_specs.py, provider_completeness a verified projection, dispatch-tightness + stream-parser parity mechanically checked. #3250 added the missing typed assembly admission hook (assembly_spec_path + validate_assembly_spec_parity vs live get_assembly_spec, with real mutation goldens). #3252 deleted the last parallel inventory (cli _ORIGIN_DESCRIPTIONS 8/11 hand dict) by deriving completion descriptions from a new required OriginSpec.display_description; regression test pins derived inventory to the full Origin enum. AC1-5 satisfied; AC6 misframed (2qx.2 already closed without OriginSpec; its blocking edge was force-closed as over-blocking — documented in PR body).","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:sources","area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-2qx.1.2","depends_on_id":"polylogue-2qx.1","type":"parent-child","created_at":"2026-07-15T20:55:23Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-2qx.1.2","depends_on_id":"polylogue-2qx.1.1","type":"blocks","created_at":"2026-07-15T20:55:23Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0} {"_type":"issue","id":"polylogue-2qx.1.1","title":"Land the OriginSpec admission kernel and conformance law","description":"Define the reusable typed source-admission contract and prove that one declaration can drive detection order, parser registration, public origin vocabulary, coverage, fixtures, and actionable completeness without absorbing provider-specific parser semantics.","design":"Add polylogue/sources/origin_specs.py as the sole typed Origin-domain registry over polylogue/declarations. OriginSpec declares public Origin token; lifecycle state executable, reserved, unsupported, or compatibility-only; accepted artifact/acquisition modes; detector callable and before/after tightness constraints; parser/stream parser/assembly entry points; physical provider and public-origin projection including many-to-one collision policy; normalized construct capabilities; authority/provenance and known fidelity loss; coverage counters; fixture ids; and semantic-reparse consequence. Keep actual detectors, parsers, and assembly implementations in their current source adapters. Derive or validate detect_provider ordering in sources/dispatch.py, parser registration, core/enums.py Origin completeness, public schema/error/completion values, provider_completeness rows, fixture census, and generated documentation. Pilot with claude-code-session as streaming/JSONL plus sidecars, chatgpt-export as document/bundle, and grok-export as reserved. Synthetic ambiguous detector precedence and missing adapter/fixture mutations must point back to one OriginSpec. Do not encode provider-specific record semantics in the kernel or treat Provider and Origin as one injective enum.","acceptance_criteria":"1. polylogue/sources/origin_specs.py defines the complete typed fields and lifecycle states and consumes polylogue/declarations without adding source semantics to that kernel. 2. Claude Code, ChatGPT export, and reserved Grok pilots derive or validate deterministic detector tightness, parser/assembly registration, public origin values, coverage/completeness rows, fixtures, docs, and reparse consequences from one declaration each. 3. sources/dispatch.py uses derived registration/order or is parity-checked by it; adding a synthetic executable origin requires one OriginSpec plus owning adapter and fixtures, not edits to parallel central inventories. 4. Missing parser, stream/parser conflict, ambiguous or cyclic detector order, absent fixture, undeclared coverage, leaked Provider token, and non-injective Provider-to-Origin collision each yield a source-locatable diagnostic and exact repair. 5. Provider implementation remains in its adapter, Origin remains the public query vocabulary, and Gemini/Drive-style many-to-one mappings require an explicit collision policy rather than accidental coercion. 6. Focused declaration, dispatch, public-schema, completeness, fixture, mutation, render, and quick verification pass.","notes":"Portfolio scheduling correction 2026-07-15: temporarily removed from active admission while hard prerequisite polylogue-o21.1 is admitted. This is scheduling only; the OriginSpec kernel remains on the mandate critical path.\nActive-set correction 2026-07-15: re-admitted after the operator rejected the arbitrary 15-leaf cap. Blocked near-next consumers remain visible alongside their admitted prerequisites; execution focus still derives readiness.\nTerra-readiness correction 2026-07-15: fixed origin_specs.py as the domain registry, named three structurally different pilots, preserved adapter ownership and detector tightness, and made non-injective Provider-to-Origin projection an explicit law.\nWarroom sweep It.17 (2026-07-18): claim orphaned -- the claiming session was closed 2026-07-17 and no matching commits exist on master since 2026-07-14. Reset to open; prior notes/receipts unchanged.","status":"closed","priority":1,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:55:21Z","created_by":"Sinity","updated_at":"2026-07-21T16:22:33Z","started_at":"2026-07-17T17:59:14Z","closed_at":"2026-07-21T16:22:33Z","close_reason":"Complete: kernel pre-existed on master (polylogue/sources/origin_specs.py, 882 lines over the o21.1 declarations kernel, covering all 11 Origin tokens — landed untagged via #3051/#3087/#3088/#3092/#3201/#3228, which is why the 2026-07-18 sweep reset this bead). PR #3246 (merged) closed the 3 audit-verified AC gaps: Provider-token leak guard on registration, validate_stream_parser_parity vs dispatch STREAM_RECORD_PROVIDERS, reserved-lifecycle contract proven synthetically (Grok graduated to executable via #3201). AC matrix: 1/3/4/5/6 satisfied; AC2 satisfied with the reserved pilot synthetic (no live reserved origin exists). 2qx.1.2 (parallel-inventory deletion audit) remains open scope.","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:architecture","area:sources","area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-2qx.1.1","depends_on_id":"polylogue-2qx.1","type":"parent-child","created_at":"2026-07-15T20:55:21Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-2qx.1.1","depends_on_id":"polylogue-o21.1","type":"blocks","created_at":"2026-07-15T20:55:22Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":10,"comment_count":0} -{"_type":"issue","id":"polylogue-1xc.14","title":"Declare workload envelopes and resource receipts once","description":"Polylogue measures costly work through incompatible one-off paths: query_memory_budget, pipeline probes, scenario execution, verify-run RSS, ingest throughput, source observations, the SLO catalog, and an append-cohort counter. That fragmentation let an MCP query process reach 8.5 GiB plus swap and a daemon catch-up process retain over 4 GiB anonymous memory without one comparable phase/resource receipt. Define one workload-envelope declaration and observation contract. It governs physical execution and evidence; it never imposes a semantic result cap or turns a valid large operation into permanently unsupported work.","design":"Define WorkloadEnvelopeSpec with stable workload/family identity, input/corpus distribution refs, phase model, process-tree/cgroup measurement scope, concurrency/admission shape, quiescence window, and dimensions for wall/CPU, current/peak RSS/PSS, anonymous/file-cache/swap, temp/storage and read/write I/O, response bytes, cancellation latency, progress, queue/backpressure, and cleanup. A WorkloadReceipt binds spec/version, build/runtime, archive/generation/frame, phase observations, measurement availability, budget verdicts, and evidence refs. Budgets declare measure-only, regression-gate, or containment semantics; exceeding them may schedule, page, stream, spill, pause, or resume but cannot create a semantic query/result limit. Consolidate existing collectors behind adapters rather than deleting domain phase instrumentation. Prove with the mandate query workload and watcher append/cohort catch-up, including peak versus quiescent and anon versus cache.","acceptance_criteria":"1. One typed WorkloadEnvelopeSpec and WorkloadReceipt represent workload/input identity, phase boundaries, build/archive/frame, process-tree and cgroup scope, wall/CPU, RSS/PSS anon/cache/swap, temp and read/write I/O, response bytes, cancellation/progress/backpressure, quiescence, and missing measurements. 2. Existing query-memory, pipeline-probe, scenario-execution, ingest/source-observation, verify-run, and SLO-catalog paths either emit the shared receipt or have an explicit adapter/exemption; unit conversion and process-scope semantics are tested. 3. The 2026-07-15 MCP query and 2026-07-13 watcher append/cohort incidents run as named canaries with comparable phase receipts that distinguish peak from retained/quiescent memory and anonymous charge from cache. 4. A valid oversized query remains logically answerable through scheduling/page/stream/spool/resume even when a physical budget is exceeded; a mutation that converts a budget into a semantic cap fails. 5. Regression gates compare like workload/input/build scopes, expose measurement unavailable separately from pass, and include anti-vacuity mutations for omitted child RSS, cgroup file cache, cancellation latency, and cleanup. 6. The common collector is bounded and does not perturb measured work by serializing the corpus or running parallel heavy readers.","notes":"Active-set expansion 2026-07-15: admitted as a high-leverage operational mechanism under the scale/raw-authority program; execution focus remains readiness- and conflict-aware.\n2026-07-16 schema-workload refinement: child polylogue-1xc.14.1 makes input/corpus distribution refs authoritative and executable. Provider observations produce a bounded privacy-safe WorkloadProfile; deterministic provider-native corpora then traverse production ingest/index/query routes and emit this bead shared receipts. This replaces handwritten realistic-fixture and one-off performance-scenario approaches without reducing scale or semantic ambition.\n2026-07-16 GPT-Pro corpus adjudication: workload/resource receipt package 1d287d6cd7c6 is blocked_but_seeded here. Retain physical measurement and no-semantic-cap rule; provider-network failure in historical ledger is not evidence that a later deliverable did not exist. Current schema-derived workload-profile child 1xc.14.1 is the authoritative next dependency.\n2026-07-16 foundation landed in PR #2934 commit 23e8b2933: deterministic real-pipeline seeded archive artifacts now publish atomically as immutable split-tier snapshots, carry stable archive/profile/build/recipe identity plus planted wire facts, and clone privately for mutating consumers. Legacy seeded_db fixtures were removed; C-03 now exercises generated Codex bytes through acquire→parse→materialize→index→query. This is substrate only: live real-archive regeneration/phase evidence and any resulting memory fix remain open.\n2026-07-27 (polylogue-a47769bba68869d49 session): correcting the \"substrate only\" characterization from the 2026-07-16 note -- this is more implemented than that framing suggested. WorkloadReceipt/WorkloadEnvelopeSpec (polylogue/scenarios/workload.py) are consumed by 6 devtools modules (query_memory_budget.py, verify.py, raw_authority_scale_proof.py, seed_receipt_compare.py, pipeline_probe/result.py, verify_slos.py) plus tests/infra/append_cohort_memory_counter.py. tests/unit/scenarios/test_workload_receipts.py has named canary specs for BOTH AC #3 incidents: exact_session_actions_canary_spec (2026-07-15 MCP query/C-03) and the append-cohort counter consumed by tests/integration/test_append_cohort_memory.py (2026-07-13 watcher catch-up), plus a passing anti-vacuity mutation test (test_physical_budget_cannot_be_expressed_as_a_semantic_result_cap).\n\nNot verified this pass, so NOT closing: AC #2 (every named path -- query-memory, pipeline-probe, scenario-execution, ingest/source-observation, verify-run, SLO-catalog -- either emits the shared receipt or has an explicit adapter/exemption, with unit-conversion/process-scope tests) needs an exhaustive per-path enumeration I did not have budget to complete confidently. AC #5/#6 (anti-vacuity mutations for omitted child RSS/cgroup file cache/cancellation latency/cleanup; bounded collector proven not to perturb measured work) also not independently re-verified. This bead is closer to closeable than \"substrate only\" implies but a confident AC-by-AC call needs a dedicated focused pass over devtools/verify.py + verify_slos.py + their mutation tests, not new implementation.","status":"open","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:45:44Z","created_by":"Sinity","updated_at":"2026-07-27T04:37:52Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:ops","area:perf","area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-1xc","type":"parent-child","created_at":"2026-07-15T20:45:44Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-20d.14","type":"relates-to","created_at":"2026-07-15T20:45:47Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-o21.1","type":"relates-to","created_at":"2026-07-15T20:45:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-s8gb","type":"relates-to","created_at":"2026-07-15T20:45:47Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-z9gh.1","type":"relates-to","created_at":"2026-07-15T20:45:46Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"polylogue-1xc.14","title":"Declare workload envelopes and resource receipts once","description":"Polylogue measures costly work through incompatible one-off paths: query_memory_budget, pipeline probes, scenario execution, verify-run RSS, ingest throughput, source observations, the SLO catalog, and an append-cohort counter. That fragmentation let an MCP query process reach 8.5 GiB plus swap and a daemon catch-up process retain over 4 GiB anonymous memory without one comparable phase/resource receipt. Define one workload-envelope declaration and observation contract. It governs physical execution and evidence; it never imposes a semantic result cap or turns a valid large operation into permanently unsupported work.","design":"Define WorkloadEnvelopeSpec with stable workload/family identity, input/corpus distribution refs, phase model, process-tree/cgroup measurement scope, concurrency/admission shape, quiescence window, and dimensions for wall/CPU, current/peak RSS/PSS, anonymous/file-cache/swap, temp/storage and read/write I/O, response bytes, cancellation latency, progress, queue/backpressure, and cleanup. A WorkloadReceipt binds spec/version, build/runtime, archive/generation/frame, phase observations, measurement availability, budget verdicts, and evidence refs. Budgets declare measure-only, regression-gate, or containment semantics; exceeding them may schedule, page, stream, spill, pause, or resume but cannot create a semantic query/result limit. Consolidate existing collectors behind adapters rather than deleting domain phase instrumentation. Prove with the mandate query workload and watcher append/cohort catch-up, including peak versus quiescent and anon versus cache.","acceptance_criteria":"1. One typed WorkloadEnvelopeSpec and WorkloadReceipt represent workload/input identity, phase boundaries, build/archive/frame, process-tree and cgroup scope, wall/CPU, RSS/PSS anon/cache/swap, temp and read/write I/O, response bytes, cancellation/progress/backpressure, quiescence, and missing measurements. 2. Existing query-memory, pipeline-probe, scenario-execution, ingest/source-observation, verify-run, and SLO-catalog paths either emit the shared receipt or have an explicit adapter/exemption; unit conversion and process-scope semantics are tested. 3. The 2026-07-15 MCP query and 2026-07-13 watcher append/cohort incidents run as named canaries with comparable phase receipts that distinguish peak from retained/quiescent memory and anonymous charge from cache. 4. A valid oversized query remains logically answerable through scheduling/page/stream/spool/resume even when a physical budget is exceeded; a mutation that converts a budget into a semantic cap fails. 5. Regression gates compare like workload/input/build scopes, expose measurement unavailable separately from pass, and include anti-vacuity mutations for omitted child RSS, cgroup file cache, cancellation latency, and cleanup. 6. The common collector is bounded and does not perturb measured work by serializing the corpus or running parallel heavy readers.","notes":"Active-set expansion 2026-07-15: admitted as a high-leverage operational mechanism under the scale/raw-authority program; execution focus remains readiness- and conflict-aware.\n2026-07-16 schema-workload refinement: child polylogue-1xc.14.1 makes input/corpus distribution refs authoritative and executable. Provider observations produce a bounded privacy-safe WorkloadProfile; deterministic provider-native corpora then traverse production ingest/index/query routes and emit this bead shared receipts. This replaces handwritten realistic-fixture and one-off performance-scenario approaches without reducing scale or semantic ambition.\n2026-07-16 GPT-Pro corpus adjudication: workload/resource receipt package 1d287d6cd7c6 is blocked_but_seeded here. Retain physical measurement and no-semantic-cap rule; provider-network failure in historical ledger is not evidence that a later deliverable did not exist. Current schema-derived workload-profile child 1xc.14.1 is the authoritative next dependency.\n2026-07-16 foundation landed in PR #2934 commit 23e8b2933: deterministic real-pipeline seeded archive artifacts now publish atomically as immutable split-tier snapshots, carry stable archive/profile/build/recipe identity plus planted wire facts, and clone privately for mutating consumers. Legacy seeded_db fixtures were removed; C-03 now exercises generated Codex bytes through acquire→parse→materialize→index→query. This is substrate only: live real-archive regeneration/phase evidence and any resulting memory fix remain open.\n2026-07-27 (polylogue-a47769bba68869d49 session): correcting the \"substrate only\" characterization from the 2026-07-16 note -- this is more implemented than that framing suggested. WorkloadReceipt/WorkloadEnvelopeSpec (polylogue/scenarios/workload.py) are consumed by 6 devtools modules (query_memory_budget.py, verify.py, raw_authority_scale_proof.py, seed_receipt_compare.py, pipeline_probe/result.py, verify_slos.py) plus tests/infra/append_cohort_memory_counter.py. tests/unit/scenarios/test_workload_receipts.py has named canary specs for BOTH AC #3 incidents: exact_session_actions_canary_spec (2026-07-15 MCP query/C-03) and the append-cohort counter consumed by tests/integration/test_append_cohort_memory.py (2026-07-13 watcher catch-up), plus a passing anti-vacuity mutation test (test_physical_budget_cannot_be_expressed_as_a_semantic_result_cap).\n\nNot verified this pass, so NOT closing: AC #2 (every named path -- query-memory, pipeline-probe, scenario-execution, ingest/source-observation, verify-run, SLO-catalog -- either emits the shared receipt or has an explicit adapter/exemption, with unit-conversion/process-scope tests) needs an exhaustive per-path enumeration I did not have budget to complete confidently. AC #5/#6 (anti-vacuity mutations for omitted child RSS/cgroup file cache/cancellation latency/cleanup; bounded collector proven not to perturb measured work) also not independently re-verified. This bead is closer to closeable than \"substrate only\" implies but a confident AC-by-AC call needs a dedicated focused pass over devtools/verify.py + verify_slos.py + their mutation tests, not new implementation.\nREFERENCE CORRECTION 2026-07-28: '(polylogue-a47769bba68869d49 session)' in these notes is an agent SESSION id, not a bead id. Same wording appears on 1xc.14.1, 1xc.14.1.1, 1xc.14.1.2 and 1xc.14.1.3 and is flagged by backlog-hygiene X2 on all five; none is a dangling bead reference.","status":"open","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:45:44Z","created_by":"Sinity","updated_at":"2026-07-28T20:05:30Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:ops","area:perf","area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-1xc","type":"parent-child","created_at":"2026-07-15T20:45:44Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-20d.14","type":"relates-to","created_at":"2026-07-15T20:45:47Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-o21.1","type":"relates-to","created_at":"2026-07-15T20:45:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-s8gb","type":"relates-to","created_at":"2026-07-15T20:45:47Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.14","depends_on_id":"polylogue-z9gh.1","type":"relates-to","created_at":"2026-07-15T20:45:46Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":2,"comment_count":0} {"_type":"issue","id":"polylogue-9e5.31.1","title":"Land the DefinitionClosureGraph kernel and representative policies","description":"Turn the completed wiring census method into an executable verification mechanism. Land the typed closure-policy/evidence graph kernel and prove it on representative storage/lifecycle, event, declaration/registry, query, and semantic-operation families so missing production wiring becomes a failing invariant rather than a future audit discovery.","design":"Build on the existing ArtifactGraph, OperationSpec catalog, live surface registries, DDL/AST inventories, and generated-contract infrastructure. A ClosurePolicy references an authoritative inventory and declares required edge kinds plus intentional-absence authority; it does not copy domain definitions. The evaluator emits stable definition refs, actual evidence refs, typed missing/bypass/tests-only/divergent outcomes, and bounded diagnostics. Seed representative policies: one durable data family, one event/write-effect family, one registry/declaration family, one query parse-to-render path, and one CLI/MCP/HTTP/Python semantic operation. Provide mutation-sensitive fixtures and a resource-bounded devtools entrypoint.","acceptance_criteria":"1. ClosurePolicy and DefinitionClosureGraph types express authoritative inventory ref, required edge kinds, evidence refs, exception authority, status, and repair diagnostic without a universal domain registry. 2. Representative storage/lifecycle, event, registry/declaration, query, and cross-surface operation policies evaluate against production registries/source and expose a durable JSON/matrix result. 3. Mutations deleting a producer, substituting a tests-only consumer, bypassing a shared substrate, dropping a lifecycle edge, and creating divergent twins each fail with the exact definition and missing edge. 4. Empty/synthetic and live-augmented runs distinguish unavailable evidence from satisfied/intentional closure. 5. The entrypoint has bounded enumeration and memory, is wired into the appropriate verification gate, and focused tests plus devtools verify --quick pass.","notes":"Active-set expansion 2026-07-15: admitted as the permanent definition-to-production closure kernel; broad adoption remains a later slice.\n2026-07-16 GPT-Pro corpus adjudication: DefinitionClosure package 4ddd843c064b remains blocked_but_seeded here. Preserve closure-policy and witness design, but wait for the single DeclarationSpec kernel polylogue-o21.1; do not invent a parallel declaration registry.","status":"open","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:40:52Z","created_by":"Sinity","updated_at":"2026-07-16T13:05:00Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-9e5.31"},"labels":["area:architecture","area:audit","area:devtools","area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-9e5.31.1","depends_on_id":"polylogue-9e5.31","type":"parent-child","created_at":"2026-07-15T20:40:52Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"polylogue-60i5.1","title":"Land the durable change-train manifest and lifecycle gate","description":"The durable-tier epic currently has lint and schema-drift children but no slice that implements its actual authority mechanism. Land one machine-readable DurableChangeTrain contract for source.db and user.db so a migration window is declared, admitted, reserved, backup-authorized, applied, proven, and released through one lifecycle rather than coordinated in prose.","design":"Define a typed train manifest and state machine keyed by tier, shipped version, target version, and numbered slot. It owns rider declarations, exact schema/runtime wiring, ordering/drop constraints, single writer reservation, backup receipt binding, rollout states, and evidence refs. Integrate p155's collision key, canonical schema inventory, durable backup verification, stopped-daemon apply, fresh-DDL parity, post-migration behavior checks, and restart convergence. The gate must work for a synthetic next source and user train independently; it does not implement a new migration engine or merge derived-tier b5l semantics.","acceptance_criteria":"1. A typed, machine-readable manifest represents tier/current/target/slot, riders, runtime consumers, ordering, owner/reservation, backup receipt, lifecycle state, and proof refs. 2. Policy/conductor admission rejects stale versions, duplicate tier/version/slot ownership, schema-only or unproven riders, absent fresh-DDL parity, missing backup authority, and a second writer before merge/apply. 3. Synthetic independent source and user trains traverse declare→admit→reserve→authorize→apply→prove→release; late riders enter a new train and failed/interrupted states expose exact recovery. 4. Apply uses existing numbered additive migrations under stopped-daemon/single-writer authority and binds pre/post integrity, row parity, and behavioral proof; restart must converge before release. 5. Replaying the source 008/009 collision and a schema-without-runtime-consumer mutation fails. 6. p155 and canonical-inventory checks become components of this lifecycle rather than parallel coordination rules; focused policy/migration/backup/runtime tests and quick gate pass.","notes":"Active-set expansion 2026-07-15: admitted as a high-leverage operational mechanism under the scale/raw-authority program; execution focus remains readiness- and conflict-aware.\n2026-07-16 GPT-Pro corpus adjudication: durable schema-change-train package 251332b72bd8 remains blocked/seeded. Retain additive durable migration plus verified backup-manifest and derived-tier canonical-DDL rebuild constraints. Do not add a second migration writer or weaken fresh-DDL parity; PR #2931/live deployment is outside this lane.","status":"open","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:32:44Z","created_by":"Sinity","updated_at":"2026-07-16T12:56:56Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-1xc"},"labels":["area:storage","area:substrate","delivery:B-storage-rebuild-bytes","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-60i5.1","depends_on_id":"polylogue-60i5","type":"parent-child","created_at":"2026-07-15T20:32:44Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-60i5.1","depends_on_id":"polylogue-p155","type":"relates-to","created_at":"2026-07-15T20:32:44Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-hg8n.1","title":"Run the first unaided external adoption receipt","description":"The outside-adoption epic's install half is live, but none of its existing children owns the remaining terminal outcome: select one person outside the project, let them install Polylogue, run the smallest evidence-audit/continuity wedge, and query their own archive without operator assistance. This is a product-validation run with an evidence receipt, not another docs or packaging project.","design":"Preflight only the minimum honest path: supported install artifact, one no-context claim/evidence export, and AI-D3 or the smallest available prior-recovery query with measured/degraded semantics. Recruit one explicit participant with consent and privacy boundary; provide only the public instructions. Record timestamps, environment/version, commands, surfaced evidence refs, errors, questions, assistance requested, abandonment/recovery, and terminal outcome. Do not coach around product defects during the primary run; after the stop condition, debrief and file friction against existing invariant owners. Preserve private content locally and publish only consented/redacted aggregate evidence.","acceptance_criteria":"1. A named supported install route and exact build are verified before the run; no operator-private setup is required. 2. One consenting person outside the project completes or attempts install, a no-context claim/evidence inspection, one continuity/recovery flow, and one query over their own data from public instructions alone. 3. The receipt records every step, elapsed time, evidence/ref resolution, degraded/unsupported state, request for help, and terminal outcome without exposing private archive content. 4. Success requires unaided completion; assisted, blocked, abandoned, or no-value outcomes remain valid falsification evidence and cannot be rewritten as adoption. 5. Every friction point maps to an existing owner or a new non-duplicate Bead, and the epic's install/activation claims are updated from the receipt. 6. A cold reviewer can reproduce the public portion and verify the redacted receipt integrity.","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T18:32:43Z","created_by":"Sinity","updated_at":"2026-07-15T18:32:43Z","labels":["area:adoption","area:legibility","delivery:L-external-legibility","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-hg8n.1","depends_on_id":"polylogue-hg8n","type":"parent-child","created_at":"2026-07-15T20:32:43Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-hg8n.1","depends_on_id":"polylogue-yeq.4","type":"relates-to","created_at":"2026-07-15T20:43:44Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} @@ -168,7 +169,7 @@ {"_type":"issue","id":"polylogue-f2qv.6","title":"Reconcile profiles and costs to exact provider usage","description":"One live Codex session has three incompatible answers: exact model usage reports 64,561 uncached input, 723,456 cache read, and 7,776 output; session_profiles reports a 4,031-token estimate; cost insight reports zero and unavailable. Across all 2,856 Codex sessions with nonzero reported lanes, zero profiles matched. Profiles are built before provider usage and both are stamped current.\n\n## Steps to Reproduce\n1. Select a Codex session with a final provider cumulative usage event.\n2. Compare session_model_usage, session_profiles, and the per-session cost insight.\n3. Observe three incompatible lane sets with the same materialization freshness; repeat the model-versus-profile comparison across Codex sessions with nonzero exact lanes.","design":"Create one canonical per-session usage snapshot with disjoint token-lane authority separate from monetary price authority. Prefer exact provider events and model rollups; use estimates only as labeled fallback. Represent exact tokens with unknown USD, unavailable pricing, estimated money, and measured zero through the cuxz.2 EvidenceValue axes rather than numeric sentinels or a usage-local confidence vocabulary. Reconcile event, rollup, profile, cost, and public surfaces to this snapshot; record contradiction debt and materialize in dependency order.","acceptance_criteria":"Exact event through rollup, snapshot, profile, and cost agree on every lane; exact tokens with no price remain exact tokens plus unknown or estimated money; estimate-only providers stay explicit; rebuild and incremental convergence agree; live Codex census has zero unexplained profile contradictions with price unknowns separate; restoring profile-before-provider order fails; focused usage, profile, cost, and convergence tests pass.","notes":"2026-07-27 first slice: PR #3299 (feature/fix/session-usage-cost-reconciliation-slice) adds build_session_usage_reconciliation() / SessionUsageReconciliation to polylogue/storage/usage.py -- a pure reconciliation function over already-loaded session_model_usage rows, session_profiles token/cost columns, and the cost-insight fields, using two new session-grain FactFamilySpecs (SESSION_USAGE_RECONCILED_TOKENS_FAMILY, SESSION_USAGE_RECONCILED_COST_FAMILY) and the cuxz.2 refine_evidence_value primitive to pick the strongest-authority value on disagreement (provider-reported session_model_usage over a structural/model-derived session_profiles estimate; a fresh catalog reprice over a legacy persisted cost), while preserving every superseded input as a labeled contribution rather than discarding it. Test tests/unit/storage/test_session_usage_reconciliation.py reproduces the bead's exact reported numbers (64,561 uncached input + 723,456 cache read + 7,776 output vs a 4,031-token estimate vs zero/unavailable cost) and proves the reconciled snapshot picks the exact rollup, not an average, and surfaces the estimate as superseded.\n\nHonest scope: this is ONE case, not the full bead. Explicitly NOT done:\n- No storage/insight wiring -- nothing in storage/insights/session/rebuild.py, storage/sqlite/archive_tiers/archive.py (_session_cost_insight_from_archive_row still reads session_profiles directly), or insights/registry.py calls this function. session_model_usage, session_profiles, and the cost insight still disagree in the live archive today; this PR does not change any read path.\n- No daemon convergence integration or contradiction-debt recording.\n- No corpus-wide census proving \"zero unexplained profile contradictions\" (AC 5) -- that requires wiring plus a live-archive audit, deferred.\n- No \"restoring profile-before-provider order fails\" regression test -- that is a materialization-ordering test against the wired path, which doesn't exist yet.\n- Broader EvidenceValue family/surface migration remains polylogue-cuxz.3 scope, unaffected by this PR.\n\nRemaining work for this bead: wire build_session_usage_reconciliation (or its successor) into the actual session-insight rebuild/cost-insight read paths so live sessions produce the reconciled snapshot instead of three independent reads; add the corpus-wide census/contradiction-debt recording; add the profile-before-provider-order regression test; decide whether this becomes a materialized/insight-registry entry (per the bead's own design note) rather than a pure function callers must invoke manually.\n2026-07-27: first slice (per-session token/cost reconciliation for one disagreement case) merged via PR #3299. Self-review before merge (CodeRabbit rate-limited) found and fixed a real cost-mispricing bug: the reconciled token total collapsed input/output/cache_read/cache_write into one combined int, then priced the whole thing as pure input tokens - a ~4x cost overstatement on the bead's own repro case ($0.99 vs correct $0.25), since cache-read tokens (723K of 795K total) got priced at full input rate instead of their real discounted rate. Fixed by threading the winning source's real per-category breakdown through to estimate_cost. Remaining scope per the PR's own honest accounting (~15-20% of full AC): storage/insight wiring so live sessions actually surface reconciled values, daemon convergence/contradiction-debt integration, corpus-wide zero-unexplained-contradictions census, cuxz.3's broader family migration.","status":"open","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:59Z","created_by":"Sinity","updated_at":"2026-07-27T05:22:37Z","labels":["area:analytics","area:insights","delivery:A-trust-floor","horizon:frontier","lane:security-privacy","spine"],"dependencies":[{"issue_id":"polylogue-f2qv.6","depends_on_id":"polylogue-cuxz","type":"relates-to","created_at":"2026-07-15T20:17:31Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-f2qv.6","depends_on_id":"polylogue-cuxz.2","type":"blocks","created_at":"2026-07-15T20:50:03Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-f2qv.6","depends_on_id":"polylogue-f2qv","type":"parent-child","created_at":"2026-07-15T06:23:58Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-f2qv.6","depends_on_id":"polylogue-f2qv.5","type":"relates-to","created_at":"2026-07-15T06:25:50Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"polylogue-37t.23","title":"Derive session resumability from open obligations, not termination text","description":"A live Codex session ended normally with an explicit unresolved deployment decision, yet its profile says clean_finish, blocker extraction is suppressed, and resume discovery excludes or zero-weights it. Among 500 recent sessions, 226 were clean finishes and at least two were manually confirmed clean-but-unfinished; keyword markers also yielded false positives. Process termination and objective posture are orthogonal: the archive needs one session-level resumability projection over authority-bearing open obligations, not a second completion truth inferred from the final message.","design":"Define ObjectivePosture as a derived projection, separate from terminal process state. Apply an explicit authority order: declared goal/question open-close-block events when available; provider/work-evidence graph claims, structured results, observed effects, and evaluated satisfaction; durable decision/blocker/handoff assertions; then bounded authored-request/structural inference; otherwise unknown. Preserve evidence refs, as-of frame, authority, contradictions, and multiple simultaneous obligations. Profiles, blocker extraction, resume ranking, and context compilation consume this one projection. The work-evidence graph and goal graph remain fact owners; this bead neither duplicates their storage nor equates a claim with completion. Keep routing in 37t.8 and descriptive proof in 212.6.","acceptance_criteria":"1. A normal final answer with an unresolved decision has terminal_state=clean_finish and objective_posture=awaiting_operator simultaneously; resume discovery includes it for the repository. 2. Completed, blocked, abandoned/inactive, awaiting_operator, awaiting_effect, and ambiguous/unknown cases preserve typed obligation/evidence refs, authority, as-of frame, and contradictions. 3. Explicit goal/work-effect evidence outranks weaker inference; a self-reported claim without observed/evaluated effect cannot become completed. 4. Protocol-only messages, final-assistant presence, and keyword matches cannot decide posture alone; removing the authority precedence recreates the known false completion/false positive. 5. Profiles, blocker extraction, ranking, and context all consume the same projection with no parallel terminal-state completion heuristic. 6. A labeled live sample records precision/coverage and the known anchor; focused profile/enrichment/ranking/context tests pass.","notes":"2026-07-15 invariant formulation: session posture is now explicitly a projection over 1vpm.6 work evidence and 7yk5 goal/question state when available, with assertions/inference as lower-authority fallbacks. Those graphs remain distinct fact lifecycles; this leaf owns the one resumability projection consumed by profiles, blockers, ranking, and context.\n2026-07-16 GPT-Pro corpus adjudication: objective-posture package 0d45bbbc8ddb remains blocked/seeded. Retained design: derive resumability from authoritative open obligations, not terminal prose; reconcile against current insight/storage authorities before any large patch.","status":"closed","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:56Z","created_by":"Sinity","updated_at":"2026-07-20T20:04:07Z","closed_at":"2026-07-20T20:04:07Z","close_reason":"Delivered in PR #3226 (squash 4799d24e1): objective_posture projection with explicit authority order (goal_graph \u003e work_evidence \u003e assertion \u003e structural_inference \u003e none); structural tier baked into session_profiles materialization (index.db-only, never emits completed), assertion tier as read-time overlay (decision/blocker/handoff outrank structural inference; contradictions surfaced not collapsed); recomputed onto reconciled terminal_state at both read sites; consumers rewired (blocker extraction gates on shared mapping replacing the unknown-missing allowlist, resume ranking posture-weighted + dead clean_finish filter replaced post-#2960, resume_brief overlays assertion tier, context preamble surfaces posture). AC1 reframed honestly (clean_finish deleted by #2960). AC6 (labeled live-sample precision run) deferred — needs live archive; assertion overlay is per-physical-session, lineage composition and goal_graph/work_evidence tiers reserved for 7yk5/1vpm.6. Verification: 381 focused + 601 sweep tests green, 3 sweep failures proven pre-existing on pristine master.","labels":["area:context","area:insights","delivery:D-agent-context-coordination","horizon:frontier","lane:agent-coordination"],"dependencies":[{"issue_id":"polylogue-37t.23","depends_on_id":"polylogue-1vpm.6","type":"relates-to","created_at":"2026-07-15T20:29:53Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-37t.23","depends_on_id":"polylogue-37t","type":"parent-child","created_at":"2026-07-15T06:23:55Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-37t.23","depends_on_id":"polylogue-7yk5","type":"relates-to","created_at":"2026-07-15T20:29:53Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"polylogue-j2zz","title":"Lower Codex orchestration child calls into typed actions","description":"Modern Codex embeds typed operations inside functions.exec JavaScript. In the newest 100-session sample, every session had nested tools calls, 14,004 envelopes held child operations, and 19,180 results yielded zero structured paths or outcomes although 1,444 texts contained exit_code. Polylogue retains only outer exec or shell semantics.\n\n## Steps to Reproduce\n1. Ingest a current Codex session containing functions.exec with nested exec_command and apply_patch calls.\n2. Query its actions and files through Polylogue.\n3. Compare with raw JSONL and observe only outer exec or shell actions, zero normalized file paths, and unknown structural outcomes.","design":"Lower functions.exec into provenance-linked child actions while retaining the outer call as transport. Use a typed registry for exec_command, apply_patch, write_stdin, update_plan, wait, web, image, MCP, and unknown shapes. Promote only structural result fields, preserve ordering and repeated calls, and feed the bounded relation owned by polylogue-z9gh.2.","acceptance_criteria":"Fixtures lower single and multiple children into ordered typed actions linked to transport; commands and patches expose normalized commands and paths; outcome fields are structural or unknown; malformed and unknown tools retain evidence; repeated calls and continuations pair deterministically without inventing recovery; live sample reports child/path/outcome coverage; removing lowering recreates zero-file outer-only results; parser/action tests and quick gate pass.","notes":"Portfolio placement 2026-07-15: execution slice and live canary of OriginSpec normalized-construct lowering and positive outcome/path provenance. The outer transport and child actions also feed 1vpm.6, but source authority stays with OriginSpec.","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:52Z","created_by":"Sinity","updated_at":"2026-07-27T02:05:51Z","closed_at":"2026-07-27T02:05:51Z","close_reason":"Satisfied: Codex functions.exec child lowering into typed actions (exec_command, apply_patch, write_stdin, update_plan, wait, web, image, mcp, unknown registry with path/outcome promotion and ordering) landed via commit 46e478fb1, PR #3063. Regression coverage green: tests/unit/devtools/test_codex_exec_child_census.py + tests/unit/sources/test_codex_event_stream_contract.py (31 tests). No later commit reverted this logic - live on master unchanged since #3063. Bead was stale (open, no close_reason). Re-verified 2026-07-27 via independent triage.","labels":["area:query","area:sources","delivery:C-read-evidence-contract","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-j2zz","depends_on_id":"polylogue-2qx","type":"parent-child","created_at":"2026-07-15T18:38:54Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-j2zz","depends_on_id":"polylogue-2qx.1.2","type":"blocks","created_at":"2026-07-15T20:55:26Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-j2zz","depends_on_id":"polylogue-9l5.6","type":"relates-to","created_at":"2026-07-15T06:25:47Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-j2zz","depends_on_id":"polylogue-z9gh.2","type":"relates-to","created_at":"2026-07-15T06:25:43Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"polylogue-ih67","title":"Enrich Codex titles from authored history in canonical ingest","description":"All 3,101 indexed Codex sessions in the live archive use native UUID as title. The canonical raw-record daemon worker bypasses provider assembly. Live Codex supplies history.jsonl, not the expected optional sidecar, and the current role=user fallback would select injected AGENTS context before the human_authored request.\n\n## Steps to Reproduce\n1. Count Codex sessions where title equals native_id in the live index.\n2. Inspect a modern Codex session whose first user-role row is runtime context and whose later row is human_authored.\n3. Follow canonical raw-record daemon ingest and observe that it calls parser entrypoints without provider assembly or history enrichment.","design":"Extend the Codex OriginSpec assembly declaration from 2qx.1.2 and make that assembly run in canonical raw-record ingest, not only direct path ingest. In polylogue/sources/assembly_codex.py, discover both session_index.jsonl thread names and the live Codex history.jsonl source with append-only newest-wins/dedup/freshness rules keyed by session/thread identity; represent sidecar identity and authority in typed assembly data rather than reading ambient files inside a parser. Resolve title in this order: non-empty provider thread name, matching authoritative history title/prompt, first message whose material_origin is human_authored, then native UUID/unknown. Never select role=user alone because runtime_context and operator protocol rows use that role. In polylogue/pipeline/services/ingest_worker.py and its parse-plan construction, pass acquired sidecar/assembly inputs through the subprocess-safe raw-record plan and call the same get_assembly_spec enrichment used by direct ingest before materialization/hash/write. Persist TitleSource plus a more specific provenance/ref/confidence field in the next appropriate batched index/source model change; title provenance must not alter session identity. Reprocess affected Codex raws through ordinary semantic reparse/rematerialization, preserving assertions and links, and emit before/after coverage counts. Keep generic display synthesis in polylogue-30h separate. Primary tests: sources assembly Codex, parsers Codex authoredness, pipeline ingest_worker/raw batch parity, storage title provenance, and a corpus-shaped UUID-title canary.","acceptance_criteria":"1. Direct source ingest and canonical raw-record daemon ingest invoke the same Codex assembly and produce identical title, TitleSource, specific provenance/ref/confidence, content hash consequences, and diagnostics for the same acquired inputs. Bypassing assembly makes the daemon parity test fail. 2. Resolution order is provider thread name, matched authoritative history entry, first human_authored message, UUID/unknown. A runtime_context or operator command in an earlier role=user row never becomes the title. 3. session_index.jsonl and history.jsonl duplicate, malformed, missing, stale, equal-timestamp, and conflicting rows have deterministic newest-wins or explicit ambiguous outcomes; ambient file changes cannot silently alter a previously acquired replay. 4. Sidecars are acquired/referenced as raw authority evidence and passed through subprocess-safe parse plans; parsers do not open live home-directory sidecars during replay. 5. Title provenance is persisted and queryable, while rematerialization preserves session_id, message/block identity where content is unchanged, lineage, assertions, and user state. Semantic hash/reparse behavior for an improved title is explicit and idempotent. 6. A live-scale privacy-safe census records UUID-title coverage before/after, improves all deterministically enrichable sessions, and leaves unresolved reasons classified rather than claiming 100 percent. 7. Focused assembly, Codex authoredness, direct-vs-daemon parity, raw replay, storage provenance, reprocess, and projection tests plus affected verification pass.","notes":"Portfolio placement 2026-07-15: execution slice and live canary of OriginSpec artifact inventory, canonical assembly, authoredness authority, title provenance, and semantic reparse. It is not an independent source-admission mechanism.\nPriority correction 2026-07-15: promoted and admitted because every indexed Codex session currently having a UUID title is a corpus-wide discovery failure tied to authoredness and source-admission authority.\nTerra-readiness correction 2026-07-15: named the current assembly and raw-worker bypass, fixed title precedence on material_origin rather than role, required acquired sidecar authority instead of ambient replay reads, and specified identity-preserving reprocessing plus a corpus canary.\n\n[2026-07-18] Named as a blocker (D6) in the ann-03-batch-runbook-r01 mass-annotation prioritization decision (full ranking recorded on polylogue-rxdo): title/topic quality is treated as an ingest/authority defect owned by this bead, not an annotation target -- labels cannot repair a route that never produced the intended title. Only a small post-fix canary annotation is recommended, and only after this bead lands. This bead therefore gates campaign D6 (title-source coverage, UUID residuals, generated-title acceptance, retrieval quality/lift) in the annotation launch order.\nWarroom It.18 (2026-07-18): first slice landed via PR #3071 -- canonical raw-record ingest now runs get_assembly_spec enrichment (keyed off recorded acquisition path; blob/foreign-machine replays degrade to parsed-content fallbacks); assembly_codex gains history.jsonl earliest-authored-entry titles with stat-fingerprint caching; resolution order = thread name -\u003e authored history -\u003e first HUMAN_AUTHORED message -\u003e native id; role=user alone never titles. Parity tests with named mutation (bypass fails 3/3). VERIFIED LOCAL DATA: ~/.codex/state_5.sqlite threads.title 2757/3040 non-empty; history.jsonl 17762 entries. REMAINING SCOPE on this bead: (a) sidecars acquired as raw authority evidence + subprocess-safe plans (AC#3/4 -- ambient reads still possible when source_path exists at reprocess time); (b) persisted TitleSource provenance/ref/confidence in a batched model change (AC#5 partial: title_source flows in parsed model only); (c) state_5.sqlite threads.title as an additional discovery source (richer than session_index.jsonl on live installs -- copy-first, it is live-locked); (d) reprocess affected Codex raws + before/after UUID-title census (AC#6).\n2026-07-26 portfolio-convergence audit: released stale in_progress claim after \u003e7 days with no recorded activity; scope remains open and must be re-claimed on real work start.\n2026-07-27: state_5.sqlite threads.title added as a discovery source, merged via PR #3292. Precedence: thread name -\u003e authored history -\u003e state_5.sqlite title -\u003e first human-authored message -\u003e native id. Remaining scope per prior notes: sidecar acquisition as raw authority evidence, persisted TitleSource/ref/confidence provenance columns, corpus-wide before/after census - not attempted this pass.\n2026-07-27 PR #3360: title_source persisted-but-unqueryable gap fixed. ArchiveStore.read_summary/list_summaries now SELECT s.title_source (ArchiveSessionSummary gained the field); archive/query/archive_execution.py + api/archive.py's duplicate _session_to_session/_summary_to_domain helpers now map title_source onto Session/SessionSummary domain models; SessionListRowPayload/SessionSummaryPayload expose it; SESSION_COLUMNS updated to match. Anti-vacuity-verified new tests in tests/unit/storage/test_title_source_queryable.py. This closes the \"queryable\" half of AC#5 for the value that already existed (TitleSource enum on the row), not a new ref/confidence field.\nREMAINING SCOPE (unchanged from 2026-07-27 prior note, not attempted this PR): (a) sidecar acquisition as raw authority evidence + subprocess-safe parse plans (AC#3/4 -- ambient reads still possible when source_path exists at reprocess time); (b) a dedicated ref/confidence provenance field beyond the existing TitleSource value; (c) corpus-wide before/after UUID-title census (AC#6). ih67 stays open.\n2026-07-28 PR #3378 (branch feature/sources/codex-title-provenance-ih67, 4 commits): landed all three items named as \"not attempted this pass\" in the 2026-07-27 note.\n(a) AC#3/#4 sidecar freeze: _resolve_codex_sidecar_snapshots (ingest_batch/_core.py) runs in the main process before dispatch, persists each Codex raw record's first-observed sidecar snapshot in history_sidecars (source.db, previously-unwired write_history_sidecar + new read_earliest_history_sidecar_for_path), and carries it across the process-pool boundary via RawSessionRecord.sidecar_snapshot (exclude=True). _enrich_parsed_sessions (subprocess) uses the frozen snapshot when present and never touches disk. Anti-vacuity: reverting the lookup reproduces the exact ambient-drift bug and fails the new test. Residual: state_5.sqlite is covered by the freeze (it's part of the persisted snapshot dict) but is still read live at first-acquisition time rather than separately blob-hashed beforehand -- not a correctness gap for AC#3 (frozen thereafter), just a smaller scope than a dedicated blob per sidecar file.\n(b) AC#5 ref/confidence: new nullable sessions.title_ref/title_confidence columns (index.db v44, additive derived-tier DDL), stamped per-lane in assembly_codex.py (thread-name=1.0, history=0.9, state-db=0.75, message-fallback=0.5), wired through the full write-\u003estorage-summary/envelope-\u003edomain-model-\u003eCLI/MCP-payload chain exactly like #3360 did for title_source. Regenerated schemas/openapi/webui client.\n(c) AC#6 census: polylogue/archive/codex_title_census.py + `polylogue ops diagnostics codex-title-census [--json|--save|--compare]`. Privacy-safe (sessions-table columns only, no message text, no paths). Classifies unresolved reason: no_messages_materialized / no_human_authored_message / not_yet_reprocessed_with_assembly / human_authored_present_synthesis_failed. Live read-only smoke test against the real archive (no mutation): 3201 total Codex sessions, 0 resolved, 2977 not_yet_reprocessed_with_assembly, 207 no_human_authored_message, 17 no_messages_materialized -- confirms the live corpus has not had a reprocess pass since #3071 landed; this is the honest \"before\" baseline.\nVerification: devtools verify --quick exit 0; mypy/ruff clean; 19 focused tests pass across all three pieces; anti-vacuity (revert/confirm-fail/restore) done for all three.\nREMAINING SCOPE (not this PR, explicit): (1) actually triggering a live reprocess of the 2977 eligible-but-stale sessions to move the corpus to an \"after\" baseline -- mutates production data, needs a separate operator-authorized step (polylogue ops reprocess / polylogued run), out of this PR's read-only scope. (2) state_5.sqlite as a dedicated content-hashed blob rather than read-then-frozen-in-snapshot (residual noted above). (3) SESSION_COLUMNS (search projection example list) intentionally not extended with title_ref/title_confidence -- separate optional surface decision.\nPR: https://github.com/Sinity/polylogue/pull/3378\n\n2026-07-28 CORRECTION to this session's earlier deploy-risk framing: previously stated the schema bump (43-\u003e44) would cause the live daemon to 'report a schema mismatch' -- that UNDERSTATES the real severity. Confirmed via polylogue/storage/sqlite/schema_bootstrap.py: decide_schema_bootstrap()'s version_mismatch branch means the runtime REFUSES TO OPEN index.db entirely (not degraded status, not a soft readout) until an operator runs `polylogue ops reset --index \u0026\u0026 polylogued run`. Since master's history is linear, this commit is now an ancestor of every later commit landed today (1vpm.6.1 #3375, t46.9/kwsb.2 phase 6 #3376, 20d.17 #3377, t46.8.2 verification, t46.8.3 #3379, ovme.2 #3380, ovme.3 #3381) -- deploying ANY of them live now necessarily deploys this schema bump too and would break the running daemon until the rebuild is performed. Currently HELD BACK: sinnix flake.lock remains pinned at 2725fc3e2 (the last commit before this one), so the live daemon is unaffected and still fully functional. All subsequent real fixes are merged to master but NOT yet deployed live, pending an explicit operator decision to deploy+immediately rebuild-index together as one coordinated action.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:49Z","created_by":"Sinity","updated_at":"2026-07-28T19:07:39Z","started_at":"2026-07-18T00:05:17Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:insights","area:sources","area:surface","delivery:C-read-evidence-contract","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-ih67","depends_on_id":"polylogue-2qx","type":"parent-child","created_at":"2026-07-15T18:38:54Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-ih67","depends_on_id":"polylogue-2qx.1.2","type":"blocks","created_at":"2026-07-15T20:55:25Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-ih67","depends_on_id":"polylogue-30h","type":"relates-to","created_at":"2026-07-15T06:25:40Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-ih67","title":"Enrich Codex titles from authored history in canonical ingest","description":"All 3,101 indexed Codex sessions in the live archive use native UUID as title. The canonical raw-record daemon worker bypasses provider assembly. Live Codex supplies history.jsonl, not the expected optional sidecar, and the current role=user fallback would select injected AGENTS context before the human_authored request.\n\n## Steps to Reproduce\n1. Count Codex sessions where title equals native_id in the live index.\n2. Inspect a modern Codex session whose first user-role row is runtime context and whose later row is human_authored.\n3. Follow canonical raw-record daemon ingest and observe that it calls parser entrypoints without provider assembly or history enrichment.","design":"Extend the Codex OriginSpec assembly declaration from 2qx.1.2 and make that assembly run in canonical raw-record ingest, not only direct path ingest. In polylogue/sources/assembly_codex.py, discover both session_index.jsonl thread names and the live Codex history.jsonl source with append-only newest-wins/dedup/freshness rules keyed by session/thread identity; represent sidecar identity and authority in typed assembly data rather than reading ambient files inside a parser. Resolve title in this order: non-empty provider thread name, matching authoritative history title/prompt, first message whose material_origin is human_authored, then native UUID/unknown. Never select role=user alone because runtime_context and operator protocol rows use that role. In polylogue/pipeline/services/ingest_worker.py and its parse-plan construction, pass acquired sidecar/assembly inputs through the subprocess-safe raw-record plan and call the same get_assembly_spec enrichment used by direct ingest before materialization/hash/write. Persist TitleSource plus a more specific provenance/ref/confidence field in the next appropriate batched index/source model change; title provenance must not alter session identity. Reprocess affected Codex raws through ordinary semantic reparse/rematerialization, preserving assertions and links, and emit before/after coverage counts. Keep generic display synthesis in polylogue-30h separate. Primary tests: sources assembly Codex, parsers Codex authoredness, pipeline ingest_worker/raw batch parity, storage title provenance, and a corpus-shaped UUID-title canary.","acceptance_criteria":"1. Direct source ingest and canonical raw-record daemon ingest invoke the same Codex assembly and produce identical title, TitleSource, specific provenance/ref/confidence, content hash consequences, and diagnostics for the same acquired inputs. Bypassing assembly makes the daemon parity test fail. 2. Resolution order is provider thread name, matched authoritative history entry, first human_authored message, UUID/unknown. A runtime_context or operator command in an earlier role=user row never becomes the title. 3. session_index.jsonl and history.jsonl duplicate, malformed, missing, stale, equal-timestamp, and conflicting rows have deterministic newest-wins or explicit ambiguous outcomes; ambient file changes cannot silently alter a previously acquired replay. 4. Sidecars are acquired/referenced as raw authority evidence and passed through subprocess-safe parse plans; parsers do not open live home-directory sidecars during replay. 5. Title provenance is persisted and queryable, while rematerialization preserves session_id, message/block identity where content is unchanged, lineage, assertions, and user state. Semantic hash/reparse behavior for an improved title is explicit and idempotent. 6. A live-scale privacy-safe census records UUID-title coverage before/after, improves all deterministically enrichable sessions, and leaves unresolved reasons classified rather than claiming 100 percent. 7. Focused assembly, Codex authoredness, direct-vs-daemon parity, raw replay, storage provenance, reprocess, and projection tests plus affected verification pass.","notes":"Portfolio placement 2026-07-15: execution slice and live canary of OriginSpec artifact inventory, canonical assembly, authoredness authority, title provenance, and semantic reparse. It is not an independent source-admission mechanism.\nPriority correction 2026-07-15: promoted and admitted because every indexed Codex session currently having a UUID title is a corpus-wide discovery failure tied to authoredness and source-admission authority.\nTerra-readiness correction 2026-07-15: named the current assembly and raw-worker bypass, fixed title precedence on material_origin rather than role, required acquired sidecar authority instead of ambient replay reads, and specified identity-preserving reprocessing plus a corpus canary.\n\n[2026-07-18] Named as a blocker (D6) in the ann-03-batch-runbook-r01 mass-annotation prioritization decision (full ranking recorded on polylogue-rxdo): title/topic quality is treated as an ingest/authority defect owned by this bead, not an annotation target -- labels cannot repair a route that never produced the intended title. Only a small post-fix canary annotation is recommended, and only after this bead lands. This bead therefore gates campaign D6 (title-source coverage, UUID residuals, generated-title acceptance, retrieval quality/lift) in the annotation launch order.\nWarroom It.18 (2026-07-18): first slice landed via PR #3071 -- canonical raw-record ingest now runs get_assembly_spec enrichment (keyed off recorded acquisition path; blob/foreign-machine replays degrade to parsed-content fallbacks); assembly_codex gains history.jsonl earliest-authored-entry titles with stat-fingerprint caching; resolution order = thread name -\u003e authored history -\u003e first HUMAN_AUTHORED message -\u003e native id; role=user alone never titles. Parity tests with named mutation (bypass fails 3/3). VERIFIED LOCAL DATA: ~/.codex/state_5.sqlite threads.title 2757/3040 non-empty; history.jsonl 17762 entries. REMAINING SCOPE on this bead: (a) sidecars acquired as raw authority evidence + subprocess-safe plans (AC#3/4 -- ambient reads still possible when source_path exists at reprocess time); (b) persisted TitleSource provenance/ref/confidence in a batched model change (AC#5 partial: title_source flows in parsed model only); (c) state_5.sqlite threads.title as an additional discovery source (richer than session_index.jsonl on live installs -- copy-first, it is live-locked); (d) reprocess affected Codex raws + before/after UUID-title census (AC#6).\n2026-07-26 portfolio-convergence audit: released stale in_progress claim after \u003e7 days with no recorded activity; scope remains open and must be re-claimed on real work start.\n2026-07-27: state_5.sqlite threads.title added as a discovery source, merged via PR #3292. Precedence: thread name -\u003e authored history -\u003e state_5.sqlite title -\u003e first human-authored message -\u003e native id. Remaining scope per prior notes: sidecar acquisition as raw authority evidence, persisted TitleSource/ref/confidence provenance columns, corpus-wide before/after census - not attempted this pass.\n2026-07-27 PR #3360: title_source persisted-but-unqueryable gap fixed. ArchiveStore.read_summary/list_summaries now SELECT s.title_source (ArchiveSessionSummary gained the field); archive/query/archive_execution.py + api/archive.py's duplicate _session_to_session/_summary_to_domain helpers now map title_source onto Session/SessionSummary domain models; SessionListRowPayload/SessionSummaryPayload expose it; SESSION_COLUMNS updated to match. Anti-vacuity-verified new tests in tests/unit/storage/test_title_source_queryable.py. This closes the \"queryable\" half of AC#5 for the value that already existed (TitleSource enum on the row), not a new ref/confidence field.\nREMAINING SCOPE (unchanged from 2026-07-27 prior note, not attempted this PR): (a) sidecar acquisition as raw authority evidence + subprocess-safe parse plans (AC#3/4 -- ambient reads still possible when source_path exists at reprocess time); (b) a dedicated ref/confidence provenance field beyond the existing TitleSource value; (c) corpus-wide before/after UUID-title census (AC#6). ih67 stays open.\n2026-07-28 PR #3378 (branch feature/sources/codex-title-provenance-ih67, 4 commits): landed all three items named as \"not attempted this pass\" in the 2026-07-27 note.\n(a) AC#3/#4 sidecar freeze: _resolve_codex_sidecar_snapshots (ingest_batch/_core.py) runs in the main process before dispatch, persists each Codex raw record's first-observed sidecar snapshot in history_sidecars (source.db, previously-unwired write_history_sidecar + new read_earliest_history_sidecar_for_path), and carries it across the process-pool boundary via RawSessionRecord.sidecar_snapshot (exclude=True). _enrich_parsed_sessions (subprocess) uses the frozen snapshot when present and never touches disk. Anti-vacuity: reverting the lookup reproduces the exact ambient-drift bug and fails the new test. Residual: state_5.sqlite is covered by the freeze (it's part of the persisted snapshot dict) but is still read live at first-acquisition time rather than separately blob-hashed beforehand -- not a correctness gap for AC#3 (frozen thereafter), just a smaller scope than a dedicated blob per sidecar file.\n(b) AC#5 ref/confidence: new nullable sessions.title_ref/title_confidence columns (index.db v44, additive derived-tier DDL), stamped per-lane in assembly_codex.py (thread-name=1.0, history=0.9, state-db=0.75, message-fallback=0.5), wired through the full write-\u003estorage-summary/envelope-\u003edomain-model-\u003eCLI/MCP-payload chain exactly like #3360 did for title_source. Regenerated schemas/openapi/webui client.\n(c) AC#6 census: polylogue/archive/codex_title_census.py + `polylogue ops diagnostics codex-title-census [--json|--save|--compare]`. Privacy-safe (sessions-table columns only, no message text, no paths). Classifies unresolved reason: no_messages_materialized / no_human_authored_message / not_yet_reprocessed_with_assembly / human_authored_present_synthesis_failed. Live read-only smoke test against the real archive (no mutation): 3201 total Codex sessions, 0 resolved, 2977 not_yet_reprocessed_with_assembly, 207 no_human_authored_message, 17 no_messages_materialized -- confirms the live corpus has not had a reprocess pass since #3071 landed; this is the honest \"before\" baseline.\nVerification: devtools verify --quick exit 0; mypy/ruff clean; 19 focused tests pass across all three pieces; anti-vacuity (revert/confirm-fail/restore) done for all three.\nREMAINING SCOPE (not this PR, explicit): (1) actually triggering a live reprocess of the 2977 eligible-but-stale sessions to move the corpus to an \"after\" baseline -- mutates production data, needs a separate operator-authorized step (polylogue ops reprocess / polylogued run), out of this PR's read-only scope. (2) state_5.sqlite as a dedicated content-hashed blob rather than read-then-frozen-in-snapshot (residual noted above). (3) SESSION_COLUMNS (search projection example list) intentionally not extended with title_ref/title_confidence -- separate optional surface decision.\nPR: https://github.com/Sinity/polylogue/pull/3378\n\n2026-07-28 CORRECTION to this session's earlier deploy-risk framing: previously stated the schema bump (43-\u003e44) would cause the live daemon to 'report a schema mismatch' -- that UNDERSTATES the real severity. Confirmed via polylogue/storage/sqlite/schema_bootstrap.py: decide_schema_bootstrap()'s version_mismatch branch means the runtime REFUSES TO OPEN index.db entirely (not degraded status, not a soft readout) until an operator runs `polylogue ops reset --index \u0026\u0026 polylogued run`. Since master's history is linear, this commit is now an ancestor of every later commit landed today (1vpm.6.1 #3375, t46.9/kwsb.2 phase 6 #3376, 20d.17 #3377, t46.8.2 verification, t46.8.3 #3379, ovme.2 #3380, ovme.3 #3381) -- deploying ANY of them live now necessarily deploys this schema bump too and would break the running daemon until the rebuild is performed. Currently HELD BACK: sinnix flake.lock remains pinned at 2725fc3e2 (the last commit before this one), so the live daemon is unaffected and still fully functional. All subsequent real fixes are merged to master but NOT yet deployed live, pending an explicit operator decision to deploy+immediately rebuild-index together as one coordinated action.\nMEASURE CORRECTION 2026-07-28 (live index v43): the description says '3,101 indexed Codex sessions use native UUID as title'. Actual:\n\n SELECT count(*) FROM sessions WHERE origin='codex-session' AND title=native_id; -\u003e 3201\n SELECT count(*) FROM sessions WHERE origin='codex-session'; -\u003e 3201\n\nIt is 3,201, and it is 100% of the Codex population -- not a large subset. The v44 fixes are merged but undeployed, so the live archive still shows the full pre-fix state; this is the correct before-baseline for AC#6's before/after UUID-title census.\n\nDeploy status: the v44 schema bump landed in PR #3378 without its lifecycle.py delta declaration, which is why the repo CLI could not read the live v43 archive at all ('no such column: s.title_ref'). The declaration now exists (SEMANTIC_REPARSE, truthful under the current vocabulary); polylogue-9rw0.1 owns making this delta class cheap enough that title_ref does not require a full-corpus replay to populate.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:49Z","created_by":"Sinity","updated_at":"2026-07-28T20:03:34Z","started_at":"2026-07-18T00:05:17Z","metadata":{"frontier":"active","frontier_program_ref":"polylogue-z9gh"},"labels":["area:insights","area:sources","area:surface","delivery:C-read-evidence-contract","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-ih67","depends_on_id":"polylogue-2qx","type":"parent-child","created_at":"2026-07-15T18:38:54Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-ih67","depends_on_id":"polylogue-2qx.1.2","type":"blocks","created_at":"2026-07-15T20:55:25Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-ih67","depends_on_id":"polylogue-30h","type":"relates-to","created_at":"2026-07-15T06:25:40Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-1xc.13","title":"Expose named-source freshness and excluded cursor degradation","description":"Dogfood traced one growing Codex JSONL across filesystem, cursor, raw revisions, index, and FTS. Its cursor was excluded after five failures, later revisions remained unparsed, and the index was stale. The bounded sample omitted it and cursor projection classified excluded as idle before byte lag. Archive totals show 3,821 excluded cursors and 1,890 broken heads.","design":"Add a source or session scoped freshness projection joining source stat, cursor offset and observed size, retry or exclusion reason, acquired and accepted raw revision, parse and authority state, index high-water, and FTS or insight convergence. Excluded is degraded before idle. Keep raw authority in polylogue-lkrc and replay prevention in polylogue-yla8.","acceptance_criteria":"A growing excluded fixture reports excluded plus lag and retained reason, never idle; a healthy quiet source reports every acquisition-to-searchable checkpoint; named miss diagnostics distinguish unseen, acquired-unparsed, parsed-unindexed, indexed-unconverged, and searchable; exact-source execution avoids archive-wide scans; live excluded and healthy receipts exist; excluded and broken-head populations are classified before reset; focused tests and quick gate pass.","notes":"Live evidence 2026-07-15 from MCP readiness_check: raw_artifact_count=41,758, materialized_raw_artifact_count=18,331, archive_session_count=18,434, join_gap_count=23,427, plus 1,890 broken active heads, 40 cursor-ahead rows, and 34 uncomparable authority rows. The named-source projection must expose these excluded/degraded populations with snapshot/freshness and must not let an archive-wide session count imply source completeness.\n2026-07-16 integration scope: implement a bounded exact-source freshness read projection and canonical query/status/MCP surface only. It will classify excluded, cursor-ahead, and broken-head evidence as degraded before idle; distinguish unseen, acquired-unparsed, parsed-unindexed, indexed-unconverged, and searchable; and use exact source predicates with no archive/root scans or live mutation. Authority classification/repair remains polylogue-lkrc; replay prevention/actuation remains polylogue-yla8. Live receipts are read-only and deferred until code safety review.\n2026-07-16 implementation accounting: bounded exact-source projection now joins filesystem stat, cursor/retry/exclusion state, accepted raw authority (observed; polylogue-lkrc), application evidence (observed; polylogue-yla8), index high-water/broken-head, FTS, and insight debt; canonical status --source and MCP named_source_freshness call it. AC: excluded-growing/healthy-quiet fixtures and all five miss stages satisfied; exact-key bounds and scan rejection satisfied; aggregate excluded/cursor-ahead now degraded before idle; focused SQLite/FTS+MCP/status tests and seeded affected verify+quick pass. Remaining AC: operator must capture two read-only exact live receipts (incident excluded path and healthy quiet control) after selecting paths, before any lkrc/yla8 remediation. No archive mutation or receipt run in this integration.\n2026-07-16 review handoff: implementation commit 2242fab26 is published as PR #2924. It remains in progress solely for the two operator-selected, read-only live receipts; no archive repair/replay/reset authority was exercised by this branch.\n2026-07-16 GPT-Pro corpus adjudication: named-source design package 8fa6ec827281 is superseded by implementation package 17d8a28e9c6, merged as PR #2924 (b6c78adfcd666358307daf64ac97e8d695a8b854). Residual exact-source operational receipts remain governed by this bead, not a revived handoff lane.\n2026-07-17 fresh-source evidence: current raw browser capture contains ChatGPT handoff chatgpt:6a580976-03d0-83eb-af6a-eb745db5ac0c (Agent Query Discovery; file mtime 07:45 CEST), but POLYLOGUE_ARCHIVE_ROOT=/home/sinity/.local/share/polylogue polylogue --json --origin chatgpt-export find 'since:8h' returned total=0. This is a direct named-origin freshness/user-visible queryability failure: a newly captured ChatGPT artifact exists yet cannot be discovered through the archive. The eventual source-freshness route must make this distinguishable as acquired/unparsed or otherwise degraded with an exact source/capture reference, rather than a misleading empty search. No archive mutation was performed.\nWarroom sweep It.17: claiming session closed; implementation fully merged (#2924). Bead remains open ONLY for two operator-selected read-only live receipts (one excluded-incident path, one healthy quiet control) -- a ~5-minute OPERATOR action, flagged on the warroom board.\n2026-07-26 portfolio-convergence audit: released stale in_progress claim after \u003e7 days with no recorded activity; scope remains open and must be re-claimed on real work start.","status":"open","priority":1,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:46Z","created_by":"Sinity","updated_at":"2026-07-26T09:13:00Z","started_at":"2026-07-16T02:30:06Z","labels":["area:daemon","area:sources","area:storage","delivery:A-trust-floor","delivery:B-storage-rebuild-bytes","horizon:frontier","lane:storage-rebuild-scale"],"dependencies":[{"issue_id":"polylogue-1xc.13","depends_on_id":"polylogue-1xc","type":"parent-child","created_at":"2026-07-15T06:23:45Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.13","depends_on_id":"polylogue-cuxz","type":"relates-to","created_at":"2026-07-15T20:17:32Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.13","depends_on_id":"polylogue-lkrc","type":"relates-to","created_at":"2026-07-15T06:25:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-1xc.13","depends_on_id":"polylogue-yla8","type":"relates-to","created_at":"2026-07-15T06:25:37Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-20d.17","title":"Serve every status surface from budgeted component snapshots","description":"Live dogfood found polylogued status produced no result within 15 seconds although daemon heartbeat and database descriptors were healthy. Coordination status independently measured 2.6 to 16.6 second compact/detail reads. Both synchronously combine millisecond facts with multi-second raw, debt, embedding, Beads, process, archive, and handoff probes, so output byte bounds do not make status interactive. A cached snapshot exists in places, but whole-payload refresh, TTL-only reuse, and missing source fingerprints allow one expensive or stale component to dominate every answer.","design":"Define one StatusComponentSpec and StatusSnapshot protocol reused by daemon/archive and agent-coordination status. Each component declares collector, dependencies, cost/detail class, deadline, refresh trigger or source fingerprint, staleness policy, privacy, and projection fields. An off-request scheduler refreshes components independently, retains last-good evidence, and records fresh, stale, refreshing, timed_out, unavailable, and degraded with observed/start/finish timestamps and evidence refs. CLI, MCP, HTTP, and coordination envelopes select compact or detail projections from snapshots and never run expensive collectors inline. Exact replay, embedding, debt, Beads, archive-family, or handoff expansion is an explicit resumable detail query. Stage timing and request telemetry measure the protocol itself; cache reuse is keyed by declared evidence changes, not TTL alone.","acceptance_criteria":"1. Daemon/archive and coordination status both consume the same component-snapshot protocol; no request path synchronously rebuilds the rich whole. 2. A stalled raw/debt/embedding/Beads/archive/handoff component cannot delay healthy components and returns its explicit state, age, last-good evidence, deadline, and detail ref. 3. polylogued status returns within the interactive live-scale budget; warm compact coordination MCP p95 improves at least 3x from the measured baseline and cold compact CLI materially improves while preserving the 8 KiB projection bound and omission counts. 4. Randomized cold CLI and warm in-process MCP sampling records per-component timing, p50/p95, archive state, git head, fingerprints, cache decisions, and raw artifact refs; product budgets are set from those distributions. 5. Refresh invalidation follows declared source fingerprints or events; a changed Beads/archive/process source cannot be hidden by an unexpired TTL, while unavailable sources remain explicit. 6. Exact expensive diagnostics are opt-in, bounded, cancellable, and resumable; limit constrains collection work rather than only rendered rows. 7. Compact/detail payload semantics, process collapse, resource exclusions, archive readiness, and handoff evidence remain correct. Production stall and stale-source mutations fail the tests; live dogfood artifacts cover daemon and coordination consumers; focused status tests, SLO benchmark, and quick gate pass.","notes":"Invariant collapse 2026-07-15: absorbs s7ae.8. Its shipped stage harness/cache groundwork and remaining randomized sampling, source-keyed invalidation, p95 budget, and live dogfood become a second consumer proof of the same component snapshot mechanism.\n2026-07-15 portfolio convergence: absorbs polylogue-703. Its one-assembly requirement is the shared StatusComponentSpec/StatusSnapshot substrate here; daemon/status, CLI status, workload diagnostics, MCP, HTTP, and coordination are consumers. The stronger contract retains 703's cross-surface fact parity and adds per-component cost, freshness, deadline, last-good, invalidation, and resumable-detail semantics.\n[2026-07-15 installed-skill dogfood reproduction] MCP readiness_check synchronously assembled 23 checks into 27,673 bytes, then lost the payload at the 25 KiB boundary. The envelope said ok=true while its summary contained one error, raw materialization_ready=false with join_gap_count=23,427, and raw_frontier_integrity state=blocked. Status snapshots must make overall/degraded semantics consistent, keep the compact projection below budget before serialization, and expose exact component/detail refs instead of a whole-report retry.\n[2026-07-18 Lane F PR 1/2-3] PR #3107 (branch feature/perf/snappy-surfaces): shared\nStatusComponentSpec/StatusComponentRegistry protocol (polylogue/operations/status_protocol.py)\n+ daemon/archive status cutover. build_daemon_status() collects its ~14 facts\nthrough a fresh per-call registry (independent deadline per component, explicit\nfresh/stale/refreshing/timed_out/unavailable/degraded states, last-good evidence\nretained). daemon_status_payload()'s previously-unbounded archive_debt call is now\nbounded the same way. polylogued status asks the running daemon's /api/status first\n(honouring POLYLOGUE_DAEMON_URL, matching the archive CLI's existing #1325 pattern),\nfalling back to the now-bounded direct path only when no daemon answers.\n\nLive-archive read-only measurement (provisional, archive mid-restore from the\n2026-07-18 incident): polylogued status + live daemon \u003e60s timeout -\u003e 2.2-2.4s\n(daemon's fresh cached snapshot, age_s\u003c1); polylogued status + no daemon (direct\npath) \u003e90s timeout -\u003e ~8.5s bounded/deterministic with raw_materialization/\nembeddings correctly timing out while search/archive_storage stay fresh. Anti-\nvacuity test added (stalled collector times out without delaying a healthy sibling\n-- fails on the pre-PR synchronous chain).\n\nAC status: #1 (shared protocol, daemon consumer) satisfied for daemon/archive status;\ncoordination status consumer is the next PR. #2 (stalled component isolation) satisfied\nand proven by the anti-vacuity test + live measurement above. #3 (polylogued status\nreturns within budget) satisfied for the daemon-reachable case (2.2-2.4s, mostly cold-\nimport tax); the no-daemon direct path is bounded but not yet \"interactive\" (~8.5s) --\ntightening deadlines from measured distributions is explicitly 20d.14's job, not\ninvented here. #4 (randomized sampling + p50/p95 product budgets), #5 (coordination\nconsumer + full fingerprint-driven invalidation across all sources), #6 (resumable\ndetail-query semantics for embedding/Beads/handoff expansion) remain open, deferred to\nthe coordination-status PR and 20d.14 per the lane's PR1/PR2/PR3 cadence. #7 (payload\ncorrectness preserved) verified via the full existing test_daemon_status.py suite (55\ntests unchanged in assertions, all green) plus mypy --strict and devtools verify --quick.\n\nDeferred, named explicitly (not silently dropped): persistent daemon-lifetime registry\nwith real cross-tick staleness reuse (this PR uses a fresh ephemeral per-call registry,\ncorrect for build_daemon_status()'s existing pure-recompute contract used by ~50\nparameterized tests, but doesn't give the daemon's own periodic refresh loop cross-tick\ncaching beyond what it already had); explicit dependency-graph declarations between\ncomponents (a few facts still combine via cheap pure post-processing after independent\ncollection).\n[2026-07-18 Lane F PR 2/3] PR #3116 (branch feature/perf/coordination-status-cache):\nbounds build_coordination_envelope's archive_evidence stage (session trees, activity\nepisodes, subagent exchanges, proof refs, context-flow refs -- one unbounded SQLite\nread) to a 3s deadline via the shared StatusComponentRegistry protocol from PR #3107,\nwith an explicit degraded fallback surfaced in advisories. Live measurement: ~10s\nunbounded -\u003e capped at 3s; polylogue agents status CLI ~11s+ -\u003e ~5.1s.\n\nAlso adds CoordinationEnvelopeCache (StatusComponentRegistry-backed, fingerprint-\ninvalidated on git HEAD/logs, .beads/issues.jsonl, active index db/WAL mtimes) as\nready substrate for a warm-cached coordination-status consumer -- NOT wired to any\nlive surface in this PR.\n\nMajor scope-narrowing discovery mid-implementation: the MCP agent_coordination tool\n(polylogue/mcp/server_tools.py, register_read_tools) is dead code -- register_tools()\n(live server wiring) only calls the six-tool cutover surface\n(server_cutover.py:register_cutover_read_tools/register_cutover_privileged_tools),\nconfirmed by tracing the call graph. Its dedicated test file was already deleted by\nthe six-tool cutover (#3095) with no replacement coverage. The live, reachable path\nis status(scope=\"coordination\") in server_cutover.py, which has its OWN pre-existing\nbug: every scope value except \"operation\" falls through to archive.stats(), so\nscope=\"coordination\" silently returns archive stats, never coordination data. Filed\npolylogue-qink for wiring CoordinationEnvelopeCache into that handler + deciding\nregister_read_tools/agent_coordination's fate -- deliberately NOT attempted in PR2\nsince it's deep in another lane's actively in-flight six-tool cutover\n(feature/mcp/retire-legacy-registrars) and risks collision.\n\nAC status update: #5 (fingerprint invalidation) substrate exists (CoordinationEnvelopeCache)\nbut is unwired pending qink. #2/#7 for coordination's dominant real cost (archive_evidence)\nsatisfied and measured. Remaining coordination AC gaps (randomized sampling, full stage\nDAG atomization beyond archive_evidence, live dogfood artifact, MCP p95 budget) still\nopen, same as before -- now additionally blocked on qink for the MCP consumer specifically.\n[2026-07-18 evening, Lane F PR 3/N] PR #3128 (branch feature/perf/snappy-surfaces, same branch as PR #3107/#3116): wires status(scope=\"coordination\") on the live six-tool MCP surface to CoordinationEnvelopeCache/build_coordination_envelope (was silently falling through to archive.stats() -- filed + tracked as polylogue-qink, closing that bead on merge). This is the first LIVE MCP consumer of PR #3116's CoordinationEnvelopeCache substrate -- AC #5 (fingerprint invalidation) now has a real consumer to validate against, though full source-fingerprint coverage beyond archive_evidence/git-HEAD/beads/index-WAL is still unaudited.\n\nAlso investigated the CLI cold-start slice (polylogue-8s70) as a possible cheap PR 3: re-attempted readiness/__init__.py + readiness/capability.py TYPE_CHECKING-only deferral of storage.repair's ArchiveDebtStatus import. Measured zero wall-clock change (before/after: ~1.7s both, 3 runs each) via python -X importtime -- root cause is that polylogue/insights/archive.py ALSO imports storage.repair at module level, reached independently via cli/shared/helper_summary.py, so closing one edge does not remove the redundant one. Reverted (no benefit), evidence recorded on 8s70 for a future dedicated pass; NOT attempted as part of this lane per the lane prompt's own guidance not to sweep lazy-imports across the package for an unmeasured win.\n\nRemaining AC gaps unchanged from PR #3116's note: #4 (randomized sampling + p50/p95 product budgets), #6 (resumable detail-query semantics for embedding/Beads/handoff expansion), live dogfood artifact. These are substantial standalone increments -- recommend a fresh session/PR per item rather than folding into this branch further.\n[2026-07-18/19 evening, Lane F PR 5/N] PR #3140 (86ca3287, same branch as PRs #3128/#3131): closes AC #4 substantively for the surfaces that matter to this bead (CLI status + MCP status(scope=coordination)), via polylogue-jtwu's new route_observation substrate (see jtwu's own note for full design/scope-decision detail -- not duplicated here).\n\nConcretely: status(scope=\"coordination\") MCP calls and `polylogue status`/`polylogue agents \u003cview\u003e` CLI invocations now record real timing + component-level detail (archive_evidence_degraded flag from the coordination envelope's own advisories; daemon-reachable vs direct-fallback for CLI status) into a new bounded route_observations ops-tier table. `polylogue analyze latency` reads it back with real p50/p95, low-confidence-flagged under 5 samples. A new pytest-benchmark (tests/benchmarks/test_cli_cold_start.py) backs a real informational cli_status_cold SLO row in docs/plans/slo-catalog.yaml with a MEASURED number (p50 ~1.80s cold subprocess, 5 rounds) -- this is the \"product budgets are set from those distributions\" clause of AC #4, satisfied with a real runnable benchmark rather than a hand-typed guess.\n\nAC #4 status: \"randomized... sampling records per-component timing\" -- satisfied via real production call sites (not a synthetic sampler) for the two surfaces this bead cares about (status CLI/MCP); \"p50/p95... archive state, git head, fingerprints, cache decisions, raw artifact refs\" -- timing/status/attributes/git_head columns exist and are populated (git_head only wired for the coordination CLI path currently, not yet MCP -- small residual gap); \"product budgets are set from those distributions\" -- satisfied for cli_status_cold specifically. NOT extended to daemon-internal/HTTP status paths (jtwu's note explains why: Lane E's daemon/http.py territory this cycle).\n\nThis closes out this lane's planned work on polylogue-20d.17 for this session. Remaining AC gaps (per PR #3116/#3131's earlier notes, still open): full fingerprint-driven invalidation audit beyond coordination/archive_evidence, resumable detail-query semantics for embedding/Beads/handoff specifically (only archive_evidence got this in PR #3131), live dogfood artifact. Recommend a fresh session for those, or folding embedding/Beads resumability into jtwu's own remaining-scope list since it's the same underlying pattern (persistent StatusComponentRegistry per expensive sub-stage) proven out on archive_evidence.\n\n2026-07-26 portfolio-convergence audit: released stale in_progress claim after \u003e7 days with no recorded activity; scope remains open and must be re-claimed on real work start.\n[2026-07-28 fingerprint-invalidation audit + embedding resumability] PR #3377\n(branch feature/perf/daemon-status-embedding-resumability) closes the\n\"embedding\" leg of the remaining resumable-detail-query scope, plus a full\nfingerprint audit of every status component beyond coordination/archive_evidence.\n\nLive measurement against the real archive (/realm/db/polylogue): embedding_readiness_info\ntakes ~5.06s standalone while build_daemon_status's declared deadline_s for it\nis 2.0s. The daemon's periodic status-snapshot refresh\n(_periodic_status_snapshot_refresh, daemon/cli.py, 10s cadence for the process\nlifetime) called daemon_status_payload -\u003e build_daemon_status, which built a\nbrand-new EPHEMERAL StatusComponentRegistry every tick -- the exact\npre-#3131 archive_evidence pathology, on the daemon status side: a component\nslower than its own deadline timed out and was discarded every single tick,\nforever, never converging, plus leaking one orphaned collector thread per\ntick (a timed-out attempt cannot be cancelled). None of build_daemon_status's\n~14 components had a fingerprint either -- AC #5 gap confirmed real here too.\n\nFix: extracted the inline StatusComponentSpec list into\n_daemon_status_component_specs() shared by the existing ephemeral per-call\npath (build_daemon_status(registry=None), unchanged, all pre-existing tests\npass) and a new periodic_status_component_registry() -- one process-wide\npersistent registry, lazily built, with a real fingerprint\n(_daemon_status_fingerprint: index db + ops db + their -wal mtimes) so a\nchanged archive/ops source forces a refresh inside the ttl_s window.\nrefresh_status_snapshot's periodic call now threads this registry through\ndaemon_status_payload(registry=...).\n\nAnti-vacuity: new test\ntest_periodic_status_component_registry_resumes_slow_embedding_readiness_across_ticks\nproves the collector runs exactly once across 3 ticks (timed_out -\u003e\nrefreshing -\u003e fresh); confirmed it fails both when the registry-reuse check\nis reverted (duplicated attempt) and when refresh_status_snapshot stops\nthreading registry= through. New test\ntest_periodic_status_component_registry_fingerprint_forces_refresh proves a\nchanged index db forces a refresh inside ttl_s. Live dogfood (read-only,\n/realm/db/polylogue): tick 0 times out at 2.0s, ticks 1-2 (0.2s apart)\nobserve refreshing without re-invoking the collector, tick after ~8s total\nreturns fresh with real embedding_coverage_percent=44.1. Artifact:\n.local/coordination/20d17-embedding-resumability-dogfood.json (untracked).\ndevtools test tests/unit/daemon/test_daemon_status.py -- 63 passed. mypy\n--strict clean. devtools verify --quick exit 0.\n\nInvestigated and found NOT to need this treatment (false alarm, same\nmethodology as polylogue-dhjz's investigation): coordination/envelope.py's\n\"beads\" and \"handoff\" sub-stages, and daemon/status.py's archive_debt/\nassertion_candidate_queue ephemeral registries.\n- beads: 3 subprocess bd probes already bounded via REAL subprocess-timeout\n cancellation (0.35s each, run concurrently via ThreadPoolExecutor) -- a\n fundamentally different (and better) contract than archive_evidence's\n unbounded blocking-SQL problem, which is WHY archive_evidence specifically\n needed a background-thread StatusComponentRegistry in the first place.\n Applying that same pattern to beads would add complexity without fixing a\n measured problem.\n- handoff: a cheap filesystem glob (.agent/scratch/*handoff*.md) + a\n LIMIT-bounded SQLite query with a 0.2s connect timeout -- not expensive.\n- archive_debt / assertion_candidate_queue (daemon/status.py): both build a\n fresh ephemeral StatusComponentRegistry per call too, same shape as the\n embedding_readiness bug -- BUT verified by grepping every call site\n (daemon_status_payload(include_archive_debt=True) only from\n daemon/cli.py's status_command no-daemon CLI fallback and\n cli/shared/check_workflow.py's `polylogue check` command) that both are\n ONLY ever reached from one-shot CLI processes, never a persistent loop\n (the live daemon's /api/status route reads the cached _SNAPSHOT via\n get_status_snapshot_payload(), never calling these with\n include_archive_debt=True per-request). No cross-call state exists for a\n persistent registry to preserve there -- the ephemeral pattern is correct,\n matching build_daemon_status's own documented pure-recompute contract.\n\nRemaining AC gaps after this PR: #4's git_head column for the MCP\ncoordination path (jtwu's small residual gap, unrelated to this PR); any\nfurther daemon-side \"expensive\"/\"moderate\" component beyond embedding_readiness\nthat might independently exceed its deadline on a still-larger archive (not\nmeasured to be a live problem for the others at this archive's current scale\n-- fts_readiness/insight_freshness/raw_materialization/raw_failures/\nblob_publication_reservations/health all now share the SAME persistent\nregistry + fingerprint mechanism via periodic_status_component_registry(),\nso they get the resumability fix \"for free\" even though only\nembedding_readiness was independently confirmed to exceed its deadline via\nlive measurement this session).","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:42Z","created_by":"Sinity","updated_at":"2026-07-28T17:57:00Z","started_at":"2026-07-18T14:26:44Z","labels":["area:daemon","area:ops","area:perf","delivery:G-live-performance","horizon:frontier","lane:interactive-performance"],"dependencies":[{"issue_id":"polylogue-20d.17","depends_on_id":"polylogue-20d","type":"parent-child","created_at":"2026-07-15T06:23:42Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-20d.17","depends_on_id":"polylogue-20d.14","type":"relates-to","created_at":"2026-07-15T06:25:27Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-20d.17","depends_on_id":"polylogue-703","type":"supersedes","created_at":"2026-07-15T20:27:07Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-20d.17","depends_on_id":"polylogue-cuxz","type":"relates-to","created_at":"2026-07-15T20:17:32Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-20d.17","depends_on_id":"polylogue-s7ae.8","type":"relates-to","created_at":"2026-07-15T06:25:30Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-9itr","title":"Repair split-tier config paths readiness regression","description":"Live dogfood on 2026-07-15 found that config paths resolves the active index symlink, treats the index-only generation directory as the complete five-tier archive root, and reports four existing tiers missing. The configured index pointer and generation index are the same inode, and ordinary multi-tier reads work. This is a residual diagnostic regression after polylogue-nkmy.\n\n## Steps to Reproduce\n1. Configure durable tiers at the archive root and point index.db at an index-only active generation.\n2. Run polylogue config paths --format json.\n3. Observe source, embeddings, user, and ops reported missing under the resolved generation even though their configured paths exist.","design":"Represent diagnostic paths as an explicit tier map: configured source, embeddings, user, and ops plus the resolved active index. Reuse ArchiveIdentity instead of rebuilding siblings from the resolved index parent. Compute readiness over that map and audit sibling diagnostics for the same derivation.","acceptance_criteria":"A split-tier fixture with an index-only symlinked generation reports all five tiers present; configured and resolved paths plus active generation are explicit; restoring resolved-index-parent sibling derivation fails the fixture; ordinary source plus index reads are unchanged; focused CLI/path tests and devtools verify --quick pass.","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-15T04:23:39Z","created_by":"Sinity","updated_at":"2026-07-15T16:38:53Z","closed_at":"2026-07-15T16:38:53Z","close_reason":"Superseded by ovme ArchiveLocation. Its split-tier config-path reproduction and canary are preserved verbatim as acceptance criteria beside the phantom benchmark write regression; both arise from ambiguous archive-root/tier/generation Path handling.","labels":["area:cli","area:ops","area:storage","delivery:A-trust-floor","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-9itr","depends_on_id":"polylogue-nkmy","type":"relates-to","created_at":"2026-07-15T06:25:24Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} @@ -399,6 +400,11 @@ {"_type":"issue","id":"polylogue-sru.2","title":"Characterize ambiguous bucket: wordless continuation vs prose-without-markers","description":"Split next-turn-is-tool-call (wordless continuation) from prose-lacking-ack-markers; state counts for both. Opus-4-7 74% ambiguous vs deepseek 17% is likely turn-structure variance, not behavior — this split disambiguates.","design":"Implementation home: the claim-vs-evidence classifier in devtools (devtools/ module behind `devtools workspace claim-vs-evidence`; tests tests/unit/devtools/test_claim_vs_evidence.py). Wordless-continuation detection: for each failure's paired next assistant message, check whether its blocks contain tool_use and no text block with \u003eN chars before the first tool_use — that is 'wordless continuation'; prose without matched ack markers stays 'ambiguous-prose'. Emit both as classification_reason variants (field already exists) and add the two counts to the report summary + by_model/by_tool cuts. Regen: `devtools workspace claim-vs-evidence --limit 5000 --out-dir .agent/demos/claim-vs-evidence --json`. Acceptance: report shows ambiguous split into wordless_continuation vs prose_no_marker with counts; per-model ambiguous variance (opus-4-7 74% vs deepseek 17%) re-examined after the split.","notes":"2026-07-03 Codex WIP: unit implementation for ambiguous split passes focused tests, but live regeneration with --limit 5000 became too slow and had to be killed twice. First attempt used correlated subqueries for next-message block shape; second used set-based CTE; third used chunked second query after sampled rows, but the full command still exceeded 90s on active archive and ignored SIGINT while inside SQLite. Do not close or commit this slice until the live regeneration path is profiled/fixed. Dirty files currently show the WIP implementation: devtools/claim_vs_evidence.py and tests/unit/devtools/test_claim_vs_evidence.py. Last passing focused proof: python -m py_compile + ruff check + devtools test tests/unit/devtools/test_claim_vs_evidence.py -\u003e 3 passed.","status":"closed","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:28Z","created_by":"Sinity","updated_at":"2026-07-03T07:45:10Z","started_at":"2026-07-03T07:09:21Z","closed_at":"2026-07-03T07:45:10Z","close_reason":"Completed: claim-vs-evidence now splits ambiguous follow-ups into wordless tool continuations and prose-without-marker buckets, reports the counts in JSON/README summaries, and regenerates the current demo on the active archive. Focused tests pass; live regen/check completed.","labels":["area:substrate","campaign"],"dependencies":[{"issue_id":"polylogue-sru.2","depends_on_id":"polylogue-sru","type":"parent-child","created_at":"2026-07-03T06:31:27Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"polylogue-sru.3","title":"Benign-recovery vs consequential-silence split by handler kind","description":"Read failures are ~94% silent but 'tried another path' is usually benign; Bash/test failures are the consequential class. Scope the headline to consequential handler kinds or add an explicit split — credibility depends on not inflating with trivial recoveries.","design":"Handler kind is already available on the paired failure row (actions lane exposes handler/tool). Define the consequential set explicitly in code (Bash/test/build/write-class handlers) and the benign-recovery set (Read/Glob/Grep-class 'tried another path'), emit split headline rows: silent-proceed among consequential vs among all. Keep the mapping a named constant with a rationale comment so reviewers can argue with it. Report both; never let the headline mix classes silently. Same regen/tests as the other methodology children.","status":"closed","priority":1,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:28Z","created_by":"Sinity","updated_at":"2026-07-03T07:58:08Z","started_at":"2026-07-03T07:55:37Z","closed_at":"2026-07-03T07:58:08Z","close_reason":"Completed: claim-vs-evidence now reports a first-class handler-class split separating consequential shell/edit/write-class tool failures from benign read/search/path-discovery failures and other tools. The regenerated active-archive artifact shows consequential=4,177 failures with 921 silent-proceed (22.0% lower bound), benign_recovery=633 with 166 silent-proceed (26.2%), and other=190 with 92 silent-proceed (48.4%). Focused tests and demo shelf checks passed.","labels":["area:substrate","campaign"],"dependencies":[{"issue_id":"polylogue-sru.3","depends_on_id":"polylogue-sru","type":"parent-child","created_at":"2026-07-03T06:31:28Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"polylogue-sru.1","title":"Expose action-unit outcome fields + followup_class as product capability","description":"Capabilities-may-not-be-silos gate for the campaign: the facts the report needs must become composable query capability. After this, the whole report is `actions where is_error:true | group by session.origin, followup_class | count` and every future cut (model/tool/repo/time) is free.","design":"1) is_error/exit_code are normalized at parse time (sources/parsers/base_models.py:74-75) but ActionQueryRowPayload (surfaces/payloads.py:~1298) carries neither — add as filterable/groupable action-unit fields. 2) Add derived followup_class (acknowledged|silent_proceed|wordless_continuation|ambiguous) + followup_message_ref computed in the source-derived lowering (no cache tables). 3) Reduce devtools workspace claim-vs-evidence to a render preset over these query strings, or retire it. Touchpoint chain: stage parser -\u003e AST to_payload -\u003e executor -\u003e metadata.py aggregate_group_fields -\u003e shell_completion_values.py -\u003e devtools render openapi + cli-output-schemas + cli-reference. Line refs pre-07-03; re-locate.","acceptance_criteria":"Fixture session with known unacknowledged failure fires via pure query strings; report README numbers reproducible from the printed queries.","notes":"Completed: action-unit outcome follow-up classification is now shared query capability. is_error/exit_code were already wired; this slice added source-derived followup_class and followup_message_ref over existing actions/messages/blocks, exposed followup_class as filterable/groupable action metadata, added action row payload fields, routed root CLI terminal-unit aggregate expressions before session-selector compilation, and moved the report classifier from scripts into polylogue.archive.actions.followup. Reproduction/query forms are now printed in .agent/demos/claim-vs-evidence/PUBLIC_REPRODUCTION.md: actions where is_error:true | group by followup_class | count; actions where followup_class:silent_proceed. Verification: focused DSL/report/CLI tests passed; active demo packet regenerated over archive root /home/sinity/.local/share/polylogue schema v23 with 41,886 structured failures and 5,000 inspected; devtools verify --quick passed run 20260703T092510Z-quick-718233-46e8b587.","status":"closed","priority":1,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:27Z","created_by":"Sinity","updated_at":"2026-07-03T09:25:36Z","started_at":"2026-07-03T09:05:37Z","closed_at":"2026-07-03T09:25:36Z","close_reason":"Completed","labels":["area:query","area:substrate","campaign"],"dependencies":[{"issue_id":"polylogue-sru.1","depends_on_id":"polylogue-sru","type":"parent-child","created_at":"2026-07-03T06:31:26Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-ze5i","title":"Four of ten lab policies fail silently: policy checks sit behind --lab, which no gate runs","description":"Measured 2026-07-28 by running each policy directly:\n\n schema-versioning exit=1 undeclared index schema deltas found: 1\n demo-tour-freshness exit=1 regenerated tour output differs from docs/examples/demo-tour/\n backlog-hygiene exit=1 74 findings across 10 checks; 1,127 issues scanned\n bead-graph exit=1 missing_ac=21\n timestamp-doctrine, insight-honesty, demo-packet-registry, docs-drift,\n campaign-archive-boundaries, archive-resolver-completeness exit=0\n\nAll ten are appended inside 'if lab:' at devtools/verify.py (the --lab branch). Default 'devtools verify' does not run them; 'devtools verify --quick' (the pre-push hook) does not run them; the CI lint job runs render all --check, verify public-claims and ruff, not these. So a policy can fail continuously without blocking a merge.\n\nLive consequence: the schema-versioning violation merged in PR #3378 and the first symptom was the live archive becoming unqueryable from the repo CLI ('no such column: s.title_ref'), diagnosed only by hand days later.\n\nThis bead is the placement question, not the individual failures: which policies are cheap and deterministic enough to gate by default, which are genuinely lab-tier, and what runs the lab-tier ones on a schedule so they cannot rot. schema-versioning has already been moved to the default gate and CI lint in the same change that filed this bead; the other three remain unassigned to any gate.","acceptance_criteria":"1. Every lab policy is classified as default-gated, CI-gated, or scheduled, with the cost and determinism evidence for that placement. 2. No policy is left in a position where continuous failure blocks nothing. 3. The three currently-red unplaced policies (demo-tour-freshness, backlog-hygiene, bead-graph) are either green or have their failures triaged into owned beads. 4. A regression proves a deliberately-introduced violation of a default-gated policy fails the gate.","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:02:53Z","created_by":"Sinity","updated_at":"2026-07-28T20:02:53Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-b5l.3","title":"Fast-forward mechanism is spread across five modules and 2,713 lines with a per-version fork","description":"Measured 2026-07-28. One concept -- bring a derived generation forward without raw replay -- is implemented in five places:\n\n devtools/index_fast_forward.py 1,085 lines, 44 defs/classes\n devtools/archive_schema_fast_forward.py 985 lines, 43 defs/classes\n devtools/index_v37_fast_forward.py ~600 lines, 26 defs/classes\n polylogue/storage/sqlite/lifecycle.py 438 lines (declarations + planner)\n polylogue/storage/sqlite/archive_tiers/index_fast_forward_executor.py 205 lines (runtime executor)\n\nindex_v37_fast_forward.py is a version-specific FORK: its docstring scopes it to 'index v36 -\u003e v37', and it imports reflink_clone from archive_schema_fast_forward. A per-version copy of a mechanism whose whole point is being version-pair-generic is the smell.\n\nIts test suite is also the one that is red: polylogue-e6a0 records 9 failing tests in tests/unit/devtools/test_index_v37_fast_forward.py, root-caused to a hardcoded v36 DDL commit that predates action_pairs. So the most-forked copy is also the least-covered.\n\npolylogue-9rw0's notes already record the origin: PR #2788 independently authored devtools/index_fast_forward.py from a base predating the merged #2804/#2805 with the same filename and purpose, producing a real add/add conflict; the 2026-07-13 reconciliation kept the deployed execution mechanism authoritative and retained the plan-declaration layer, but did not collapse the modules.\n\nBound the duplication before cutting: this bead is an audit-then-collapse, not a blind delete -- the offline devtools actuator and the runtime on-connect executor may legitimately differ in clone/promote responsibilities even after the planning layer is shared.","acceptance_criteria":"1. A written map of which module owns planning, clone, proof, execution, and promotion, with the duplicated responsibilities named. 2. One planning authority (lifecycle.py declarations) consumed by every actuator; no actuator carries its own version knowledge. 3. index_v37_fast_forward.py is either generalized into the shared path or deleted with its transition recorded as a declaration; a per-version module does not survive. 4. polylogue-e6a0's 9 failing tests are resolved by the collapse rather than by repairing a fixture for a module that should not exist. 5. Line count and module count after the collapse are reported against the 2,713/5 baseline.","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:02:52Z","created_by":"Sinity","updated_at":"2026-07-28T20:02:52Z","labels":["area:daemon","area:ops","area:storage","delivery:B-storage-rebuild-bytes","horizon:frontier","lane:storage-rebuild-scale","size:L","spine"],"dependencies":[{"issue_id":"polylogue-b5l.3","depends_on_id":"polylogue-b5l","type":"parent-child","created_at":"2026-07-28T22:02:51Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-cijx.1","title":"Repository identity fragments across URL spellings and worktrees: 106 repo_ids for one repo","description":"Measured on the live archive 2026-07-28. repos is keyed (origin_url, root_path), so one repository splits across every remote-URL spelling and every checkout root:\n\n repo_name distinct repo_ids sessions\n sinex 28 4,922\n polylogue 106 4,369\n sinnix 31 1,466\n sinity-lynchpin 2 1,809\n\nFor polylogue the largest shards are: ''+/realm/project/polylogue (3,276), https://github.com/Sinity/polylogue+same root (583), git@github.com:Sinity/polylogue.git+same root (200), plus ~15 /realm/worktrees/polylogue-* roots and .claude/worktrees/* agent roots at 7-33 sessions each.\n\nThree orthogonal identity facts are conflated into one key: (1) the repository (one remote, however spelled -- empty/HTTPS/SSH), (2) the checkout root (a worktree is evidence OF a repository, not a distinct repository), (3) an unrelated directory that merely shares a basename.\n\nConsequence: any aggregate grouped by repository silently under- or over-counts, and the query field repo: resolves to whichever shard the writer happened to record. This affects every repo-scoped read surface, not one report.\n\npolylogue-cijx's design already states the target ('Repository identity survives multiple worktrees and renames and never relies on cwd alone when stronger git evidence exists') without an executable slice; this is that slice. polylogue-j5xg's closure routed session_commits here explicitly: 'rebuilt with repo-identity care'.","acceptance_criteria":"1. Remote-URL spellings that denote one remote normalize to one repository identity; the normalization is a pure function with tests over the observed spelling set (empty, https, ssh, .git suffix). 2. Checkout root becomes worktree evidence attached to a repository, not part of the identity key. 3. A basename collision between unrelated paths does not merge them. 4. Live re-measure: distinct repository identities for polylogue/sinex/sinnix collapse to one each, with worktree roots enumerable underneath. 5. The repo: query field resolves through the normalized identity, and a regression test proves a session recorded under one spelling matches a query using another.","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:02:15Z","created_by":"Sinity","updated_at":"2026-07-28T20:02:15Z","labels":["area:insights","area:interop","horizon:mid","tech-tree"],"dependencies":[{"issue_id":"polylogue-cijx.1","depends_on_id":"polylogue-cijx","type":"parent-child","created_at":"2026-07-28T22:02:15Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-cuxz.4","title":"tool_result outcome NULL is undifferentiated: refusal, absence, and unparsed share one token","description":"Measured on the live archive 2026-07-28 (5,042,564 blocks; 1,844,545 tool_result blocks):\n\n origin unknown ok err %unknown\n codex-session 879,993 127,200 10,728 86%\n claude-code-session 415,109 332,165 37,045 53%\n chatgpt-export 22,992 0 0 100%\n hermes-session 15,107 1,848 382 87%\n claude-ai-export 0 1,353 39 0%\n gemini-cli-session 0 559 19 0%\n tool_result_exit_code present: 144,616 / 1,844,545 (8%)\n\nCrucially this is NOT simply a parser gap. sources/parsers/claude/code_parser.py:309-380 shows deliberate, correct refusal: _task_output_outcome trusts toolUseResult.task only when retrieval_status=='success' (a successful poll of a FAILED command otherwise surfaces envelope is_error=false), and _mark_background_task_start overwrites is_error with None because a start acknowledgement 'must not be projected as a completed-command success'.\n\nThe defect is that three distinguishable states collapse into one NULL:\n (a) provider emitted no outcome signal at all;\n (b) provider emitted a signal the parser deliberately distrusts (the refusals above);\n (c) this provider carries a signal the parser does not yet read.\nAll three are knowable at parse time. Without the distinction, every downstream efficacy/failure/rework measure is computed over a 28% sample with unstated and non-uniform bias, and no surface can caveat it.","acceptance_criteria":"1. A typed outcome-unknown reason accompanies every NULL tool_result_is_error, populated at parse time by the code that made the decision. 2. The refusal paths in code_parser.py record their specific reason rather than a generic unknown. 3. Case (c) is enumerated per origin, so 'we do not read this provider's field' is a countable backlog rather than an invisible one. 4. Read surfaces that aggregate outcomes report coverage alongside the aggregate and refuse a bare success-rate scalar when coverage is below a declared floor. 5. Live re-measure reports the reason distribution per origin.","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:01:45Z","created_by":"Sinity","updated_at":"2026-07-28T20:01:45Z","labels":["area:storage","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-cuxz.4","depends_on_id":"polylogue-cuxz","type":"parent-child","created_at":"2026-07-28T22:01:45Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-4ts.10","title":"session_links.status and .method are NULL on every row: TopologyEdgeStatus is declared but never written","description":"Measured on the live archive 2026-07-28 (index v43, 18,871 sessions):\n\n SELECT count(*), sum(status IS NULL OR status=''), sum(method IS NULL OR method=''),\n sum(resolved_dst_session_id IS NULL) FROM session_links;\n -\u003e 9179 | 9179 | 9179 | 222\n\nEvery one of the 9,179 topology edges has empty status and empty method. TopologyEdgeStatus (unresolved/resolved/repaired/quarantined) is a declared vocabulary with no writer, so a reader cannot distinguish a resolved parent from an asserted-but-absent one except by the weaker proxy resolved_dst_session_id IS NULL (222 rows).\n\nLink-type distribution: subagent 8,824 | continuation 308 | sidechain 31 | branch 16.\n\nConsequences: resume/continuity composition can compose from an unverified parent reference with no typed signal; polylogue-xl25's 'quarantined' BlockAnchorState has no source to read; any lineage-integrity claim rests on a column that is uniformly empty.","acceptance_criteria":"1. Every session_links row written by resolve_session_links_for_session carries a TopologyEdgeStatus value and a method token; no code path writes an empty status. 2. Existing rows acquire status through ordinary derived-tier rebuild, not a bespoke backfill script. 3. A reader can filter edges by status, and composition refuses (or degrades visibly) on a non-resolved parent rather than silently composing. 4. Live re-measure shows zero empty status/method rows and a status distribution consistent with the 222 unresolved-destination rows.","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:01:17Z","created_by":"Sinity","updated_at":"2026-07-28T20:01:17Z","labels":["area:lineage","delivery:F-lineage-compaction","horizon:frontier","lane:lineage-compaction"],"dependencies":[{"issue_id":"polylogue-4ts.10","depends_on_id":"polylogue-4ts","type":"parent-child","created_at":"2026-07-28T22:01:17Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-l2cd","title":"Migrate paths/_roots.py duplicate ArchiveLocation resolver call sites","description":"Follow-up from polylogue-ovme.2.1 (PR #3382): devtools/verify_archive_resolver_completeness.py now inventories every call site of polylogue/paths/_roots.py's four ArchiveLocation-duplicating resolvers (active_index_db_path: 43 call sites, resolve_active_index_db_path: 7, sibling_index_db: 16, archive_file_set_root_for_paths: 27 -- 93 total) and prevents growth beyond a recorded baseline, but none of the 93 existing call sites were migrated -- that was explicitly deferred as too large/risky for one session. This bead tracks the actual migration: retire each resolver's call sites in favor of the equivalent ArchiveLocation accessor (active_index_path / configured_tier / active_tier), one resolver/file-batch at a time (largest call-site count last per polylogue-ovme.2.1's own design note), shrinking BASELINE_CALL_SITES in devtools/verify_archive_resolver_completeness.py as each batch lands, verified per-batch with devtools test on the affected directory plus mypy --strict. Full removal ultimately allows deleting the four resolver functions from polylogue/paths/_roots.py entirely.","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T19:34:58Z","created_by":"Sinity","updated_at":"2026-07-28T19:34:58Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-ovme.2.1","title":"Finish ArchiveLocation migration: IndexGenerationStore, online bulk_rebuild, duplicate path resolvers","description":"Remaining scope from polylogue-ovme.2 (migrate storage/status/maintenance/transitions to ArchiveLocation): (1) IndexGenerationStore.__init__ (polylogue/storage/index_generation.py) still takes a bare archive_root: Path and manually re-derives .index-active-pointer/generations_root/transactions_root logic that duplicates ArchiveLocation.resolve()'s pointer-following, plus performs first-touch pointer bootstrapping (writing the anchor file) that ArchiveLocation.resolve() deliberately does not do -- migrating its constructor to accept ArchiveLocation (or a dedicated classmethod) touches ~4 production call sites (daemon/bulk_rebuild.py x3, storage/sqlite/archive_tiers/archive.py) plus ~30 test call sites (tests/unit/storage/test_index_generation.py, tests/unit/daemon/test_bulk_rebuild.py, tests/unit/devtools/test_index_v37_fast_forward.py, tests/unit/storage/test_incremental_rebuild_equivalence.py, tests/unit/daemon/test_embedding_orphan_reconcile_daemon.py) and needs careful verification of the bootstrap-write behavior. (2) OwnedArchiveLocation is now wired into the OFFLINE rebuild path (polylogue/maintenance/rebuild_index.py, ovme.2) but NOT into daemon/bulk_rebuild.py's three IndexGenerationStore construction sites (the online/daemon-driven rebuild path) or any devtools benchmark/scale campaign -- per the ovme parent epic's own docstring examples, campaigns are explicitly ovme.3 scope, but the online bulk_rebuild path is arguably ovme.2 scope and was not reached this session. (3) polylogue/paths/_roots.py carries at least four resolvers that duplicate or bypass ArchiveLocation's pointer/tier resolution instead of delegating to it: active_index_db_path() (duplicate .index-active-pointer read, ~25 call sites), resolve_active_index_db_path() (same duplicate logic, ~6 call sites including polylogue/daemon/status.py and polylogue/daemon/health.py), sibling_index_db() (literal sibling-derivation-from-anchor-parent anti-pattern, ~18 call sites across daemon/cli/storage/insights), and archive_file_set_root_for_paths() (derives root from db_anchor.parent when name==index.db, ~20 call sites across cli/api/archive/mcp read paths). These span nearly every read-path surface (cli, mcp, api, daemon, insights), well beyond storage/status/maintenance/transition boundaries ovme.2 named -- full removal needs its own scoped slice with a static grep/completeness check (AC4's remaining ask) rather than being folded into ovme.2's already-large diff.","design":"Suggested sequencing: (a) add a devtools static/grep completeness check that flags new ambiguous db_path/root Path parameters at archive/generation boundaries (the AC4 gate ovme.2 did not add); (b) migrate IndexGenerationStore's constructor behind a classmethod that accepts ArchiveLocation while preserving its first-touch pointer-bootstrap write; (c) wire OwnedArchiveLocation into daemon/bulk_rebuild.py's three call sites; (d) retire paths/_roots.py's four duplicate resolvers one at a time (largest call-site count last), replacing each call site with the equivalent ArchiveLocation accessor, verified per-batch with devtools test on the affected directory plus mypy --strict.","notes":"Session 2026-07-28 (worktree agent-a21a71d5e2d6d8f90): PR #3382 (feature/storage/archive-location-generation-store-migration), not yet merged.\n\nLANDED (all three pieces named in this bead):\n\n1. IndexGenerationStore constructor migration (polylogue/storage/index_generation.py):\n __init__ now takes ArchiveLocation instead of a bare archive_root: Path.\n Preserved the first-touch .index-active-pointer bootstrap write verbatim\n (ArchiveLocation.resolve() is a pure read and deliberately never performs\n it) plus the .index-generations-anchor sanity check. Added\n for_archive_root(Path) classmethod for callers that only have a raw root.\n Updated all ~4 production call sites (daemon/bulk_rebuild.py,\n maintenance/rebuild_index.py via owned.location, storage/sqlite/\n archive_tiers/archive.py, devtools/index_v37_fast_forward.py x2) and all\n ~30 test call sites (test_index_generation.py, test_bulk_rebuild.py,\n test_index_v37_fast_forward.py, test_incremental_rebuild_equivalence.py,\n test_embedding_orphan_reconcile_daemon.py).\n Anti-vacuity: test_bootstrap_writes_active_pointer_anchor_on_first_touch\n (proves bootstrap write survives the migration) and\n test_store_trusts_the_passed_location_instead_of_rereading_disk (proves\n the retired duplicate-derivation bug class -- re-reading the anchor from\n disk independently of a caller's already-resolved ArchiveLocation -- is\n now impossible).\n Commits: bff6670cc.\n\n2. Online bulk_rebuild ownership (polylogue/daemon/bulk_rebuild.py):\n resolve_or_start_daemon_bulk_rebuild_transaction now resolves an\n ArchiveLocation and acquires OwnedArchiveLocation before any\n discard_if_inactive/discard_transaction/create_transaction mutation,\n releasing in a finally -- same shape rebuild_index_from_source already\n uses for the offline path. has_resumable_daemon_bulk_rebuild_transaction\n and run_daemon_bulk_rebuild_pass's own IndexGenerationStore construction\n are left as plain reads (no ownership proof needed; the actual write pass\n goes through rebuild_index_from_source_sync, which already owns the\n location for the duration of its write).\n Residual (documented in code + PR body): IndexGenerationStore's\n constructor still performs its one-time idempotent anchor-bootstrap write\n before ownership is proven, needed to answer the read-only \"does a\n resumable transaction already exist\" fast path -- pre-existing, narrow,\n unchanged from before this session, not something this change introduces.\n Anti-vacuity: new tests/unit/daemon/test_bulk_rebuild_ownership.py mirrors\n test_rebuild_index_ownership.py exactly -- confirmed both new tests fail\n (AttributeError, since the un-migrated call site can't even construct\n IndexGenerationStore against the now-typed constructor) against the\n pre-piece-2 bulk_rebuild.py via git stash.\n Commit: 85e5d588e.\n\n3. Archive-resolver completeness lint (devtools/verify_archive_resolver_completeness.py):\n a grep-based static scan, same shape as verify_campaign_archive_boundaries.py\n (ovme.3), inventorying every call site of paths/_roots.py's four\n duplicating resolvers (active_index_db_path: 43 call sites,\n resolve_active_index_db_path: 7, sibling_index_db: 16,\n archive_file_set_root_for_paths: 27 -- 93 total, all currently baselined).\n Fails when a NEW call site appears in a file outside the recorded\n BASELINE_CALL_SITES map; shrinking the baseline (migrating a call site to\n ArchiveLocation) is always safe. Wired into `devtools verify --lab` as\n \"lab policy archive-resolver-completeness\", registered in\n command_catalog.py, docs/devtools.md regenerated.\n This is the completeness/visibility half of AC4 -- it does NOT migrate\n any of the 93 existing call sites (deliberately deferred: full migration\n was judged too large/risky for one session, matching this bead's own\n design note (d) \"retire ... one at a time ... verified per-batch\").\n Commit: f7a9fc853.\n\nVERIFICATION: mypy --strict, ruff check/format --check clean on all touched\nfiles. devtools test per-piece (22 + 6 + 10-pre-existing-failures + 2 + 4 +\n91-with-2-pre-existing-failures = confirmed all NEW failures are zero;\nevery failure classified via git stash A/B as byte-for-byte identical\nbefore/after this session's changes: test_index_v37_fast_forward.py's 10\nfailures are polylogue-e6a0 (named in this bead's own task description);\ntest_verify.py's 2 failures are the stale \"lab policy bead-graph\" gap\n(named in ovme.3's notes) and a pre-existing \"verify hash-boundary-census\"\nlist-order mismatch). devtools render all --check clean. Pre-push quick\ngate (17 steps: format/lint/mypy/render/topology/layering/closure-matrix/\nmanifests/ci-workflows/doc-commands/docs-coverage/test-infra-currency/\ntest-clock-hygiene/pytest-timeout-overrides/degrade-loudly/\nhash-boundary-census) all green.\n\nAC STATUS (this bead had no formal AC list; its description IS the scope\nstatement per all three numbered items):\n1. IndexGenerationStore migration: DONE, verified.\n2. Online bulk_rebuild ownership: DONE for the transaction resolve/retire\n mutation path; the store's own bootstrap-write residual noted above is\n an honest, narrow, pre-existing gap, not claimed closed.\n3. Duplicate path-resolver retirement: NOT migrated (93 call sites remain);\n completeness/tracking gate landed instead, per this bead's own design\n note (a)+(d) sequencing and the parent epic's explicit scoping-down\n language (\"full removal needs its own scoped slice\"). Follow-up: a new\n bead should track the actual call-site-by-call-site migration, to be\n filed once this PR merges (not filed yet this session -- flagging here\n so it isn't lost).\n\nPR: https://github.com/Sinity/polylogue/pull/3382 (not yet merged by this\nsession; per repo policy, agent does not merge its own PR without CI/triage\ncompleting, and this session leaves that step to the next review pass).","status":"closed","priority":2,"issue_type":"task","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T18:52:25Z","created_by":"Sinity","updated_at":"2026-07-28T19:35:11Z","started_at":"2026-07-28T19:31:34Z","closed_at":"2026-07-28T19:35:11Z","close_reason":"Fixed and merged via PR #3382 (feature/storage/archive-location-generation-store-migration): (1) IndexGenerationStore constructor migrated to accept ArchiveLocation, preserving first-touch pointer-bootstrap write; (2) OwnedArchiveLocation wired into the online/daemon-driven bulk_rebuild transaction resolve/retire path, mirroring the offline path; (3) devtools lab policy archive-resolver-completeness lands the completeness/visibility gate AC4 asked for, inventorying all 93 call sites of the four duplicate resolvers and preventing growth. Actual migration of those 93 call sites was explicitly out of scope (too large/risky for one session, per this bead's own design note) and is tracked in the new follow-up polylogue-l2cd.","labels":["area:daemon","area:ops","area:storage","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-ovme.2.1","depends_on_id":"polylogue-ovme.2","type":"parent-child","created_at":"2026-07-28T20:52:24Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-dpc2","title":"Parse-failure retry tracking: 59 truncated-read raws show mixed cursor/history correlation","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T12:22:56Z","created_by":"Sinity","updated_at":"2026-07-28T12:22:56Z","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -537,7 +543,7 @@ {"_type":"issue","id":"polylogue-rxdo.9.3","title":"Pre-registration with graph-provable ordering + registered badge","description":"Rigor mechanism C. A REGISTERED analysis = finding-candidate with expected set and statistic null, citing a query hash; a later run fills actuals. Ordering provable from the graph: registration timestamp \u003c query_run timestamp AND run archive_epoch \u003e registration epoch (tested on data that arrived after the hypothesis). Claims render 'confirmed (pre-registered)' ONLY under that ordering, else 'exploratory' — a provable badge, not moral credit. Prior art: polylogue-e5b5. DEP: rxdo.4 + rxdo.3.","design":"Pre-registration is a versioned assertion/ExperimentDefinition state whose timestamp and archive epoch\nprecede assignment/exposure/query execution. Bind hypothesis/expected result, query+MetricDefinition,\nframe, exclusions, stopping, and analysis plan. Later executions link rather than mutate it. Registered\nstatus is graph-provable; missing/late fields render exploratory. Reuse stc for experiments and finding\nassertions for non-experimental registered claims; do not create another registry object.","acceptance_criteria":"A pre-execution registration plus later evaluation earns registered status; reversing timestamps,\nchanging the metric/query after exposure, or using data already observed renders exploratory/new-version.\nGraph traversal proves ordering and all fields. Post-hoc metrics remain separated. Removing the ordering\ncheck makes the production fixture fail.","notes":"Implemented: polylogue/insights/measurement/registration.py -- evaluate_registration() proves pre-registration ordering from the graph: 'registered' only when run_at \u003e registered_at AND run_epoch \u003e registered_epoch AND the bound metric/query refs are unchanged between registration and run; otherwise renders an explicit exploratory / exploratory-post-hoc / exploratory-definition-drift status (never silently upgraded). Pure function over injected registration+run records -- durable finding.v1/ExperimentDefinition storage that would supply real graphs is explicitly deferred (doesn't exist in this tree yet; rxdo.4/rxdo.3 dependency). Verification: devtools test tests/unit/insights/measurement/test_registration.py -\u003e passing (covers reversed timestamps, changed metric/query after exposure, and already-observed data cases rendering exploratory). devtools verify --quick -\u003e exit 0. PR: https://github.com/Sinity/polylogue/pull/2888 (open, not merged).","status":"closed","priority":2,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-13T00:55:44Z","created_by":"Sinity","updated_at":"2026-07-15T00:00:04Z","closed_at":"2026-07-15T00:00:04Z","close_reason":"Satisfied by PR #2888 (registration.py): pre-registration ordering proof (run_at\u003eregistered_at AND run_epoch\u003eregistered_epoch AND refs unchanged), else exploratory/exploratory-post-hoc/exploratory-definition-drift status. Independently reviewed (approved).","labels":["area:substrate","delivery:C-read-evidence-contract","horizon:mid","lane:read-contracts","tech-tree"],"dependencies":[{"issue_id":"polylogue-rxdo.9.3","depends_on_id":"polylogue-rxdo.9","type":"parent-child","created_at":"2026-07-13T02:55:44Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-rxdo.9.2","title":"Ratios as derived objects: numerator_ref + denominator_ref","description":"Rigor mechanism B. No bare percentages in finding.v1 consumption: a proportion cites two result-set refs (or result-set + cohort). Renderer always answers '% of WHAT'. fnm.1's merged aggregate machinery already computes explicit denominators/n/unknown buckets — this is a schema/renderer convention, not new computation. DEP: rxdo.4 (finding.v1).","design":"Represent a ratio as a canonical MetricDefinition whose formula references numerator and denominator\nrelation/query/metric refs plus grain, frame, null/unknown policy, and measurement authorities. The\ncomputed value is a normal metric result/evaluation receipt, not a new durable ratio-object family.\nRenderer always exposes numerator, denominator, excluded/unknown buckets, and compatibility checks.","acceptance_criteria":"Equivalent numerator/denominator definitions yield one metric identity. Grain/frame/authority mismatch\nfails closed. A seeded unknown bucket is neither dropped nor coerced to zero. Every percentage renders\nnumerator, denominator, frame, and null policy, and the result resolves through rxdo.9.1/9l5.7 without a\nsecond ratio registry/table.","notes":"Implemented: polylogue/insights/measurement/ratio.py -- ratios as derived MetricDefinitions over numerator_ref/denominator_ref component pairs (build_ratio_definition), failing closed on grain/frame/measurement_authority incompatibility between numerator and denominator. RatioResult always carries an explicit unknown bucket (never dropped or coerced to zero) regardless of null_policy. Equivalent numerator/denominator definitions resolve through the same canon.py content_ref as rxdo.9.1 -- no second ratio registry/table. Verification: devtools test tests/unit/insights/measurement/test_ratio.py -\u003e passing. devtools verify --quick -\u003e exit 0. PR: https://github.com/Sinity/polylogue/pull/2888 (open, not merged).","status":"closed","priority":2,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-13T00:55:39Z","created_by":"Sinity","updated_at":"2026-07-15T00:00:04Z","closed_at":"2026-07-15T00:00:04Z","close_reason":"Satisfied by PR #2888 (ratio.py): ratios as derived MetricDefinitions over numerator/denominator refs, fail-closed on grain/frame/authority mismatch, explicit unknown bucket preserved regardless of null_policy. Independently reviewed (approved, no blocking findings): tests/unit/insights/measurement/, 116 passed.","labels":["area:substrate","delivery:C-read-evidence-contract","horizon:mid","lane:read-contracts","tech-tree"],"dependencies":[{"issue_id":"polylogue-rxdo.9.2","depends_on_id":"polylogue-rxdo.9","type":"parent-child","created_at":"2026-07-13T02:55:38Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-cuxz.1","title":"Retrofit time_confidence contract into predecessor consumers + ArchiveStore reads","description":"Follow-through on cuxz's merged consumer contract (#2786): (1) z29t (#2576), rvtu (#2575), 2seq (#2577) shipped before the contract with a bare COALESCE inclusion pattern and no time_confidence signal — retrofit them to consume it (cuxz note records this sequencing miss); (2) live ArchiveStore-backed API/CLI/MCP reads still drop the stored source tag in storage/sqlite/archive_tiers/ (deferred out of the temporal-provenance lane's footprint). Storage-tier ownership. AC: the three predecessor query paths emit time_confidence; archive_tiers reads forward the stored tag end-to-end; parity fixture proves payload shape.","design":"Define one TimeEvidence projection adapter from stored temporal provenance into EvidenceValue: resolved timestamp, source kind, time_confidence, definition/evidence refs, and unknown/degraded state. Route z29t, rvtu, 2seq, and ArchiveStore result models through it before CLI/API/MCP serialization; remove COALESCE-derived confidence and surface-local defaults. Exact, provider-derived, filesystem-derived, and unknown fixtures cross the real SQL/read/renderer path, and a consumer census fails when a timestamp-bearing public model drops the evidence fields.","acceptance_criteria":"1. z29t, rvtu, and 2seq query paths emit the stored `time_confidence` value rather than inferring confidence from COALESCE. 2. ArchiveStore-backed API, CLI, and MCP readers forward the tag end to end with surface parity. 3. Exact, derived, and unknown timestamp fixtures retain distinct provenance through filtering, ordering, serialization, and rendering. 4. Missing tags remain unknown and never become exact by default. 5. Focused predecessor, archive-tier, and cross-surface tests fail if the production tag forwarding is removed.","notes":"Horizon classification 2026-07-15: current executable contract or program; classified frontier rather than leaving P2 scheduling ambiguous.","status":"open","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:43:18Z","created_by":"Sinity","updated_at":"2026-07-15T19:27:26Z","labels":["area:storage","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-cuxz.1","depends_on_id":"polylogue-cuxz","type":"parent-child","created_at":"2026-07-13T01:43:17Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-cuxz.1","depends_on_id":"polylogue-cuxz.2","type":"blocks","created_at":"2026-07-15T20:17:30Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0} -{"_type":"issue","id":"polylogue-0v5b","title":"Cap browser-extension test worker concurrency","description":"Evidence 2026-07-13: extension-redesign lane's npm test spawned a 32-process vitest/jest worker swarm inside an 8G sinnix-background scope; systemd-oomd killed the whole scope mid-iteration (session survived via resume). Cap workers in the extension test config (e.g. vitest maxWorkers/poolOptions or npm test wrapper) so the suite fits agent scopes. AC: npm test peak RSS stays under scope limits with workers capped; suite runtime regression acceptable (\u003c2x).","design":"Make test resource envelopes part of the verification-lane declaration. The browser-extension lane declares worker-count, memory expectation, timeout, watch-mode policy, and CI/local overrides once; the runner translates that declaration into the actual Vitest/Jest pool options and emits an execution receipt with effective workers, duration, and peak RSS. The risk model treats an ignored/missing envelope as an escape risk. Preserve useful parallelism within the measured envelope rather than hard-coding a machine-specific single-worker policy.","acceptance_criteria":"1. The browser-extension test runner has an explicit worker cap honored in local, agent-scope, and CI invocations. 2. A representative full extension suite records peak RSS below the configured background-scope limit and completes without oomd termination. 3. Runtime remains below twice the uncapped baseline on the same machine/corpus, or the measured tradeoff is explicitly accepted. 4. Focused and watch modes retain expected parallelism, and a config test fails if the cap is removed or ignored.","notes":"Priority correction 2026-07-15: promoted P3 to P2 during invariant review. The bead covers a current single-writer, resource-containment, durable-lifecycle, verification-gate, or interactive-latency contract with concrete evidence; promotion does not automatically admit it to the active execution set.","status":"open","priority":2,"issue_type":"chore","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:42:51Z","created_by":"Sinity","updated_at":"2026-07-15T19:53:05Z","labels":["area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-0v5b","depends_on_id":"polylogue-88jp","type":"parent-child","created_at":"2026-07-15T19:06:55Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-0v5b","title":"Cap browser-extension test worker concurrency","description":"Evidence 2026-07-13: extension-redesign lane's npm test spawned a 32-process vitest/jest worker swarm inside an 8G sinnix-background scope; systemd-oomd killed the whole scope mid-iteration (session survived via resume). Cap workers in the extension test config (e.g. vitest maxWorkers/poolOptions or npm test wrapper) so the suite fits agent scopes. AC: npm test peak RSS stays under scope limits with workers capped; suite runtime regression acceptable (\u003c2x).","design":"Make test resource envelopes part of the verification-lane declaration. The browser-extension lane declares worker-count, memory expectation, timeout, watch-mode policy, and CI/local overrides once; the runner translates that declaration into the actual Vitest/Jest pool options and emits an execution receipt with effective workers, duration, and peak RSS. The risk model treats an ignored/missing envelope as an escape risk. Preserve useful parallelism within the measured envelope rather than hard-coding a machine-specific single-worker policy.","acceptance_criteria":"1. The browser-extension test runner has an explicit worker cap honored in local, agent-scope, and CI invocations. 2. A representative full extension suite records peak RSS below the configured background-scope limit and completes without oomd termination. 3. Runtime remains below twice the uncapped baseline on the same machine/corpus, or the measured tradeoff is explicitly accepted. 4. Focused and watch modes retain expected parallelism, and a config test fails if the cap is removed or ignored.","notes":"Priority correction 2026-07-15: promoted P3 to P2 during invariant review. The bead covers a current single-writer, resource-containment, durable-lifecycle, verification-gate, or interactive-latency contract with concrete evidence; promotion does not automatically admit it to the active execution set.\n2026-07-28: Implemented + PR opened (not merged), https://github.com/Sinity/polylogue/pull/3383 (feature/fix/cap-extension-test-workers). Root cause corrected from the bead's framing: the swarm was Vitest's default 'forks' pool (child_process per test file since Vitest 2.0), not worker_threads — poolOptions.threads alone would have been a no-op. Fix: browser-extension/vitest.config.js derives maxWorkers (default 4, mirrors devtools/verify.py DEFAULT_TESTMON_WORKERS), wires into poolOptions.forks + poolOptions.threads + top-level test.maxWorkers/minWorkers fallback, with a validated POLYLOGUE_EXTENSION_TEST_WORKERS env override. One config covers vitest run (local/CI/agent-scope) and watch mode -- no separate CI test command exists. Added tests/vitest_config.test.js as a config-shape regression guard (parses source text rather than importing the live config module, since re-importing vitest.config.js inside this suite's jsdom env trips an esbuild startup invariant). Measured on the 24-core dev workstation (corrected RSS accounting -- sum VmRSS once per distinct PID, not per pstree-listed thread/LWP): uncapped default-fork-pool baseline 27 processes / ~2.86GB peak RSS / 9.5s wall; capped at 4 workers 10 processes / ~1.1GB peak RSS / 9.6s wall (no runtime regression); env override to 8 workers scales to 13 processes / ~1.56GB. Focused single-file run (tests/common.test.js) 349ms, parallelism unaffected. All 4 AC satisfied per PR body. Found + filed a pre-existing unrelated flaky test (tests/build.test.js backfill archive vi.waitFor timing assertion, fails identically at 4/8/24 workers) as polylogue-07pt rather than fixing it here.","status":"open","priority":2,"issue_type":"chore","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:42:51Z","created_by":"Sinity","updated_at":"2026-07-28T19:55:23Z","labels":["area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-0v5b","depends_on_id":"polylogue-88jp","type":"parent-child","created_at":"2026-07-15T19:06:55Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-of39","title":"Post-billing-unlock CI re-verification sweep","description":"GitHub Actions was account-billing-locked the whole night of 2026-07-12/13; 16+ fanout PRs merged to master on local gates only (devtools verify --quick + focused tests + review-fleet evidence; heavy suite runs post-merge normally but could not). When billing unlocks: gh run rerun --failed for master workflows (CI, Nix, CodeQL, typecheck) across the merged range (#2772..#2805+), triage any red as regression-vs-infra, and re-enable the merge-train assumption that CI gates work. AC: master workflows green or every red triaged with a fix/issue; note the outcome on this bead.","design":"RUNBOOK (design pass 2026-07-13). Trigger: GitHub billing unlock (watch: any Actions job starts instead of instant-failing).\n1. Re-enable the 30 disabled repository workflows (Polylogue 17, Sinex 10, Sinnix 2, Lynchpin 1 -- inventory in notes) via gh workflow enable; restore any branch-protection required checks that were relaxed.\n2. On polylogue master HEAD: dispatch the heavy test suite + full workflow set (gh workflow run / gh run rerun --failed for the merge-window SHAs where rerun is still possible).\n3. Classify every failure against the 2026-07-12/13 local-gate merge log (the ~45-PR train: 2770s-2830s). For each: (a) latent defect the local gates missed -\u003e file bug bead citing the merging PR; (b) infra/flake -\u003e d45p flake-ledger evidence; (c) environment drift -\u003e fix workflow.\n4. Report the delta as the e6ja adjudication dataset: what did local-gate-only merging actually miss? Close this bead with that written verdict (it is the falsification receipt for the local-verify attestation option).","acceptance_criteria":"1. While account billing is locked, every repository-owned workflow is disabled and no branch-protection rule requires an unavailable check. 2. Local publish gates and their limitations are documented on affected repositories; disabled CI is never described as green. 3. After an explicit billing-unlock decision, workflows are re-enabled intentionally and one default-branch run per workflow is classified as product regression, infrastructure failure, or green. 4. Every product regression receives an owning bead and verification receipt; infrastructure failures remain named rather than retried indefinitely.","notes":"BILLING-LOCK RESPONSE 2026-07-13: confirmed GitHub annotation 'The job was not started because your account is locked due to a billing issue.' Disabled all 30 repository-owned workflows: Polylogue 17, Sinex 10, Sinnix 2, Lynchpin 1. Left GitHub-managed Copilot/Dependabot services untouched. Polylogue branch protection has no required status checks, so disabled runners cannot deadlock merging. Re-enable only after an explicit billing-unlock decision and run the classified sweep in this bead's AC.\nRELEASE-PLEASE ADDENDUM 2026-07-13: the stale v0.3.0 PR (#2701, 100+ commits behind) is a symptom of the billing lock — release-please regenerates on every master push via Actions, which is disabled. On unlock, verify #2701 catches up automatically; the 0.3 SCOPE decision is tracked separately (decision bead: v0.3.0 release scope).","status":"open","priority":2,"issue_type":"chore","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:42:18Z","created_by":"Sinity","updated_at":"2026-07-13T09:43:08Z","labels":["area:verification","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-of39","depends_on_id":"polylogue-88jp","type":"parent-child","created_at":"2026-07-15T18:54:37Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-p5r4","title":"Validate fast-forward samples by replaying source evidence","description":"The declared index fast-forward actuator currently proves clone structural preservation by hashing representative index rows before and after SQL. It does not independently rebuild sampled sessions from retained raw source bytes, so it cannot establish parser/reparse equivalence. Implement a clone-safe validation lane that selects a bounded deterministic sample of source-backed sessions, rebuilds those sessions through the production replay/materialization route into an owned inactive generation, hash-compares normalized material against the SQL fast-forward clone, records the evidence in the receipt, and rejects activation on mismatch. Preserve the explicit parser-drift caveat: a semantic-reparse declaration remains a rebuild/reprocess route, not permission to run SQL fast-forward.\\n\\nDiscovered during adversarial review of polylogue-9rw0 / PR #2788.","design":"DESIGN (2026-07-13, grounded in merged #2788/#2804/#2805): the fast-forward executor proves clone structural preservation (counts + row hashes); this bead adds SEMANTIC equivalence — replaying source evidence for a bounded sample and comparing against fast-forwarded rows.\nMECHANISM: (1) sample manifest — deterministic seed, per-table stratified sample of session_ids from the receipt's structural_counts, recorded in the receipt; (2) for each sampled session, re-run parse+materialize from source.db raw_sessions into a THROWAWAY schema-current index (the demo/reprocess path already does this) and diff the normalized rows (sessions/messages/blocks + FTS search_text) against the fast-forwarded clone's rows for the same ids; (3) the comparison must be canonical-form aware (NFC, generated columns excluded, insight tables excluded — they rematerialize); (4) verdict + per-table mismatch counts land in the receipt as equivalence_sample; any mismatch fails validate.\nWIRING: extend devtools index-fast-forward validate with --replay-sample N; consume lifecycle.py plan declarations to know which tables a delta touched (only those need replay comparison — index-only deltas can skip to hash checks). Closing this also satisfies 9rw0's last open AC clause (see its design).\nPITFALL: replay must pin the SAME parser version the archive used or mismatches are parser drift, not fast-forward corruption — record parser fingerprint in the receipt and compare fingerprints before diffing rows.","acceptance_criteria":"A non-semantic fast-forward receipt records a bounded source-backed replay sample and hash comparison; mutating the replay result or bypassing the replay fails a production-route test; activation rejects failed/missing replay evidence; semantic deltas remain rebuild/reprocess-only.","status":"closed","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:28:33Z","created_by":"Sinity","updated_at":"2026-07-14T23:22:54Z","closed_at":"2026-07-14T23:22:54Z","close_reason":"Superseded by polylogue-9rw0: source-backed replay sampling was the sole deferred fast-forward acceptance criterion and is now explicit in the owning plan/proof bead.","labels":["area:storage","delivery:B-storage-rebuild-bytes"],"dependencies":[{"issue_id":"polylogue-p5r4","depends_on_id":"polylogue-9rw0","type":"discovered-from","created_at":"2026-07-13T01:28:33Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-s01p","title":"Ingest complete Beads issue history and workspace intake","description":"The Beads interaction-ledger parser ingests only issues that appear in .beads/interactions.jsonl after an explicit import. Complete the source so all repository issues have baseline/history evidence and normal configured intake can discover a Beads workspace without relying on an operator-supplied ledger path.","design":"Define an acquired, repository-scoped Beads export/bundle that combines current issue snapshots with interaction history (and, where available, Dolt or issues.jsonl git history) before parser lowering, so one issue session is materialized without separate imports overwriting each other. Add an explicit configured source or watcher discovery contract for .beads; do not make parsers read sibling live files implicitly.","acceptance_criteria":"A fixture with an issue absent from interactions produces a stable baseline session/event; importing the full Beads bundle preserves baseline plus later interactions in one session; a configured normal intake discovers the workspace and reaches acquire→detect→parse→store; focused tests use the real intake route. Cross-session Bead correlation remains out of scope and is tracked separately by polylogue-za9y.","notes":"[2026-07-14 reconciliation] Closed with no close_reason recorded. Confirmed satisfied by PR #2800 (feat(sources): ingest Beads issue histories, merged 2026-07-13T00:32:53Z): adds explicit-import ingestion for the Beads interaction ledger as the beads-issue origin, each observed interaction becomes searchable text plus a structured event in a deterministic workspace-scoped issue timeline session.","status":"closed","priority":2,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-12T23:10:23Z","created_by":"Sinity","updated_at":"2026-07-15T01:12:17Z","closed_at":"2026-07-14T23:19:28Z","labels":["area:ingest","delivery:D-agent-context-coordination","horizon:frontier","lane:agent-substrate"],"dependencies":[{"issue_id":"polylogue-s01p","depends_on_id":"polylogue-1vpm.6","type":"supersedes","created_at":"2026-07-15T01:19:28Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-s01p","depends_on_id":"polylogue-7fj","type":"discovered-from","created_at":"2026-07-13T01:10:23Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} @@ -720,6 +726,10 @@ {"_type":"issue","id":"polylogue-rii.1","title":"Agent work-event write-leg -\u003e session_events -\u003e materialized read-models","description":"record_work_event/emit_decision write surface routed through the existing idempotent ingest seam (no parallel writer); flows into the run-projection read models. Today agents can only record_correction/blackboard_post/tag — there is no 'I ran this tool / spawned this subagent / decided X' write. GH issue thread (body + comments) is input, not authority; this bead's scope statement wins where they conflict.","design":"Route through the existing idempotent ingest seam (write_raw_and_parsed / the daemon ingest path) — no parallel writer (gh#2459 body is code-grounded here). Surface: MCP tools record_work_event/emit_decision (mutation role) accepting typed events (tool run, subagent spawn, decision, artifact change) with evidence/session refs; land in session_events; run-projection read models pick them up through the normal materializer. MCP registration trap: EXPECTED_TOOL_NAMES + TOOL_CONTRACT + role gating + render openapi/cli-output-schemas regen (see bd memories). Acceptance: an agent posts a work event mid-session; it is queryable via observed-events within one convergence cycle; re-posting is idempotent.","acceptance_criteria":"- MCP tools record_work_event / emit_decision are registered with the mutation role: EXPECTED_TOOL_NAMES + TOOL_CONTRACT updated, role gating enforced, and `devtools render openapi \u0026\u0026 devtools render cli-output-schemas` regenerated with `devtools render all --check` clean.\n- Typed events (tool run, subagent spawn, decision, artifact change) with evidence/session refs route through the existing idempotent ingest seam (write_raw_and_parsed / the daemon ingest path) into session_events — no parallel writer (grep confirms reuse).\n- Behavior test: an agent posts a work event mid-session and it is queryable via observed-events (session_work_events / DSL) within one convergence cycle; re-posting the same event is idempotent (no duplicate row). `devtools test \u003cmcp work-event test\u003e` green.","notes":"[Delivery upgrade 2026-07-07T00:05:00Z] Release=D-agent-context-coordination; lane=agent-coordination; readiness=A-implementation-ready; proof=two-agent separate-worktree proof with before/after coordination envelopes. Original readiness=A-implementation-ready.\n[Prework packet 2026-07-07] Static execution packet (anchors, mechanism, plan, tests, verification): .agent/handoffs/polylogue-gpt-pro-2026-07-07/prework-v2/task_packets/071_polylogue_rii_1.md (depth: bead-localized-from-export; urgency: T2-foundation-before-feature-proof). Generated from master @ 8a975a40 2026-07-06 — verify source anchors before coding; line numbers are snapshot-relative.\nRECONCILED 2026-07-13 with 37t.2 inline protocol: the agent work-event write-leg and the marker channel are ONE channel with two encodings (structured MCP writes; prose markers extracted at enrichment). Unify vocabularies — work-event kinds and marker kinds must share the registry (a ::phase marker IS a work event). Do not build parallel event taxonomies.","status":"open","priority":2,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:43Z","created_by":"Sinity","updated_at":"2026-07-13T04:00:08Z","external_ref":"gh-2459","labels":["area:substrate","delivery:D-agent-context-coordination","horizon:frontier","lane:agent-coordination"],"dependencies":[{"issue_id":"polylogue-rii.1","depends_on_id":"polylogue-rii","type":"parent-child","created_at":"2026-07-03T06:31:43Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"polylogue-fs1.3","title":"Per-source coverage/fidelity declaration for Hermes imports","description":"Every Hermes acquisition tier and schema version needs a machine-readable fidelity declaration that distinguishes what is exact, absent, redacted, degraded, or inferred. The declaration is the guard against a parser test going green while silently dropping forensic history or cost/addressing provenance.","design":"Extend the OriginSpec/fidelity surface with: producer/schema version; installation/profile namespace; acquisition method (sqlite_backup, stable export, JSON fallback, runtime spans); exact retained-blob-to-normalized reproducibility verdict; counts and coverage for active, rewound, compacted, and observed messages; addressing/material-origin semantics; actual/estimated cost with status/source/pricing/billing provenance; lifecycle/relationship coverage; runtime-span coverage and explicit missingness. The snapshot and span lanes may enrich one logical session revision only with per-field provenance; they may not double-count or silently prefer a lower-fidelity tier.","acceptance_criteria":"explain-import on Hermes v16, a later schema, JSON fallback, and a spans-plus-snapshot merge names every capability as exact, absent, redacted, degraded, or inferred; exact-blob reproducibility is stated and verified; the same logical session from two tiers remains one revision with field-level provenance; message-state/addressing and cost-provenance counts reconcile to fixtures; deliberately dropping observed mapping, cost provenance, snapshot proof, or an unpaired span changes the declared fidelity and surfaces a downstream forensics caveat. OriginSpec fixtures and mutation-style negative tests pass.","notes":"[Delivery upgrade 2026-07-07T00:05:00Z] Release=K-interop-origin-export; lane=origin-interop-export; readiness=D-horizon-ready; proof=OriginSpec detector/parser/fixture/fidelity suite and content-hash export/import roundtrip. Original readiness=E-spec-needed.\n2026-07-12 fanout lane finding: blocked as scoped — explain-import cannot inspect SQLite Hermes state DBs and its payload lacks a fidelity-declaration field; both surfaces (import_explain.py + payload schema) must be in scope to implement. Evidence: 37bdfa04c; import_explain.py decodes JSON/JSONL only.","status":"closed","priority":2,"issue_type":"feature","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:40Z","created_by":"Sinity","updated_at":"2026-07-12T23:15:18Z","closed_at":"2026-07-12T23:15:18Z","close_reason":"PR #2789 merged: Hermes per-source coverage/fidelity declaration shipped (import_explain.py, hermes_state.py, generated CLI-output schema regenerated)","labels":["area:ingest","area:substrate","delivery:K-interop-origin-export","delivery:ac-patched","horizon:frontier","lane:origin-interop-export"],"dependencies":[{"issue_id":"polylogue-fs1.3","depends_on_id":"polylogue-fs1","type":"parent-child","created_at":"2026-07-03T06:31:40Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":3,"comment_count":0} {"_type":"issue","id":"polylogue-tf2.2","title":"Fold agent_forensics.py into polylogue analyze","description":"~70% already materialized (cost_rollups, archive_coverage, total_credit_cost, portfolio, cost_outlook). Real gaps: reasoning-token lane on SessionProfile; usage_timeline archive insight (tokens/cost per month per model) registered in insights/registry.py; optional markdown forensics renderer. Drop the script's hand-rolled _CREDIT_RATES; delete the script. Sequenced AFTER the campaign regen (the campaign uses the script one last time). GH issue thread (body + comments) is input, not authority; this bead's scope statement wins where they conflict.","status":"closed","priority":2,"issue_type":"feature","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-07-03T04:31:34Z","created_by":"Sinity","updated_at":"2026-07-03T11:54:39Z","started_at":"2026-07-03T11:31:18Z","closed_at":"2026-07-03T11:54:39Z","close_reason":"Completed: usage forensics is no longer a standalone script surface. Added registered usage_timeline archive insight with CLI/API/MCP registry coverage, reused the shared subscription-pricing catalog for credit estimates, deleted scripts/agent_forensics.py and its private-helper tests, and rewrote README/docs around polylogue analyze insights coverage/cost-rollups/usage-timeline plus devtools workspace claim-vs-evidence. Verification: focused claim-vs-evidence/insights tests passed, render all --check passed, devtools verify --quick passed, and live active-archive usage-timeline smoke returned valid JSON. Follow-up polylogue-5nn tracks the observed 18s whole-archive aggregation latency for unfiltered month-origin-model usage-timeline.","external_ref":"gh-2480","labels":["area:usage","campaign"],"dependencies":[{"issue_id":"polylogue-tf2.2","depends_on_id":"polylogue-tf2","type":"parent-child","created_at":"2026-07-03T06:31:34Z","created_by":"Sinity","metadata":"{}"},{"issue_id":"polylogue-tf2.2","depends_on_id":"polylogue-tf2.1","type":"blocks","created_at":"2026-07-03T06:31:34Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-1hal","title":"backlog-hygiene X2 check reports dangling bead refs for binaries, filenames and its own output","description":"Measured 2026-07-28: the X2 'names nonexistent bead' check reports 9 findings; classified by hand, 5 of the 6 distinct cases are false positives because the matcher does not exclude non-prose contexts.\n\n polylogue-3gd.3 -\u003e 'polylogue-mcp' is a BINARY NAME in /nix/store/.../bin/polylogue-mcp\n polylogue-yyvg.6 -\u003e 'polylogue-all' is a FILENAME, 00-polylogue-all.tar.gz\n polylogue-8jg9.1 -\u003e 'polylogue-all' is the check QUOTING ITS OWN OUTPUT about yyvg.6\n polylogue-yla8 -\u003e 'polylogue-a92969b6e4c8d728b' is an agent SESSION id\n polylogue-1xc.14(.1/.1.1/.1.2/.1.3) -\u003e 'polylogue-a47769bba68869d49' is an agent SESSION id (5 findings, one cause)\n polylogue-yyvg.7 -\u003e 'polylogue-x2q3s' is the ONLY genuine dangling bead reference\n\nA check whose findings are 5/6 noise trains readers to skip it, which is worse than not having it -- the one real dangling reference was invisible inside the noise.\n\nBead ids have a known shape (short base36 suffix, optional dotted child path). Session ids are long hex. Filenames and store paths are recognisable by their surrounding characters.","acceptance_criteria":"1. The matcher excludes tokens inside filesystem paths, filenames with extensions, and code/quoted-output spans. 2. It rejects candidates that do not match the bead-id shape (long hex is not a bead id). 3. Re-run reports the genuine dangling reference and not the five false positives. 4. A fixture covers each of the five false-positive shapes so they cannot regress.","status":"open","priority":3,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:05:32Z","created_by":"Sinity","updated_at":"2026-07-28T20:05:32Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-cuxz.5","title":"26,188 tool_use blocks have no paired tool_result and the actions view cannot say so","description":"Measured on the live archive 2026-07-28:\n\n tool_use blocks: 1,870,733\n tool_result blocks: 1,844,545\n difference: 26,188 (1.4%)\n action_pairs rows: 1,870,733\n\nactions is a VIEW left-joining tool_use to tool_result by tool_id, so an unpaired tool_use appears as an action with null result -- indistinguishable from a tool call whose result exists but carries no outcome signal (see polylogue-cuxz.4). Unpaired calls are real evidence (interrupted session, truncated transcript, provider-side drop, in-flight background task) and should be a typed, countable state.\n\nBound before fixing: classify the 26,188 by origin and by cause before deciding whether this is honest truncation evidence, a parser pairing defect, or in-flight background work.","acceptance_criteria":"1. The 26,188 are classified by origin and cause with counts, not treated as one bucket. 2. A reader can distinguish 'no result row exists' from 'result exists with unknown outcome'. 3. Any subset attributable to a pairing defect is fixed and re-measured; the remainder is a named, expected state.","status":"open","priority":3,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:02:17Z","created_by":"Sinity","updated_at":"2026-07-28T20:02:17Z","labels":["area:storage","horizon:frontier"],"dependencies":[{"issue_id":"polylogue-cuxz.5","depends_on_id":"polylogue-cuxz","type":"parent-child","created_at":"2026-07-28T22:02:16Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-5xng","title":"paste_spans materializes 4 rows across the entire archive: paste detection is effectively dead","description":"Measured on the live archive 2026-07-28:\n\n SELECT count(*) FROM paste_spans; -\u003e 4\n SELECT sum(paste_count), count(*) FROM sessions WHERE paste_count\u003e0; -\u003e 4 | 3\n\nFour paste spans across 18,871 sessions and 5,042,564 blocks, concentrated in 3 sessions. Operator pasting is routine in Claude Code and ChatGPT use, so this is not a true-negative result.\n\nTwo outcomes are acceptable and one is not: either the wire genuinely carries paste markers for some origin and the detector fails to read them (fix the detector), or no supported provider emits a paste signal (delete paste_spans, sessions.paste_count, and the runtime index idx_paste_spans_session rather than carrying a table, a column, an index and a materializer that produce nothing). What is not acceptable is leaving a schema surface that implies a measurable signal it never measures.\n\nNote polylogue-623q measured idx_paste_spans_session at an 800x speedup for its intended shape -- an index whose table holds 4 rows.","acceptance_criteria":"1. Determine per origin whether a paste/attachment-of-pasted-content marker exists on the wire, citing the provider record shape. 2. Either the detector reads it and a live re-measure shows a plausible count, or the table/column/index/materializer are removed in one change. 3. No half state: a retained paste_spans surface must have a named origin that populates it.","status":"open","priority":3,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T20:01:46Z","created_by":"Sinity","updated_at":"2026-07-28T20:01:46Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"polylogue-07pt","title":"Flaky timing assertion in browser-extension build.test.js backfill archive test","description":"tests/build.test.js \u003e build.mjs full archive emission \u003e executes the packaged service worker fixture without foreground tab activation fails with 'expected false to be true' on a vi.waitFor() timing assertion around line 274 (pageRequests.some(...) check after polylogue.backfill.start). Observed as a pre-existing failure across multiple uncapped and capped npm test runs during polylogue-0v5b (worker concurrency cap) work 2026-07-28, unrelated to that change (fails identically at 4, 8, and 24 workers). Needs investigation: likely a timing/race issue in the fake service-worker backfill fixture rather than the worker-cap change.","status":"open","priority":3,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T19:52:25Z","created_by":"Sinity","updated_at":"2026-07-28T19:52:25Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-sg80","title":"Semantic-frontier quarantine refinement: byte-proof actuator cannot resolve semantically-accepted raws","status":"open","priority":3,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-28T12:14:51Z","created_by":"Sinity","updated_at":"2026-07-28T12:14:51Z","dependencies":[{"issue_id":"polylogue-sg80","depends_on_id":"polylogue-zaiz","type":"discovered-from","created_at":"2026-07-28T14:15:14Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-e6a0","title":"index_v37_fast_forward test fixture predates action_pairs runtime index; v36 baseline shape unclear","description":"tests/unit/devtools/test_index_v37_fast_forward.py has 9 failing tests, all with the same root cause: its _archive() helper builds a synthetic \"v36-era\" index.db by git-show'ing INDEX_DDL from commit 5d99611f4^ and executing it, then calling ensure_runtime_indexes_sync(conn) to add the runtime-index extensions.\n\nensure_runtime_indexes_sync (polylogue/storage/sqlite/runtime_indexes.py) now includes an index on the action_pairs table (added by PR #2fb16467f / #3210, 2026-07-20), but action_pairs did not exist yet in the genuine v36-era DDL fetched from that hardcoded commit -- so the call raises sqlite3.OperationalError: no such table: main.action_pairs before the test's fast-forward-under-test even runs.\n\nI attempted the obvious fix (removing the premature ensure_runtime_indexes_sync call from the v36 fixture, since a real v36 archive predating action_pairs's table could never have picked up that index either) but this broke MORE tests: _prove_v36_delta (the fast-forward tool's own schema-shape verification) then reports the v36 fixture is missing a much larger list of tables/indexes/triggers (delegation_facts, work_evidence_edges/nodes/graphs, messages_fts_identity, query_unit_frame_state + its triggers, action_pairs + its indexes/triggers, etc.) -- meaning the fixture's intended \"before\" shape is NOT simply raw-v36-DDL-plus-runtime-indexes; it's supposed to already reflect every same-version 'benign DDL convergence' schema addition (see PR #3176's same-version benign-DDL convergence mechanism, apply_index_benign_ddl_convergence) that a real long-lived v36 archive would have accumulated over time without ever bumping its user_version. Reverted my attempted fix (git checkout -- the file) rather than ship an incomplete/wrong construction.\n\nProperly fixing this needs someone to determine the exact intended \"before\" schema shape for a genuine v36 archive as of the v36-\u003ev37 cutover point (possibly: advance _v36_ddl()'s hardcoded commit SHA to the actual last commit before the version bump, AND run apply_index_benign_ddl_convergence in the fixture too, not just ensure_runtime_indexes_sync), then verify test_prepare_and_activate_preserve_surviving_rows_without_raw_replay and friends pass end-to-end. This is index_v37_fast_forward's OWN test suite validating a one-time migration tool (probably fine to leave broken if the v36-\u003ev37 fast-forward has already been run against the one live archive it exists for -- but if the tool needs to run again or be trusted as a template for a future vNN-\u003evNN+1 fast-forward, this needs fixing first).\n\nConfirmed via git log that this is pre-existing (unrelated to any change in the current session): the runtime_indexes.py action_pairs addition landed 2026-07-20, a week before this triage.","notes":"Follow-up session (2026-07-28): tried the concrete next step proposed in the\noriginal description (advance _v36_ddl()'s commit SHA to the true last commit\nbefore the v36-\u003ev37 bump, and apply apply_index_benign_ddl_convergence in the\nfixture too). Result: does NOT converge, and the reason is bigger than a\nfixture-construction problem.\n\n1. The hardcoded commit `5d99611f4^` in _v36_ddl() is ALREADY the correct\n \"last commit before the version bump\" -- confirmed via\n `git log -p -S \"INDEX_SCHEMA_VERSION = 37\"`: commit 5d99611f4 itself is\n the one that bumps INDEX_SCHEMA_VERSION 36-\u003e37 (removes session_runs/\n session_observed_events/session_context_snapshots). So there was no SHA\n to advance; the prior session's \"advance the SHA\" framing was based on an\n incorrect premise.\n\n2. Instrumented `_prove_v36_delta` directly (built the v36 DDL from that\n commit, executed it in :memory:, called forward._schema_objects, and\n diffed against forward._canonical_schema_objects()) to see the *actual*\n full gap rather than the first exception. Live INDEX_SCHEMA_VERSION is\n now 43 (checked `polylogue/storage/sqlite/archive_tiers/index.py:35`).\n The diff shows 52 missing schema objects (table:action_pairs,\n table:delegation_facts, table:work_evidence_edges/nodes/graphs,\n table:messages_fts_identity... wait, table:query_unit_frame_state,\n table:delegation_refresh_scope, table:derived_refresh_guard,\n view:delegation_facts_source, plus their indexes/triggers) and 19 surplus\n objects (the 3 genuinely-retired v37 cache tables + their indexes, which\n is expected, PLUS table:model_prices/table:session_reported_costs).\n\n3. Root cause: `_canonical_schema_objects()` in\n devtools/index_v37_fast_forward.py computes canonical schema by executing\n the CURRENT `INDEX_DDL` import (live HEAD shape), not a schema frozen at\n v37. This was correct at the moment the tool was written (right after\n the v36-\u003ev37 bump, when HEAD DDL *was* v37 DDL), but every subsequent\n INDEX_SCHEMA_VERSION bump (v38 action_pairs via #3210, and 5 more bumps\n up to the current v43 -- delegation_facts, work_evidence_*,\n query_unit_frame_state + triggers, messages fts identity, paste_spans,\n etc.) silently drifted what \"canonical\" means out from under this frozen\n one-time migration tool. `apply_index_benign_ddl_convergence` only\n explains 2 of the 19 surplus entries (model_prices,\n session_reported_costs, both dropped by that same-version convergence\n registry) -- it has no bearing on the 52 missing objects, which are real\n cross-version schema additions, not same-version benign DDL.\n\nConclusion: this is not a fixable-in-place fixture bug. A real fix requires\neither (a) freezing a true point-in-time v37 canonical DDL snapshot (e.g.\nby diffing successive version-bump commits 5d99611f4..9163d0134 to\nreconstruct exactly what schema existed between the v37 bump and the v38\nbump) and teaching the test (or the production tool) to compare against\nthat frozen shape instead of live HEAD DDL, or (b) accepting that\ndevtools/index_v37_fast_forward.py is a completed one-time migration tool\nthat is now permanently non-re-runnable/non-testable as designed, and\nexplicitly retiring/skipping its test suite rather than trying to keep it\ngreen against a moving target. Did not attempt either since both are\ndesign decisions beyond \"try the concrete next step\" scope authorized for\nthis session. No code changes made; working tree left clean.","status":"open","priority":3,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-07-27T17:26:22Z","created_by":"Sinity","updated_at":"2026-07-28T14:44:39Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"polylogue-q7ol","title":"mypy: fix hypothesis timezones strategy arg-type in test_timestamp_guards.py","description":"devtools verify --quick / mypy --strict fails on tests/unit/core/test_timestamp_guards.py:424 with: Argument \"timezones\" to \"datetimes\" has incompatible type \"SearchStrategy[timezone | None]\"; expected \"SearchStrategy[None] | None\". Pre-existing on master, unrelated to any in-flight change; discovered while verifying polylogue-5en. Likely a hypothesis version/stub drift (pyproject pins hypothesis\u003e=6.161.5). Fix the strategy construction or type annotation so mypy --strict is clean again.","status":"closed","priority":3,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-07-27T16:47:02Z","created_by":"Sinity","updated_at":"2026-07-27T16:53:49Z","closed_at":"2026-07-27T16:53:49Z","close_reason":"Fixed in feature/chore/5en-dev-loop-verify-close (commit cb6383eda), discovered+fixed while unblocking the polylogue-5en push gate. Root cause: hypothesis.strategies.datetimes() has no @overload covering explicit min/max bounds + an optional (tzinfo|None) timezones strategy; runtime behavior is correct, only the overload set is incomplete. Added a scoped type: ignore[arg-type] with an explanatory comment on tests/unit/core/test_timestamp_guards.py:424 rather than reshaping the test. devtools verify --quick now exits 0.","dependency_count":0,"dependent_count":0,"comment_count":0} diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7f8b156abe..2f06167fe1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -31,6 +31,9 @@ jobs: - run: uv sync --extra dev --frozen - run: uv run devtools render all --check - run: uv run devtools verify public-claims --json + # An index schema bump merged without its lifecycle.py delta declaration + # downgrades every live generation to a full raw replay (polylogue-9rw0). + - run: uv run devtools lab policy schema-versioning - run: uv run ruff check polylogue/ tests/ devtools/ - run: uv run ruff format --check polylogue/ tests/ devtools/ diff --git a/devtools/verify.py b/devtools/verify.py index 07a1dce71a..139775ef70 100644 --- a/devtools/verify.py +++ b/devtools/verify.py @@ -1772,6 +1772,13 @@ def build_verify_steps( ("verify pytest-timeout-overrides", _devtools_cmd("verify pytest-timeout-overrides")), ("verify degrade-loudly", _devtools_cmd("verify degrade-loudly")), ("verify hash-boundary-census", _devtools_cmd("verify hash-boundary-census")), + # Static, archive-independent, sub-second: an index bump that + # lands without its lifecycle.py delta declaration silently + # downgrades every existing generation to a full raw replay + # (polylogue-9rw0). Gated here, not behind --lab, because the + # failure surfaces as an unqueryable live archive rather than + # as a test failure. + ("lab policy schema-versioning", _devtools_cmd("lab policy schema-versioning")), ] ) @@ -1850,7 +1857,6 @@ def _isolated_report_arg(arg: str) -> str: if lab: steps.append(("lab smoke", _devtools_cmd("lab smoke", "run", "archive-smoke", "--tier", "0"))) steps.append(("bench slo", _devtools_cmd("bench slo", "--include-lab"))) - steps.append(("lab policy schema-versioning", _devtools_cmd("lab policy schema-versioning"))) steps.append(("lab policy timestamp-doctrine", _devtools_cmd("lab policy timestamp-doctrine"))) steps.append(("lab policy insight-honesty", _devtools_cmd("lab policy insight-honesty"))) steps.append(("lab policy demo-packet-registry", _devtools_cmd("lab policy demo-packet-registry"))) diff --git a/polylogue/storage/sqlite/archive_tiers/index.py b/polylogue/storage/sqlite/archive_tiers/index.py index 322e550481..ff4d0ac2f9 100644 --- a/polylogue/storage/sqlite/archive_tiers/index.py +++ b/polylogue/storage/sqlite/archive_tiers/index.py @@ -28,10 +28,19 @@ ) from polylogue.storage.sqlite.delegation_facts import delegation_facts_insert_sql -# polylogue-1xc.12: v43 adds the messages_fts_identity rowid/block_id ledger -# and its trigger-body writes. index.db is a rebuildable derived tier (no -# migration chain) -- an archive still on v42 needs `polylogue ops reset -# --index && polylogued run`, not an in-place upgrade helper. +# polylogue-ih67: v44 adds sessions.title_ref/title_confidence. +# +# index.db is rebuildable derived state, but "rebuildable" is not the same as +# "always rebuilt": every bump above INDEX_FAST_FORWARD_COMPATIBILITY_FLOOR +# must declare its delta class in storage/sqlite/lifecycle.py, and a declared +# non-semantic delta upgrades an existing generation in place through +# index_fast_forward_plan()/apply_index_fast_forward() on connect. Only a +# SEMANTIC_REPARSE delta -- one whose result depends on parser semantics -- +# routes to `polylogue ops reset --index && polylogued run`. +# +# A bump without a declaration is a policy violation, not a free rebuild: +# `devtools lab policy schema-versioning` fails, and the archive silently +# falls back to full raw replay. See polylogue-9rw0 / polylogue-b5l. INDEX_SCHEMA_VERSION = 44 # polylogue-v6i3: shared WHEN-clause fragment gating the blocks_command_trigram diff --git a/polylogue/storage/sqlite/lifecycle.py b/polylogue/storage/sqlite/lifecycle.py index 26b3e5a5eb..25a4dd0ea1 100644 --- a/polylogue/storage/sqlite/lifecycle.py +++ b/polylogue/storage/sqlite/lifecycle.py @@ -342,6 +342,24 @@ class IndexDeltaDeclarationReport(TypedDict): ), ), ), + IndexDeltaDeclaration( + version=44, + # Adds sessions.title_ref/title_confidence (polylogue-ih67 AC#5). The + # DDL surface is two nullable columns on an 18,871-row table -- trivial + # to copy forward -- but their VALUES come from the v44 Codex title + # resolver (thread name -> authored history -> first HUMAN_AUTHORED + # message), so a shape-only fast-forward would leave every row NULL + # while a cold rebuild populates them. That divergence is exactly what + # DerivedDeltaClass cannot currently express: there is no class for + # "additive columns, clone-safe shape, values via targeted reprocess". + # SEMANTIC_REPARSE is therefore the truthful classification today, and + # it keeps the existing full-rebuild behaviour rather than silently + # promoting a generation whose new columns disagree with a cold + # rebuild. polylogue-9rw0.1 owns extending the vocabulary so this exact + # delta class becomes a bounded shape fast-forward plus a Codex-scoped + # reprocess instead of a full-corpus replay. + classes=(DerivedDeltaClass.SEMANTIC_REPARSE,), + ), )