Repository navigation
feat(admission): enforce record exclusions across indexing and recall - #180
HengYangDS wants to merge 9 commits into
Conversation
|
The upstream CI run for this fork PR is currently |
|
Current author follow-up on #176-#180 and #184: the PRs remain open/mergeable, but their exact-head fork CI is still action_required, not green. Please approve the pending workflows when possible so actual failures can be addressed. I am also reviewing source-versus-generated scope and declared-toolchain compatibility to keep each contribution generic and independently mergeable; local test counts do not replace hosted acceptance. This is not a request to bypass branch protection. |
|
Follow-up on the new revision a8eadb5: The source/product-boundary review found one concrete isolated-summary shutdown gap: after a successful result, protocol error or timeout, a Codex child that ignored SIGTERM could leave the parent waiting indefinitely for close. The minimal repair adds an isolated-only 1024 ms TERM-to-KILL grace, cancels that timer on close, and keeps temporary-directory cleanup close-bound. It does not change the normal summary route, session-ID forks, provider/model selection, or record-admission defaults. Three actual TERM-resistant child regressions failed on the submitted predecessor and pass on this revision. Native Node 24.19.0 acceptance on macOS arm64 includes build/typecheck, 41 focused tests, eight generated-dist routing tests, and 451/451 final tests across 67 files with zero skips. The unrelated MCP bundle is preserved byte-for-byte. This is local evidence, not a hosted Node 22/24, Windows, real-provider, or descendant-held-stdio claim. Exact revised-head Actions run 38096647228 is still action_required and has not executed. The existing approval request remains sufficient; this comment supplies the new defect, fix and verification facts for review rather than another unchanged approval ping. I will address actionable review/CI findings and maintain the contribution through merge and relevant stable-release adoption. |
A retained search row whose transcript has since been removed made the whole request fail with "Conversation changed during record admission". Codex replaces archived rollouts with compressed .jsonl.zst files, so a transcript indexed before compression disappears and every query that matched one of its rows failed. A removed transcript has nothing left to verify or expose, so search now withholds its rows and continues. Symlink substitution, indexed-range rewrites and other admission failures still fail closed. Generated with Claude Code
|
Follow-up on a24fde1: Search admission failed the whole request with A removed transcript has nothing left to verify or expose, so search now withholds those rows and continues. Symlink substitution, indexed-range rewrites and other admission failures still fail closed. Three regression cases cover the text, vector and both modes; they fail before the change and pass after it. The full local suite passes: 67 files, 454 tests. As in the earlier fix commits, only |
|
Consolidating the current CI request here for the independent PRs, to keep it in one place:
These six runs still require repository approval. #184 is different: run 36964618130 ended with Could you approve the waiting runs and indicate which fix you want reviewed first? If any proposal is outside the project's current scope, a scope decision would also let me finish its disposition. Local results in the descriptions remain separate from hosted acceptance. |
Opt-in record screening must keep rejected records out of archives, indexes, embeddings, summaries and recall, including previously retained records. Safe neighboring records remain usable, and source transcripts are preserved.
This defines record/tool-call admission once and reuses it at ingress and recall. An optional absolute, SHA-256-pinned Gitleaks executable runs with per-record bounds and validated reports. Scanner configuration and secret-bearing diagnostics are not inherited. Rejected or unavailable screening is distinct from corruption, and repair refuses while screening is unresolved. Users who do not opt in need no scanner.
Recall rechecks retained rows. A disappeared source withholds its rows; changed indexed ranges or substituted sources remain refusals. Search scans the complete ordered candidates so excluded matches cannot exhaust the vec0 KNN candidate ceiling. Large-index cost is unbenchmarked.
Screened or sessionless Codex text uses a fresh ephemeral Codex thread. Source-session forks remain separate, and failures never switch to Claude. The isolated child's shutdown has a 1024 ms TERM-to-KILL grace; its directory is retired only after close. Routing tests use a synthetic app server, so real-provider acceptance remains open.
Integration boundaries
These are integration conditions, not a claim that the branches have been combined. Maintainers still need to choose the screening/redaction defaults and shared pipeline. I favor retaining the narrow independent fixes first, then rebasing this broader contribution against the accepted owners.
Validation
Head:
a24fde13d15be82debc64a29dc030bfcb778de75; base:7e06519357777badd7a115d2014a7ef845904310.Earlier macOS arm64 Node 24.19.0/npm 12.2.0 qualification passed native installation/build/typecheck, 98 focused tests and 451 tests across 67 files. The follow-up on a24fde13 adds three base-red regressions for missing transcripts in text/vector/both search modes; that local run passes all 454 tests across 67 files. It regenerates only
dist/record-admission.js; the earlier complete-bundle qualification retains its recorded revision. Focused checks include packaged STDIO MCP admission/search and TERM-resistant child shutdown. The native build reproduces the complete reachable MCP bundle without unrelated dependency drift or vendor edits. The preseeded public model produced finite 384-dimensional embeddings; cold downloads are not established.Existing dev-tool advisories remain under the recorded trusted-isolated-test limits. Node 22/26, hosted Linux/Windows, coverage, large-index performance, real providers, merge, release and installed adoption remain unverified. No background sync or summary backfill is enabled. Earlier qualification and shutdown evidence are in the retained commit history and follow-up; the previous approval requests remain sufficient.