fix(code-index): verify fresh waits cheaply and type delivered saves

[ScriptedAlchemy]

Two truth bugs in the code-index readiness surface, fixed in one lane.

wait_for { state: fresh } means verified against the source as of the request

Cause. #2314 made a fresh wait whose current reading already said current return reached without checking the worktree, because the check was expensive and waited behind the scheduler mutex: a stat walk plus a content hash of every candidate. So a save no hook reported stayed invisible until the 15 minute backstop, while the wait said fresh.

Change.

• fresh always sweeps the source before it answers. ready and graph_ready still return at once when the current reading satisfies them.
• The sweep reads only the freshness fence, never the scheduler mutex (request_fresh_now records the observed change through the hint/epoch authority itself).
• SourceSweepCacheV1 (on the fence) makes the sweep cheap without weakening it:
  • a per-file digest is reused only while the file's full stat key (dev, inode, mode, size, mtime, ctime) holds and that key was settled (ctime more than 2 s before the stat that recorded it). ctime can't be set back, so touch -d, cp --preserve and rsync -a rewrites are still re-read and caught;
  • the candidate roster is reused while every directory the Git walk descends into, each .gitignore, info/exclude, the global excludes file and .git/config keep their settled keys (git's untracked-cache model: adding, removing or renaming an entry moves the directory's ctime). Otherwise gix walks again (thread cap from perf(daemon): reuse worker threads and release mimalloc heaps #2295);
  • stats run on at most 4 scoped threads, off the indexing pool; a handful of changed files are hashed inline, a cold cache uses the pool with its CPU permits.
  • Non-Unix keys never settle (no change time in std), so Windows re-derives every digest as before (ponytail: note).
• One sweep: every content proof goes through the cache — the freshness ladder, the fix(code-index): sweep source after a publication's text projection #2331 post-projection sweep (request_fresh_now_background now delegates to the fence), and the retained/restore reconcile paths. WorktreeStatSweepV1::content_matches is deleted; the stat signature stays only as the durable restart witness's negative cache.
• The caller's budget still bounds the sweep: one that can't finish returns timed_out { last_state }, never reached.
• fix(code-index): sweep source after a publication's text projection #2331's hole stays closed: the post-projection sweep still runs whatever the proof says, only cheaper; its test passes.

Tests (code-index-runtime).

• fresh_wait_catches_an_unreported_save_and_returns_after_its_reindex: edit without a hint, wait_for_readiness(Fresh, 2 min) → Reached, generation changed, served texts ["pub fn source() -> u32 { 2 }", "pub fn source() -> u32 { 2 }\n"]. Fails on master (fix(cli): type readiness waits and projectless refusals end to end #2314 early return restored): assert_ne! sees the same generation generation.v1.defde92f.00000001.dfae… before and after.
• fresh_wait_verifies_the_source_while_a_pass_holds_the_scheduler: with the scheduler mutex held, a quiet 1 s wait → Reached; after an unreported edit a 500 ms wait → TimedOut { staleness_state: Refreshing, latest_generation_id: <old> }; after release → Reached with { 3 } served. Fails on master: an unreported save must not read as fresh: Reached. Fails with the early return removed but the sweep still taking the lock: quiet wait TimedOut { last: Some(… staleness_state: Some(Fresh) …) }.
• fresh_wait_on_a_current_index_reaches_inside_one_second: 1 s budget → Reached, elapsed < 1 s.
• source_sweep_rereads_only_files_whose_settled_stat_moved: after the stats settle, sweeps report (true, walked: true, candidates: 2, hashed: 2), then (true, walked: false, candidates: 2, hashed: 0); a same-length rewrite with its mtime restored → (false, walked: false, candidates: 2, hashed: 1).

A delivered save makes status stale until the new generation seals

Fixes #2330

Cause. notify_hook_event wrote the request, shut down its write half and returned Delivered without reading the reply. The daemon admits the edited paths into the code-index queue while handling that request, so a status read issued right after Delivered could run before the hint existed, and report the outdated generation current / fresh with hook_hint_count: 0. (Before #2314 a ready wait's sweep hid this; the early return exposed it.)

Change. Delivery keeps the connection open and reads the daemon's JSON-RPC reply (it answers once the event is admitted): result → Delivered, an error → Malformed, EOF → Unavailable, still inside the existing 750 ms HOOK_EVENT_NOTIFY_TIMEOUT. Production hook callers already ignore the outcome, so a slow daemon yields TimedOut rather than blocking a host.

Test. daemon_suite::dirty_worktree_symbol_reads_test (real daemon). Fail-before (fire-and-forget restored in the test binary, same daemon binary): code_index_journey.rs:548 left: Some("daad5f9e046416f8e1f60976c33d663239abb391") right: None, the #2330 literal. Pass-after: 1 passed.

Runtime proof (branch perf+production,hotpath CLI, isolated HOME, one daemon under systemd-run --scope -p MemoryMax=6G -p MemorySwapMax=1G, shallow clone of this repo: 6,555 files, 5,701 indexed)

The 6 GB cap parks the native graph decode on this corpus (#2123: needs 2819267149 resident bytes; 1980021145 are available), so the profile sets index.native_graph_activation.v1 = false at the project layer. fresh does not depend on the graph.

Sweep cost, Hotpath, same corpus and daemon session (load average 25–200 on a shared 96-core host):

	per call
before: stat_signature + content_verify (the old sweep, still run by the restart witness)	516 ms + 318 ms (another session: 887 ms + —)
after, cold (first sweep after restart; runs during restore, primes the cache)	588 ms – 1.07 s
after, warm source_sweep	13.7 – 18.9 ms (final binary, 22 calls)

$ tracedecay tool status --args '{"wait_for":{"state":"fresh","timeout_ms":1000}}'   # index current, ×10
**wait:** reached   wall=0.26–0.32 s (CLI start included)
$ echo 'pub fn unreported_save_probe_two() -> u32 { 8 }' >> crates/tracedecay-domain/src/lib.rs   # no hook
$ tracedecay tool status --args '{"wait_for":{"state":"fresh","timeout_ms":1000}}'
**code_index_freshness.status:** warming
**wait:** timed_out (warming)
$ tracedecay tool status --args '{"wait_for":{"state":"fresh","timeout_ms":110000}}'   # repeated
15:05:26 **wait:** timed_out (warming)
15:06:41 **wait:** reached
$ tracedecay tool tracedecay_search --query unreported_save_probe_two
freshness: fresh
- **unreported_save_probe_two** (function, exact_message), rank 1

The reindex takes minutes because a whole generation is rebuilt on a loaded host; the wait reports that honestly.

Verification

• cargo test -p tracedecay-code-index-runtime -p tracedecay-mcp --lib: 529 + 387 passed (full runs under load average 200–460 timed out 1–8 unrelated 2–5 s bounds; each passed alone and full runs passed at normal load)
• cargo test -p tracedecay --features test-helpers --test daemon_suite (perf CLI as TRACEDECAY_TEST_BIN): 55 passed
• cargo test -p tracedecay-cli --test core_cli_suite -- tool_status tool_daemon projectless_json hook: 30 passed
• cargo test -p tracedecay --features test-transport,test-helpers --lib -- core_hooks dispatch_tests hook_event: 50 passed
• cargo clippy -p tracedecay-code-index-runtime -p tracedecay --all-targets -- -D warnings, with and without tracedecay/test-transport,tracedecay/test-helpers: clean
• cargo fmt --all -- --check: clean
• cargo check --workspace --all-targets --target x86_64-pc-windows-gnu --features tracedecay/test-transport,tracedecay/test-helpers,tracedecay-cli/test-transport: exit 0

[changeset-bot]

⚠️ No Changeset found

Latest commit: 784aa6d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-27T15:26:23.430832Z	784aa6d	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 784aa6d782

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Propagate hook admission failures before returning Delivered

When routing or code-index admission fails, the daemon still returns an empty successful result: on_custom_request calls dispatch_notification, whose dispatch discards HostAdmissionOutcome, and then unconditionally returns {} (tracedecay-mcp/src/server/rmcp.rs:883-886; crates/tracedecay/src/mcp/server/requests.rs:344-350). Consequently this branch reports Delivered even when the edited paths never entered the scheduler, so an immediate status read can still expose the stale generation—the condition this acknowledgement is intended to prevent. The response needs to carry the actual admission outcome rather than treating every result as delivery.

AGENTS.md reference: AGENTS.md:L189-L191

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Follow symlink targets when validating ignore files

On Unix, if core.excludesFile is a symlink—a common dotfiles setup—editing its target does not change the symlink metadata returned here. Because roster_evidence records that configured excludes path through sample, CachedCandidateRosterV1::holds continues reusing the old candidate list after the ignore rules change. Newly included or excluded source files therefore remain invisible to fresh waits until unrelated Git metadata or a recorded directory changes; file evidence should follow the target or decline to cache symlinked ignore files.

Useful? React with 👍 / 👎.