semantic_search: warn when the index predates the current commit - #151
Merged
Conversation
The index is already incrementally rebuilt (sig_id is the reuse key,
not mtime -- unchanged functions carry their embedding over), so
rebuilding is cheap, but nothing ever told a caller a rebuild might be
needed: a stale index answered a query exactly as confidently as a
fresh one, with no way to tell the difference short of noticing wrong
results after the fact.
Config gains built_at_commit -- the git HEAD sha at build time, ""
when not built inside a git checkout or written before this field
existed (index_build.lex's git_head(), matching the existing
docs_json's proc.run pattern). semantic_search compares it against
the CURRENT head at query time and appends a one-line note naming
both commits when they differ; matching or either-side-empty stays
silent, since this is a heuristic ("HEAD moved") and not a guarantee
("a function actually changed") -- built_at_commit's own comment says
so, and reusable() deliberately still ignores it for cache-reuse
decisions, only base_url/model/dims matter there.
Verified with a driver isolating render()/staleness_note() from the
real embedding HTTP call (a hand-built index header, real HEAD via
git rev-parse): a mismatched commit produces the note naming both
shas, a matching one produces the exact same output as before this
change, byte for byte.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sig_idis the reuse key, not mtime — unchanged functions carry their embedding over), so rebuilding is cheap, but nothing ever told a caller a rebuild might be needed: a stale index answered a query exactly as confidently as a fresh one.Configgainsbuilt_at_commit— the git HEAD sha at build time,""when not built inside a git checkout or written before this field existed (index_build.lex's newgit_head(), matching the existingdocs_json'sproc.runpattern).semantic_searchcompares it against the CURRENT head at query time and appends a one-line note naming both commits when they differ; matching or either-side-empty stays silent — this is a heuristic ("HEAD moved"), not a guarantee ("a function actually changed"), andreusable()deliberately still ignores it for cache-reuse decisions (onlybase_url/model/dimsmatter there).Test plan
lex check/lex fmt --check/lex doc-sync --check— cleanlex test tests— 4/4 existing tests still passexamples {}blocks forstaleness_notecover match/mismatch/either-side-emptyrender()/staleness_note()from the real embedding HTTP call (hand-built index header, real HEAD viagit rev-parse): a mismatched commit produces the note naming both shas; a matching one produces byte-for-byte the same output as before this change🤖 Generated with Claude Code