feat: relevance ranking, evals, chronicle, embeddings, skill drift-check#1760
Open
BarrettHolien wants to merge 1 commit into
Open
feat: relevance ranking, evals, chronicle, embeddings, skill drift-check#1760BarrettHolien wants to merge 1 commit into
BarrettHolien wants to merge 1 commit into
Conversation
Five features grafting the strongest ideas from tobi/qmd (hybrid-search ranking) and garrytan/gbrain (evals, versioned skills, history) onto graphify's structural spine. Each is a module + CLI + tests. graphify's edge: it fuses structural signals (graph proximity, hub centrality, community) that pure-retrieval tools can't compute. 1. Ranking (graphify/ranking.py): rank query results by Reciprocal Rank Fusion over lexical + structural + optional semantic backends, replacing the degree-only render order. New query flags --top-k, --explain, --semantic; also on the MCP query_graph tool. Shared core: serve._resolve_and_rank / rank_query_nodes. 2. Evals (graphify/evals.py): `graphify bench` scores P@k/recall/MRR/nDCG against a JSONL fixture via the real ranking pipeline. --init scaffolds, --save/ --replay gate regressions (exit 1). 3. Embeddings (graphify/embed.py): opt-in `graphify embed` builds a local sidecar (Ollama nomic-embed-text with search_query:/search_document: prefixes, or sentence-transformers; no API). --semantic fuses cosine similarity and seeds traversal when lexical finds nothing. 4. Chronicle (graphify/chronicle.py): `graphify chronicle OLD NEW` (or --rev via `git show`) diffs two graph snapshots: nodes/edges, god-nodes emerged/vanished, community shifts. API-free. 5. Skill drift (graphify/skill_migrations.py): `graphify skill status|check-update` reports installed-skill-vs-package drift (exit 1 for CI) with a migration registry. Skill fragments document the new query flags + a new analysis.md reference; regenerated + blessed all artifacts (skillgen --check/--audit-coverage pass). Version 0.9.12 -> 0.10.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Five features grafting the strongest ideas from tobi/qmd (hybrid-search ranking) and garrytan/gbrain (evals, versioned skills, history) onto graphify's structural spine. Each is a module + CLI + tests. graphify's edge: it fuses structural signals (graph proximity, hub centrality, community) that pure-retrieval tools can't compute.
Features
graphify/ranking.py) — rank query results by Reciprocal Rank Fusion over lexical + structural (proximity/centrality/community) + optional semantic backends, replacing the degree-only render order. Newqueryflags--top-k,--explain,--semantic; also on the MCPquery_graphtool.graphify/evals.py) —graphify benchscores P@k / recall / MRR / nDCG against a JSONL fixture using the real ranking pipeline.--initscaffolds;--save/--replaygate regressions (exit 1).graphify/embed.py) — opt-ingraphify embedbuilds a local sidecar (Ollamanomic-embed-textwith the model'ssearch_query:/search_document:prefixes, or sentence-transformers; no API).--semanticfuses cosine similarity and seeds traversal when lexical finds nothing.graphify/chronicle.py) —graphify chronicle OLD NEW(or--revviagit show) diffs two graph snapshots: nodes/edges, god-nodes emerged/vanished, community shifts. API-free.graphify/skill_migrations.py) —graphify skill status|check-updatereports installed-skill-vs-package drift (exit 1 for CI) with a migration registry.Rebased onto current
v8. Version 0.9.12 → 0.10.0 with CHANGELOG entry. Skill fragments regenerated + blessed (skillgen --check/--audit-coveragepass). New tests across all five features; full suite green apart from pre-existing ollama/gemini env-dependent failures unrelated to this change (they fail identically on unmodifiedv8).