fix(skill): rank recalled skills by relevance rather than by slug

[ion-alpha-dev]

What

Skill recall stops selecting alphabetically. A text search in either store is now ordered by how well the row matches (bm25 in sqlite with the description weighted highest, a field-weighted count in the in-memory store), recall gathers a candidate pool wider than the offer it can make, and Rank weights each term by how few of the candidates carry it, deciding relevance before evidence breaks a tie.

Two descriptions in the bundled pack also gained a word their own objectives are phrased with: deletion never said "clean up" or "leftovers", and domain-language never said "call".

Why

Which skills an objective was offered depended on the alphabet. A store answered a term with its matches ordered by slug and cut to the limit, and Gather asked for exactly the five skills recall can offer, so for any word six or more skills carried, everything sorted later was never a candidate and nothing reported it. Rank then scored on a raw count of matching terms, so a word most of the library carries counted the same as one only this skill has.

The result is a library that gets worse as it grows: adding a skill displaces an unrelated one, and the skill that loses is not the one being added. Assembled with every authored skill present, the pack failed five rows of its own retrieval table, deletion losing "clean up the leftovers from the last change" and domain-language losing "what should I call this thing" among them. The pack is at nine skills, the dev pack alone is planned for twenty-four, and a user's library is larger still.

The two description edits are the other half of the same finding. Both of those rows had been passing because nothing else matched either, not because those skills matched: the words a person types were never in the text the ranker reads.

How to verify

go test ./skill/... ./storage/sqlite/ is green.

Four tests carry the change, and each fails on the old behaviour:

• TestRecallOffersTheBestMatchWhateverItsSlugSortsAs: seven decoys mention the word in passing and the skill about it sorts last. Before, it was cut before ranking and the offer was the five decoys.
• TestRankWeighsARareTermAboveACommonOne: the candidate carrying the one discriminating term is ranked first, ahead of four that share a common one.
• TestSearchCapsToTheBestMatches and TestSkillSearchRanksTheDescriptionAboveThePassingMention: one per store, a capped search answers with the skill whose description is about the query rather than the one whose slug sorts first.

TestPackIsRetrievable passes with all thirteen authored skills present, including the four still in worktrees, with no further description tuning. TestRankBreaksTiesOnEvidenceAndNeverOverRelevance still holds.

Notes for reviewers

The document frequency behind the term weights is counted over the gathered candidates rather than the whole library. Those are the skills being chosen between and it costs no second pass over the store, but it does mean a weight is relative to the pool, not absolute.

candidatePool is 4, so recall issues the same number of queries as before, each asking for twenty rows instead of five.

Three tests in cmd/flynn fail on this machine (TestNewExternalAgent, TestExternalAdapterIsBuiltForEveryBackend, TestAttestedDeclarationMatchesRecordedEvents). They fail the same way on a clean main: a local codex install with no vendored binary for this platform.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!