Skip to content

feat: rank exact definition sites and release v0.7.0#64

Merged
aryamthecodebreaker merged 1 commit into
mainfrom
codex/definition-site-ranking
Jul 22, 2026
Merged

feat: rank exact definition sites and release v0.7.0#64
aryamthecodebreaker merged 1 commit into
mainfrom
codex/definition-site-ranking

Conversation

@aryamthecodebreaker

Copy link
Copy Markdown
Owner

What changed

  • add bounded definition-site evidence for distinctive task identifiers and exact code/literal fragments
  • preserve truncated literals from public issue excerpts and attach explainable ranking reasons
  • cap extraction at 24 identifiers and 8 literal fragments, and precompile identifier patterns once per ranking run
  • add focused ranking, extraction, noise, and bound regressions
  • refresh generated Action/workspace artifacts and the frozen external result
  • prepare coherent v0.7.0 package, MCP, Action, README, website, changelog, launch-kit, and growth-baseline metadata

Why

The frozen Zod #5944 case named cidrv6 and included the start of the faulty regex, but token overlap ranked JSON-schema consumers above the file that defined the pattern. The ranker had no punctuation-preserving literal evidence and no explicit definition-site signal.

The new signal is general and explainable: code-shaped identifiers can support their definition, and a distinctive exact fragment receives an additional boost only when it occurs at a nearby definition/assignment. Generic short names such as level are deliberately excluded from identifier evidence.

Measured impact

The unchanged six-repository benchmark moves from:

  • Top-1: 3/6 (50%) → 4/6 (67%)
  • Top-3: 5/6 (83%) → 6/6 (100%)
  • Top-5: 5/6 (83%) → 6/6 (100%)

Zod #5944 moves from outside Top-5 to Top-1. No task text, expected path, case, or selection rule changed. The checked-in per-case output remains public, and the docs explicitly avoid turning six cases into a general accuracy claim.

Validation

  • npm run ci
    • 122 tests
    • typecheck and ESLint
    • high/critical production audit gate (two known moderate findings remain in the MCP SDK's unused Hono static adapter)
    • CLI, Action, and workspace smokes
    • production Next.js build
    • Action metadata, MCP manifest, and generated bundle checks
    • internal evaluation gate
    • deterministic 1,000-file scanner check
  • node scripts/evaluate-external.mjs --gate --check-recorded
    • Top-1 0.667, Top-3 1.0, Top-5 1.0

Closes #59

@aryamthecodebreaker
aryamthecodebreaker marked this pull request as ready for review July 22, 2026 11:06
@aryamthecodebreaker
aryamthecodebreaker merged commit 04d9ddd into main Jul 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve definition-site ranking for exact identifiers (Zod #5944)

1 participant