Skip to content

Add hybrid retrieval and Qwen reranking - #10

Open
B0use wants to merge 3 commits into
mainfrom
boss_ranker
Open

B0use wants to merge 3 commits into
mainfrom
boss_ranker

Conversation

@B0use

@B0use B0use commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Store 1024-d chunk vectors with Qwen/Qwen3-Embedding-0.6B, and load chunk text into local Postgres (index/load_chunks.py, index/embed.py).
  • retrieve() fuses BM25 and exact cosine with reciprocal rank fusion, then rescores the top 20 with Qwen/Qwen3-Reranker-0.6B. The returned score is that reranker's probability of yes.
  • Unit tests for embed, fusion, and rerank run in the existing GitHub Actions workflow. The real model downloads stay marked slow and are skipped in CI, same as the other checkpoint tests.

Test plan

  • uv run pytest tests/test_retrieval.py tests/test_rerank.py tests/test_embed.py (23 passed, 2 slow tests deselected)
  • Confirm the tests workflow passes on this pull request
  • A corpus embed and a live retrieve() call are still separate from this PR; no rows have been embedded yet

Made with Cursor

B0use and others added 2 commits November 9, 2026 12:42
Rank with reciprocal rank fusion when chunk vectors exist, and record the model that produced them.

Co-authored-by: Cursor <cursoragent@cursor.com>
The top 20 fused hits now get a yes-probability from Qwen3-Reranker-0.6B, and the CI suite covers that stage without downloading the model.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants