Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions performance/store/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,5 @@ Measurements of the SQLite store that tg-cli, max-cli, memo and drive share thro
the relationship key's index condition makes it 0.2 s (2026-11-09).
- [project-activity](project-activity/README.md) — every project's linked chats, messages in a window and busiest
chats in one read, at the owner's scale: 2 ms for 30 days, 31 ms for all time (2026-11-10).
- [vector-purge](vector-purge/README.md) — dropping a rebuild's old vectors: the purge's two lookups by their index,
a forum rebuild from 889 s to 52 s (2026-11-18).
69 changes: 69 additions & 0 deletions performance/store/vector-purge/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Dropping a rebuild's old vectors

## Why it was run

cli-core's linking-rules version went from 5 to 6 ([WireCatLabs/cli-core#3107](https://github.com/WireCatLabs/cli-core/pull/3107)):
a conversation no longer mixes the topics of a forum group. With a new version, every chat a user built before is
rebuilt. On a test store, rebuilding one large forum group took about an hour, and most of that time came after the new
build was already in place, while `purgeVectorHashes` checked each old chunk text: does a current chunk still have it,
and if not, drop its vectors.

The question: is that time in the two lookups the purge makes per text, and does using the right index remove it? It
decided [WireCatLabs/cli-core#3118](https://github.com/WireCatLabs/cli-core/pull/3118).

## Setup

- Machine: AMD Ryzen AI 9 HX 470, 30 GB RAM, NVMe disk, Linux 7.0, Node 24.19.0.
- Data: a copy of a real store (private; only totals here): 220,180 conversation chunks, 306,068 vectors of two models,
one forum group of about 112,000 messages with 42,639 chunks after its rebuild.
- Code: cli-core with #3107's change, built into tg-cli. "Before" is the purge as it was; "after" adds #3118's change:
- the candidates query joins on `+k.chunkable_type`, so SQLite uses `chunks_by_content_hash` and not the
`(chunkable_type, chunkable_id, position)` index;
- the vector delete names the model too, so it uses the `(model, content_hash)` key.
- Whole rebuild: the forum group rebuilt with an unchanged result, so the purge checks every one of its old chunk texts
(about 42,600) and finds each still current. Timed with `/usr/bin/time`, load average 4 to 8:

```sh
tg personal conversations links rebuild --chat <forum> --offline --timeout 120m --json
```

- The two lookups alone: [`bench.mjs`](bench.mjs) opens the store read-only and times both forms of each lookup over
200 chunk texts, with their query plans. The delete is timed as the `SELECT` with the same `WHERE`, so the store is not
changed.

```sh
node bench.mjs <wirecat.db> 200
```

## Results

Whole rebuild of the forum group:

| code | time |
|---|---|
| before | 889 s, and 889 s on a second run |
| after | **52 s** |

The two lookups, per old chunk text:

| lookup | before | after |
|---|---|---|
| candidates (a current chunk with this text) | 19.7 ms, reads every conversation chunk | 0.13 ms, by `chunks_by_content_hash` |
| vectors of this text | 13.0 ms, reads the whole key | 0.01 ms, by `(model, content_hash)` |

## What it showed

- The time was in the lookups, not in writing the new build: at about 20 ms per text and 42,600 texts, the candidates
query alone is about 14 minutes, close to the whole difference (889 s against 52 s).
- SQLite chose the type index because the join condition named `chunkable_type`. The unary `+` keeps the condition and
takes the column out of index choice.
- The vector lookup by hash alone matters when a rebuild changes many chunks: in the first rebuild of this forum group
(version 5 to 6, about 28,000 changed texts) each of those also paid about 13 ms.

## Not measured

- A rebuild that changes most chunks with the new code: the timed rebuilds left every text current, so the vector
delete itself did not run; its lookup was timed with `bench.mjs` instead.
- Stores of other sizes. Before the change, the cost per text grew with every conversation chunk in the store; after,
it should not.
- Bun.
51 changes: 51 additions & 0 deletions performance/store/vector-purge/bench.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Usage: node bench.mjs <wirecat.db> [texts]
// Opens a store read-only and times the two lookups purgeVectorHashes makes for each old chunk text, as cli-core
// wrote them before and after WireCatLabs/cli-core#3118. Prints the query plans and totals only.
import { DatabaseSync } from "node:sqlite"

const [path, textsArg = "200"] = process.argv.slice(2)
const texts = Number(textsArg)
const db = new DatabaseSync(path, { readOnly: true })

const candidates = (plus) => `SELECT k.chunkable_id, r.first_message_id, r.last_message_id, r.text_start, r.text_end
FROM chunks k JOIN chunk_messages r ON r.chunk_id = k.id
JOIN conversations c ON ${plus}k.chunkable_type = 'conversation' AND c.id = k.chunkable_id
JOIN conversation_state s ON s.chat_id = c.chat_id AND s.current_build = c.build
WHERE k.content_hash = ? ORDER BY k.chunkable_id, k.position LIMIT 100 OFFSET 0`
const byHash = "SELECT count(*) FROM embeddings WHERE content_hash = ?"
const byKey = "SELECT count(*) FROM embeddings WHERE model = ? AND content_hash = ?"

const count = (sql) => Number(Object.values(db.prepare(sql).get())[0])
const hashes = db
.prepare("SELECT content_hash AS hash FROM chunks WHERE chunkable_type = 'conversation' ORDER BY id LIMIT ?")
.all(texts)
.map(({ hash }) => hash)
const models = db
.prepare("SELECT DISTINCT model FROM embeddings")
.all()
.map(({ model }) => model)
const say = (line) => process.stdout.write(`${line}\n`)
say(
`conversation chunks ${count("SELECT count(*) FROM chunks WHERE chunkable_type = 'conversation'")}, ` +
`vectors ${count("SELECT count(*) FROM embeddings")}, models ${models.length}, texts timed ${hashes.length}`,
)

const plan = (sql, ...args) =>
db
.prepare(`EXPLAIN QUERY PLAN ${sql}`)
.all(...args)
.map(({ detail }) => detail)[0]
const time = (label, sql, run) => {
const statement = db.prepare(sql)
const started = performance.now()
for (const hash of hashes) run(statement, hash)
const each = (performance.now() - started) / hashes.length
say(`${label}: ${each.toFixed(2)} ms per text — ${plan(sql, ...(sql === byKey ? ["m", "h"] : ["h"]))}`)
}

time("candidates before", candidates(""), (statement, hash) => statement.all(hash))
time("candidates after", candidates("+"), (statement, hash) => statement.all(hash))
time("vector lookup before", byHash, (statement, hash) => statement.get(hash))
time("vector lookup after", byKey, (statement, hash) => {
for (const model of models) statement.get(model, hash)
})
Loading