fix(db): avoid "database is busy" during CJK FTS rebuild

[epruseal]

What

rebuildFTSForCjkNormalization() streamed source rows via a better-sqlite3
.iterate() cursor while later beginning a transaction on the same
connection to flush each batch. Holding that cursor open across the
BEGIN raises "database is busy" instead of completing the rebuild.

Why

The streaming cursor was introduced by #737 specifically to avoid loading
every document body into memory at once (the original OOM bug). That fix
was correct for the memory problem but introduced this one: two concurrent
statements (an open iterator + a transaction) on the same connection.

How

Replaced the .iterate() cursor with bounded, keyset-paginated batches
(WHERE id > ? ORDER BY id LIMIT ?) — each SELECT fully finalizes before
its batch's insert transaction starts, so the connection is never busy with
two concurrent statements. Batches stay bounded (500 rows at a time, same
as before), so this doesn't reintroduce the OOM bug #737 fixed — confirmed
by that PR's own large-library migration test, still passing.

Updated the structural regression test to assert the actual protected
invariant (bounded LIMIT + keyset pagination, never one unbounded .all()
over every active document) instead of a specific API shape
(.iterate(), not .all()), since that literal check no longer applies now
that bounded .all() batches are the mechanism.

Testing

• tsc -p tsconfig.build.json --noEmit clean
• test/store-cjk-fts.test.ts (6 tests) and test/store-concurrency.test.ts
  (2 tests) all pass

Split out of #761 per review feedback there, so this can be reviewed and
reverted independently of the embedding-provider change.

[PowderAddicts]

Dear tobi/qmd, We would like to acknowledge that we have received your request and a ticket has been created. A support representative will be reviewing your request and will send you a personal response.(usually within 24-48 hours). Thank you for your patience.

…

On Mon, Jul 27 2026, at 04:44 AM, tobi/qmd ***@***.***> wrote: What rebuildFTSForCjkNormalization() streamed source rows via a better-sqlite3 .iterate() cursor while later beginning a transaction on the same connection to flush each batch. Holding that cursor open across the BEGIN raises "database is busy" instead of completing the rebuild. Why The streaming cursor was introduced by #737 specifically to avoid loading (#737) every document body into memory at once (the original OOM bug). That fix was correct for the memory problem but introduced this one: two concurrent statements (an open iterator + a transaction) on the same connection. How Replaced the .iterate() cursor with bounded, keyset-paginated batches ( WHERE id > ? ORDER BY id LIMIT ?) — each SELECT fully finalizes before its batch's insert transaction starts, so the connection is never busy with two concurrent statements. Batches stay bounded (500 rows at a time, same as before), so this doesn't reintroduce the OOM bug #737 fixed — confirmed (#737) by that PR's own large-library migration test, still passing. Updated the structural regression test to assert the actual protected invariant (bounded LIMIT + keyset pagination, never one unbounded .all() over every active document) instead of a specific API shape ( .iterate(), not .all()), since that literal check no longer applies now that bounded .all() batches are the mechanism. Testing tsc -p tsconfig.build.json --noEmit clean test/store-cjk-fts.test.ts (6 tests) and test/store-concurrency.test.ts (2 tests) all pass Split out of #761 per review feedback there, so this can be reviewed and (#761) reverted independently of the embedding-provider change. You can view, comment on, or merge this pull request online at: #797 (#797) Commit Summary c538c8a fix(db): avoid "database is busy" during CJK FTS rebuild (c538c8a) File Changes ( 2 files) (https://github.com/tobi/qmd/pull/797/files) M src/store.ts (25) (https://github.com/tobi/qmd/pull/797/files#diff-2717c7fb31c2070dd96f07394997809bc9be7f3fff03ab638db696cc480f39b6) M test/store-cjk-fts.test.ts (55) (https://github.com/tobi/qmd/pull/797/files#diff-2a8b73c2ea238681e82b26a462634b8abe24cd91bc77fd34936ee6e7ace3c897) Patch Links: https://github.com/tobi/qmd/pull/797.patch (https://github.com/tobi/qmd/pull/797.patch) https://github.com/tobi/qmd/pull/797.diff (https://github.com/tobi/qmd/pull/797.diff) — Reply to this email directly, view it on GitHub, or (#797?email_source=notifications&email_token=BRF3HGZZA33WCJIIKF6KW6L5G3M2TA5CNFSNUABEM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UF42DCMZZGMZTKMBWGOTHEZLBONXW5KTTOVRHGY3SNFRGKZFFMV3GK3TUVRTG633UMVZF6Y3MNFRWW) unsubscribe. (https://github.com/notifications/unsubscribe-auth/BRF3HG4ZWZ5DJ4T76C62XCT5G3M2TAVCNFSNUABGKJSXA33TNF2G64TZHMYTCMJSGM3DKMZQGE5US43TOVSTWNBZHA2DGOJYG42TNILWAI) Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and (https://github.com/notifications/mobile/ios/BRF3HG5TPFI27CB62G2QEB35G3M2TA5CNFSNUABEM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UF42DCMZZGMZTKMBWGOTHEZLBONXW5KTTOVRHGY3SNFRGKZFFMV3GK3TUVJTG633UMVZF62LPOM) Android. Download it today! (https://github.com/notifications/mobile/android/BRF3HG3BTRNDYZHRAX4UIXL5G3M2TA5CNFSNUABEM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UF42DCMZZGMZTKMBWGOTHEZLBONXW5KTTOVRHGY3SNFRGKZFFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE) You are receiving this because you are subscribed to this thread. Message ID: <tobi/qmd/pull/797 @ github . com> [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#797?email_source=notifications\u0026email_token=BRF3HG6IJXLHUFZEW6CEYB35G3M2TA5CNFSNUABEM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UF42DCMZZGMZTKMBWGOTHEZLBONXW5KTTOVRHGY3SNFRGKZFFMV3GK3TUVNTW2YLJNRPWG3DJMNVQ", "url": "#797?email_source=notifications\u0026email_token=BRF3HG6IJXLHUFZEW6CEYB35G3M2TA5CNFSNUABEM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UF42DCMZZGMZTKMBWGOTHEZLBONXW5KTTOVRHGY3SNFRGKZFFMV3GK3TUVNTW2YLJNRPWG3DJMNVQ", "name": "View Pull Request" }, "description": "View this Pull Request on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]