Skip to content

fix(categorization): escape the NUL delimiter instead of embedding the byte - #119

Merged
KenTaniguchi-R merged 1 commit into
mainfrom
fix/engine-nul-byte-literal
Aug 30, 2026
Merged

fix(categorization): escape the NUL delimiter instead of embedding the byte#119
KenTaniguchi-R merged 1 commit into
mainfrom
fix/engine-nul-byte-literal

Conversation

@KenTaniguchi-R

Copy link
Copy Markdown
Owner

The byte

engine.ts carried a literal 0x00 byte in its source, inside the template literal that builds the assignment-grouping key:

const key = `${assignment.categoryId}<0x00 byte>${assignment.source}`;

NUL is a sound delimiter here — it cannot occur in a category id or a source string, so the composite key cannot collide. The problem is writing it as a raw byte rather than the \0 escape: grep classifies the whole file as binary and silently drops matching lines. No error, no warning, just absent results.

$ grep -rn "categoryRules" src/ --include=*.ts
src/db/schema/categories.ts:43:export const categoryRules = pgTable(

One hit — reads as "nothing consumes this table."

$ grep -ran "categoryRules" src/ --include=*.ts     # -a: treat binary as text
src/lib/categorization/engine.ts:3:import { ... categoryRules ... }
src/lib/categorization/engine.ts:106:      id: categoryRules.id,
...
src/lib/categorization/engine.ts:112:    .from(categoryRules)
src/db/schema/categories.ts:43:export const categoryRules = pgTable(

engine.ts queries the table on lines 106–113 and runs the rules on every sync. The file was invisible to the search that would have shown that.

This is a live trap for anyone investigating this codebase — it produced a confidently wrong conclusion about the categorization pipeline while triaging #89.

The fix

\0 instead of the raw byte. Verified byte-identical at runtime:

escape === raw byte: true
codepoint at delimiter: 0
length: 10  expected: 10

file now reports ASCII text instead of data, so grep, git diff, and editors all read it normally.

Test

Adds the regression test that nothing covered: the grouping itself. Two transactions reaching two different categories through two different tiers (rule and merchant_default) in a single call must each land on the right category and the right source.

Verified the test has teeth rather than assuming it: collapsing the key to a constant (const key = "COLLIDE") fails this test — and only this test, out of the six in that file. The existing five pass either way, which is why this gap survived.

Full suite green: 753 passed / 113 files. Typecheck clean, npx eslint src tests clean.

Note: pnpm lint on this branch reports noise from .claude/worktrees — that is #117, not this change.

🤖 Generated with Claude Code

…e byte

engine.ts carried a literal 0x00 byte in its source, inside the template
literal that builds the assignment-grouping key:

    const key = `${assignment.categoryId}<0x00 byte>${assignment.source}`;

NUL is a sound delimiter here -- it cannot occur in a category id or a
source, so the composite key cannot collide. Writing it as a raw byte
rather than the \0 escape is the problem: grep classifies the whole file
as binary and silently drops matching lines from its output. No error,
no warning, just absent results.

That is not hypothetical. Searching for `categoryRules` across src/
returned only the schema definition, which reads as "nothing consumes
this table" -- while engine.ts in fact queries it on lines 106-113 and
runs it on every sync. The file was invisible to the search that would
have shown otherwise.

`\0` is byte-identical at runtime (verified: same string, same code
point at the delimiter, same length) and the file is now ASCII text, so
grep, git diff and editors all read it normally.

Adds a regression test for the grouping itself, which nothing covered:
two transactions reaching two different categories through two different
tiers in one call must each land on the right category and source.
Verified it has teeth -- collapsing the key to a constant fails this
test and only this test, out of the six in that file.
@KenTaniguchi-R

Copy link
Copy Markdown
Owner Author

Mutation gate: baseline comparison

killed timeout survived no-cov errors score
main's engine.ts 23 0 1 25 53 46.94%
this PR 20 3 1 25 53 46.94%

Identical score. The killed/timeout split differs (23/0 vs 20/3), but Stryker scores a timeout as killed, so both are 23 — the difference is runner load, not code.

Score-neutral is the expected result: the change is \0 for a raw NUL byte, verified byte-identical at runtime, so the mutants are the same ones; and the added test can only kill more, never fewer.

The file is structurally capped regardless — 25 no-coverage + 53 errors out of 102 mutants, because categorizeSyncedTransactions is DB-backed and vitest.stryker.config.ts excludes tests/integration/. That is #103.

Note the earlier typecheck · lint · test red on this PR was not code either — the self-hosted runner's pnpm store failed to reflink:

[ERR_PNPM_ENOENT] the source path is not an existing regular file, reflink
'/Users/taniguchiryusei/setup-pnpm/node_modules/.bin/store/v11/files/29/844c...'

Re-ran, and it passed.

Merging on the same basis as #102, #104 and #105.

@KenTaniguchi-R
KenTaniguchi-R merged commit 4e47e6e into main Aug 30, 2026
5 of 7 checks passed
@KenTaniguchi-R
KenTaniguchi-R deleted the fix/engine-nul-byte-literal branch August 30, 2026 01:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant