perf(local): quote SQL literals with a bash expansion, not a fork - #948
Merged
Conversation
Picks up the work in #897. The local send/inbox/history/watch path forked `printf | sed` to double single quotes at eight sites; these are bash parameter expansions now, and the fork is gone from that path. The idea and the eight sites are from #897. What changed is the shape. Seven of the eight were written as `${v//\'/\'\'}`, with the quote inline in the pattern. bash 3.2 -- which is `/bin/bash` on macOS, and what the macOS CI jobs run -- keeps the backslash of a `\'` REPLACEMENT, so that form doubles a quote into `\'\'` there while producing `''` on bash 4+. Measured on both: bash 3.2.57 backslash form -> a\'\'b q-variable form -> a''b bash 5.3.15 backslash form -> a''b q-variable form -> a''b That difference is invisible on Linux, which is why it reads as correct. It is also why the macOS jobs went red on #897. So the quote is held in a variable at all eight, the shape `_sqlite_sync_lit_into` in sqlite-sync.sh already uses -- and documents, for this exact reason. An undefined quote variable is the worst thing this change can do: `${v///}` with an empty pattern returns the value UNCHANGED, so quotes stop being doubled, silently, on every bash. Three of the seven files had no declaration in scope after a first mechanical pass. tests/test_local_quoting.bats counts substitutions against declarations per file so that cannot pass unnoticed. Held equal to the forking form it replaces, on the inputs that matter to SQL quoting, under the suite's own bash AND under /bin/bash. Two mutations: back to the backslash form green on modern bash, RED under /bin/bash -- the failure #897 hit, reproduced a declaration removed red on the scope count End to end, through the installed scripts: a message carrying a quote, doubled quotes and a backslash arrives byte-identical. Co-authored-by: Gon <kornbetterclub@gmail.com>
Review found it, and the mutation confirms it: reverting an inline site to the
backslash form left all four tests green. That site stops being a counted usage
and is not the forking form either, so it falls through both static guards and
the per-file check compares zero to zero.
Three changes, and the first is the one that closes it:
the per-file COUNT is pinned, plus the total of 8, so a site that leaves the
set is named rather than silently uncounted;
the backslash shape is refused directly, so reverting to it fails on shape
even before the count notices;
the bash-3.2 coverage no longer stops at _sqlite_lit. That function is one of
the eight; the other seven are inline in a SQL string with no call reaching
them, and inline is exactly where the broken form was.
For the third, the claim is split rather than re-running each site by rebuilding
its expression from a grep -- string surgery that can be wrong in ways the test
cannot see, and was, twice, while writing this. Instead:
every site uses this operator form pinned by the count and the shape
refusal above
this form is correct under bash 3.2 measured directly
The same test also asserts the form it REPLACED still fails there, so a machine
where bash 3.2 does not have this behaviour reports that rather than passing and
appearing to have measured something.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Picks up the work in #897, with credit to its author in the commit trailer.
The local send/inbox/history/watch path forked
printf | sedto double single quotes at eight sites. Those are bash parameter expansions now, and the fork is gone from that path.The idea and the eight sites are from #897. What changed is the shape.
Why the shape matters
Seven of the eight were written as
${v//\'/\'\'}, with the quote inline in the pattern.bash 3.2 — which is
/bin/bashon macOS, and what the macOS CI jobs run — keeps the backslash of a\'replacement. Measured on both:a\'\'ba''ba''ba''bThe difference is invisible on Linux, which is why the form reads as correct. It is also why the macOS jobs went red on #897.
So the quote is held in a variable at all eight — the shape
_sqlite_sync_lit_intoinsqlite-sync.shalready uses, and documents, for this exact reason.The failure this change can have, and what stops it
An undefined quote variable is the dangerous one.
${v///}with an empty pattern returns the value unchanged, so quotes silently stop being doubled — on every bash, with no error.Three of the seven files had no declaration in scope after a first mechanical pass. That is not a hypothetical: it was this change, before it was checked.
tests/test_local_quoting.batscounts substitutions against declarations per file, so it cannot pass unnoticed again.Equivalence
Held equal to the forking form it replaces, on the inputs that matter to SQL quoting — quote alone, doubled, leading, trailing, newline, tab, backslash, backslash next to a quote, empty, and
sed's andprintf's own metacharacters — under the suite's own bash and under/bin/bash.The second test skips with a reason when
/bin/bashis 4+, rather than passing quietly while testing nothing.Mutations
/bin/bash— the failure #897 hit, reproducedA is the one worth reading: it is exactly why this could not just be merged as proposed.
Ran
test_messaging27,test_api18,test_export13,test_team79,test_inbox9,test_watch22,test_storage25,test_local_quoting4 — 197 tests, 0 failures, on the committed branch.End to end through the installed scripts: a message carrying a quote, doubled quotes and a backslash arrives byte-identical.
Not run locally: the full matrix. CI's job.
Scope
sqlite-sync.shis deliberately untouched — its remaining sites belong to separate work.scripts/. This converts the 8 on the hot local path, not the rest; the remainder is a sweep with its own risk and its own measurement.