fix(lexer): don't treat backslash as string escape in SQL Server and Oracle

[databergs]

Summary

SQL Server (T-SQL) and Oracle both follow ANSI-SQL string semantics: backslash is an ordinary character, and a quote inside a string literal is escaped by doubling it (''). The lexer, however, unconditionally treated \ as an escape character in scanStringWithDelimiter.

As a result, a literal like ESCAPE '\' (whose contents are a single backslash) had its closing quote treated as escaped, so scanning kept going and swallowed the rest of the batch up to the next quote. Everything in between collapsed into one string literal → obfuscated to a single ?, and the query text appeared truncated in DBM.

Reported for SQL Server in SDBM-2792; Oracle has the same root cause and is fixed here too. A previous "special case" only rescued this when the query ended with '\'; the bug remained whenever more SQL followed.

Before / after (mssql)

-- input (two ESCAPE N'\' literals with SQL between and after)
... WHERE col LIKE @p1 ESCAPE N'\' AND col LIKE @p2 ESCAPE N'\';

-- before (truncated — the first '\' escapes its closing quote and swallows
-- forward to the second literal's opening quote)
... WHERE col LIKE @p1 ESCAPE N?\?

-- after
... WHERE col LIKE @p1 ESCAPE N? AND col LIKE @p2 ESCAPE N?;

The fix

Disable backslash-escaping for the two ANSI-flavored dialects:

backslashEscapes := s.config.DBMS != DBMSSQLServer && s.config.DBMS != DBMSOracle
...
if backslashEscapes && ch == '\\' { ... }

For SQL Server and Oracle, '\' now terminates correctly whether or not more SQL follows.

Dialect coverage

DBMS	Backslash escapes in '...'?	Behavior
SQL Server	No (quotes doubled)	fixed
Oracle	No (quotes doubled)	fixed
MySQL	Yes (default)	unchanged — real MySQL semantics
Snowflake	Yes	unchanged
PostgreSQL	No for ordinary '...', but yes for E'...'	left as-is (see below)
default (no dbms)	generic fallback	unchanged

MySQL and Snowflake genuinely use C-style backslash escaping, so disabling it there would introduce a bug — hence they're deliberately untouched.

Tests

All test queries are synthetic (col/tbl/%foo%/%bar%); they reproduce the problematic shape without any data from the ticket.

• Tokenizer specs — TestLexer/sqlserver_escape_backslash_mid-query and .../oracle_escape_backslash_mid-query pin '\' as its own STRING token when more SQL follows.
• Obfuscator regressions — SQL Server (DECLARE ... N'...' with two ESCAPE N'\' literals) and Oracle (two ESCAPE '\' literals) with SQL both between and after the escapes. The two-literal shape is the one that produced the truncation (the first '\' would otherwise consume forward to the opening quote of the second literal), so it is covered explicitly on both dialect paths.
• Boundary guards — MySQL and Snowflake obfuscator tests assert they still absorb the trailing text (i.e. backslash-escaping stays on), proving the fix didn't over-broaden.
• All specs were written failing first; the full suite passes after the fix.

Scope / follow-ups

• PostgreSQL shares the same latent bug for ordinary strings, but its E'...' escape-string syntax does use backslash. The lexer tokenizes the E prefix as a separate token and cannot yet tell an escape string from an ordinary one, so a correct Postgres fix needs that distinction first — deferred to a follow-up.
• Doubled-quote ('') escaping is still not handled for any dialect — separate issue.
• Downstream: the production SQL Server / Oracle DBM paths must invoke the obfuscator with the matching dialect for customers to see the fix.

🤖 Generated with Claude Code

[datadog-official]

More details

The PR correctly fixes a SQL Server string-parsing bug where backslash was unconditionally treated as an escape character. Since SQL Server doesn't use backslash for escaping (it uses quote-doubling), strings like ESCAPE '\' were incorrectly treated as having an escaped closing quote, causing the lexer to swallow the rest of the query. The fix gates backslash-escape behavior to non-SQL-Server dialects, restoring correct parsing for SQL Server while preserving MySQL and default dialect behavior. All three test suites pass, and edge cases validate correctly.

Was this helpful? React 👍 or 👎

_{📊 Validated against 15 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 5abf8da · What is Autotest? · Any feedback? Reach out in #autotest}

[Copilot]

Pull request overview

This PR fixes dialect-specific string scanning so the lexer does not treat \ as a string escape character for SQL Server (T-SQL) and Oracle, preventing unterminated-string behavior that could swallow subsequent SQL and lead to truncated/over-obfuscated output.

Changes:

• Update scanStringWithDelimiter to gate backslash-escape handling behind a dialect check (disabled for SQL Server and Oracle).
• Add lexer specs ensuring ESCAPE '\' terminates correctly mid-query for SQL Server and Oracle.
• Add obfuscator regression tests for SQL Server/Oracle and “boundary guard” tests for MySQL/Snowflake to confirm their behavior remains unchanged.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
sqllexer.go	Disables backslash-escape handling for SQL Server and Oracle during string scanning.
sqllexer_test.go	Adds tokenizer tests that reproduce the mid-query ESCAPE '\' termination case for SQL Server and Oracle.
obfuscator_test.go	Adds regressions verifying obfuscation no longer truncates SQL Server/Oracle queries and remains unchanged for MySQL/Snowflake.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.