Skip to content

mssql: core query support (joins, FROM sources, query shapes, literals, macros) - #3

Open
EliSauder wants to merge 43 commits into
mainfrom
mssql/core-queries
Open

EliSauder wants to merge 43 commits into
mainfrom
mssql/core-queries

Conversation

@EliSauder

@EliSauder EliSauder commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

SQL Server query support in the analysis core: what a query's scope holds and how its clauses bind, how literals and conversions are typed, placeholders sqlc lost, user functions in FROM, and the sqlc macros and star expansion.

  • Joins and FROM: outer-join nullability, CROSS/OUTER APPLY, OPENJSON/STRING_SPLIT/GENERATE_SERIES, VALUES tables, PIVOT/UNPIVOT, scalar and table-valued functions the schema creates.
  • Query shapes: recursive CTEs (and WITH scope), WITH on INSERT, set-operation branches seeing the outer query, FOR JSON/XML, OUTPUT INSERTED/DELETED.
  • Literals and conversions: TRY_CAST/TRY_CONVERT, PARSE/TRY_PARSE, @@ variables, N'' and decimal literals, TOP/OFFSET types, the core lookup of a type given default arguments.
  • Placeholders: INSERT … SELECT, FOR SYSTEM_TIME, CONTAINS/FREETEXT, TABLESAMPLE.
  • Macros and stars: sqlc.arg/narg/slice/embed for SQL Server, bracket quoting of expanded stars, and the review fixes to them.

Commits

  • 1b00606c0 fix(mssql): make the optional side of outer joins nullable
  • cc0906800 feat(mssql): support CROSS APPLY and OUTER APPLY
  • 42ea003ed fix(mssql): type TOP and OFFSET/FETCH parameters as bigint
  • 3b901d64c feat(mssql): support recursive common table expressions
  • b4fd639c6 feat(mssql): support OPENJSON, STRING_SPLIT and GENERATE_SERIES in FROM
  • 94a9d7a61 feat(mssql): support sqlc.arg, sqlc.narg and sqlc.slice
  • 948ff95d3 feat(mssql): support sqlc.embed
  • e46519f90 fix(core): let set operation branches see the enclosing query
  • d130dd565 feat(mssql): support VALUES as a derived table
  • 6bdd726fb fix(mssql): resolve INSERTED and DELETED in OUTPUT against the target
  • 108d8a1bf fix(mssql): type TRY_CAST and TRY_CONVERT as nullable
  • dd318d5bc feat(mssql): type PARSE and TRY_PARSE
  • 142a71322 feat(mssql): type @@rowcount and the other system variables
  • 6476de912 feat(mssql): support PIVOT and UNPIVOT in FROM
  • 4ac96f6bf fix(mssql): quote expanded identifiers with brackets
  • 033df822c feat(mssql): type FOR JSON and FOR XML results
  • b6aef71c5 fix(golang): expand every use of a named slice
  • e85a2ac44 fix(mssql): expand a star qualified with double quotes
  • 969a10e81 fix(mssql): embed a table only when the star is all of its columns
  • 6a3e9b2f6 fix(core): keep a WITH clause's names out of the enclosing query
  • 1fe1a2d2b fix(mssql): leave bracketed identifiers alone when preprocessing
  • d4a19b553 fix(mssql): reject a macro name that is not a valid @name
  • d630cd1eb fix(mssql): keep the placeholders of FOR SYSTEM_TIME
  • c3ceb10d7 fix(mssql): keep the placeholders of CONTAINS and FREETEXT
  • de920bcfc feat(mssql): support WITH on INSERT
  • 0e560ff5a feat(mssql): type a call to a scalar function the schema creates
  • a90357665 feat(mssql): select from a table-valued function the schema creates
  • 2420922ba fix(core): type a placeholder INSERT ... SELECT inserts as its column
  • 2861faa7e fix(mssql): type an N'...' literal as nvarchar
  • 444e97c30 fix(core): find the family of a type given default arguments
  • 043680e89 fix(mssql): type a number with a decimal point as decimal
  • fe55ab361 fix(mssql): keep the placeholders of TABLESAMPLE
  • 896a11245 fix(core): stop pairing INSERT ... SELECT items with columns after a star
  • 70864b872 fix(mssql): leave sqlc.embed out of the @name check
  • 916869af9 fix(mssql): read a placeholder's name through #, $ and @
  • 8e0928ffa fix(mssql): measure a star qualifier with escaped quotes as written
  • d1d16c96a fix(mssql): embed a table only when its columns are as nullable as the table's

Test changes for independence

  • mssql_correlated_union: the UNION branches select integer literals, since a UNION column's type and source table come from the expression typing PR.
  • mssql_output_clause: books.price is FLOAT rather than DECIMAL, which maps to another Go type in the Go codegen PR.
  • analyze_national_literals, analyze_uninterned_decimal, analyze_numeric_literals: an expression whose type comes from the expression typing PR (COALESCE, int − decimal) is left out.
  • The CONTAINS/FREETEXT converter case, which mssql: close SQL Server analysis and codegen gaps #1 committed with FOR SYSTEM_TIME by mistake, is in its own commit here.

Independence. This is one of five PRs split from #1, each off main with no dependency on the others: it builds, passes the base suite and keeps the core context at or above main's, alone. Its goldens are generated on this branch, and they hold unchanged with all five merged, whose code then equals the original branch. Parallel PRs add lines next to each other in convert.go, functions.jsonl and the dialect settings, so whichever merges later may need a rebase that keeps both sides.

Verification. Behavior was checked against SQL Server 2025 in #1 (goldeneye or sp_describe_*). go test ./internal/endtoend -run TestReplay/base/ and the touched packages' tests pass; generated Go goldens pass go vet.

EliSauder and others added 30 commits November 12, 2026 18:47
The core analyzer bound every relation in a join with the nullability its
columns were declared with, so LEFT, RIGHT and FULL joins reported the
optional side as NOT NULL. A new dialect setting, outer_join_nullable,
makes those columns nullable. SQL Server turns it on; ClickHouse, which
fills unmatched rows with defaults, keeps the old behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
APPLY reached the analyzer as an unconverted node and failed with
"unsupported FROM item". Convert it to the join it is: CROSS APPLY to an
inner join and OUTER APPLY to a left join. The analyzer already binds
each FROM item against the ones before it, so the applied subquery's
references to the left side resolve.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A placeholder counting rows fell back to the core's default int. SQL
Server describes the parameter of TOP (@n), OFFSET @n ROWS and FETCH
NEXT @n ROWS as bigint, so the dialect now says so.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A CTE that read itself failed: the analyzer bound a CTE only once its
whole query was analyzed, so the self-reference named an unknown
relation. When a WITH clause is recursive, the anchor member is now bound
first and the full query analyzed against it. SQL Server describes every
column of a recursive CTE as nullable and read from no table, and so
does sqlc. T-SQL has no RECURSIVE keyword, so the converter marks every
WITH clause recursive; only a CTE that names itself is treated as one.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The table-valued built-ins reached the analyzer unconverted and failed
with "unsupported FROM item". The converter now reports each as a range
function with a column definition list: OPENJSON's key, value and type,
or the columns its WITH clause declares; STRING_SPLIT's value, plus its
ordinal when enable_ordinal is passed; GENERATE_SERIES's value.

The core analyzer reads a range function's column definition list,
where a definition with no type is the function's own result, and a
seeded "$n" result over an untyped placeholder takes the parameter's
declared type rather than leaking the pseudo-type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SQL Server had no preprocessor dialect, so sqlc macros reached the T-SQL
parser as function calls and produced no parameter. Add one that rewrites
them to native @name placeholders, numbers bare @name alongside them and
folds name case as SQL Server does, leaving @@ system functions alone.

Named-arg queries cannot expand a slice into positional ?s, so expand it
into the @p<ordinal> names go-mssqldb gives unnamed arguments and pass the
other parameters through sql.Named.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The core analyzer path expanded a rewritten sqlc.embed(t) star into the
table's columns but never folded them back into one embedded column, so
SQL Server queries got flat rows instead of the table's struct. Fold each
embed star's columns into a column naming its source table, which also
resolves an embed of an alias to the table behind it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A subquery's analyzer took the scope of the query it is nested in as its
outer scope. A set operation binds no scope of its own, so the branches
of a UNION nested in another query got none, and a correlated reference
in them, as in CROSS APPLY (SELECT ... WHERE x = a.id UNION ALL ...),
failed as an unknown column. They now see the scope the set operation
itself sees.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
(VALUES ...) AS v(a, b) in FROM reached the analyzer unconverted. It is
now the subquery it stands for, each row a SELECT combined with UNION
ALL, and its columns are named by the alias list. A VALUES table can be
joined, applied, and refer to the relations before it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The converter stripped the INSERTED and DELETED qualifiers from an
OUTPUT clause so the columns resolved against the target table. With a
FROM clause joining other tables, the bare names were ambiguous, so
DELETE b OUTPUT DELETED.id FROM books b JOIN authors a ... failed. And a
star lost its qualifier, so OUTPUT INSERTED.* was rewritten into broken
SQL: the expansion replaced only the star's first character and named
columns without the prefix SQL Server requires.

A dialect may now name, in returning_aliases, the pseudo-relations its
RETURNING list qualifies the target's columns with. The analyzer reads
them as the target while it types that list, and the converter keeps
the qualifiers, so INSERTED.* expands to inserted.id, inserted.name, ...

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TRY_CAST and TRY_CONVERT yield NULL for a value they cannot convert, but
were converted as plain casts, which are NOT NULL over a NOT NULL
operand. Their target type is now marked nullable, as SQL Server
describes it. A placeholder cast this way is still typed NOT NULL by the
cast, as with CAST.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PARSE and TRY_PARSE were left unconverted, so their result was untyped
and a placeholder in them was lost. They are now the conversions they
are: PARSE as CAST, TRY_PARSE as the nullable TRY_CAST. What they parse
is a string, so a placeholder there is the nvarchar(4000) SQL Server
takes it as, and a culture placeholder is nvarchar.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@@rowcount, @@IDENTITY, @@TRANCOUNT and their relatives were left
unconverted, so a column reading one was untyped. Each is now the call
of a function seeded under its name, with the type and nullability SQL
Server describes it with.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both reached the analyzer as an unconverted node. Their columns are the
source's columns less the rotated ones, plus the ones the rotation makes,
which only the analyzer knows, so the converter passes them on in a new
RangePivot node that the analyzer binds against its source.

A star over a PIVOT expands to columns named after its values, as 2023,
so the expansion quotes names that are no regular identifier.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A star expanded into a column named by a reserved word, or by something
that is not an identifier at all, such as PIVOT's [2023], was quoted
with double quotes, which SQL Server reads as an identifier only while
QUOTED_IDENTIFIER is on. SQL Server's own brackets quote it whatever the
setting, with a closing bracket doubled.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A query ending in FOR JSON or FOR XML returns one document column, but
was reported as returning its select list. It is now that column:
nvarchar(max) for JSON, ntext for XML, and xml with TYPE, nullable and
named as SQL Server names it; as a subquery, FOR XML without TYPE is
nvarchar(max). The query itself is kept as a subquery of the column, so
its placeholders and stars are still analyzed.

A query with no FROM, like that column's, binds no relations, so what
is nested in it now sees the enclosing query, as a set operation's
branches already did, and a correlated FOR JSON subquery resolves.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A query that uses one sqlc.slice() twice binds a single parameter, but
the generated code for a query bound by name replaced only the first
/*SLICE:x*/@x marker, leaving the others unbound. Every marker is now
replaced with the same placeholders, which the driver binds by position
however many times they appear.

Addresses review comment r4557024903 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Star expansion measures the reference it replaces by the quoting the
dialect writes, which for SQL Server became brackets. A qualifier the
user wrote in double quotes, which SQL Server also reads as an
identifier, no longer matched, so "o".* was replaced only in part and
left broken SQL. A double-quoted qualifier is now measured as written.

Addresses review comment r4557024959 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sqlc.embed() on the core path folded a star into the table its first
column came from, whatever else the star expanded to. The struct scans
every column of the table in order, so an embed of a subquery that
selects some of its columns, or reorders them, scanned the wrong number
of values or into the wrong fields. A star is now folded only when it is
exactly the table's columns in order, and is otherwise left flat.

Addresses review comment r4557025012 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The analyzers of nested queries share the map of CTEs they inherit, so
a WITH clause in a subquery added its names to the enclosing query's
map, where a CTE could hide a table of the same name: with an inner
WITH authors AS (...), a later SELECT name FROM authors in the outer
query read the CTE and failed. A WITH clause now binds its names in its
own copy of the map.

Addresses review comment r4557025055 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The preprocessor skips the quoted regions of a query, but did not know
SQL Server's bracketed identifiers, so [sqlc.arg(x)] was rewritten to
[@x], an @name inside brackets was taken for a parameter, and a
semicolon inside them split the statement. A dialect may now say that
[...] quotes an identifier, with ]] standing for a bracket in it, and
SQL Server does.

Addresses review comment r4557025105 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SQL Server binds a parameter by the name sqlc writes into the query, so
sqlc.arg('user-id') became @user-ID, which the server reads as @user
minus id, and a name with a space did not parse; the generated Go did
not compile either way. For a dialect that binds @name, a name passed to
sqlc.arg, sqlc.narg or sqlc.slice must now be one an @name can spell,
and is otherwise reported at the call.

Addresses review comment r4557025173 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A table read FOR SYSTEM_TIME AS OF @t, BETWEEN, FROM ... TO or
CONTAINED IN converted to the plain table, dropping the placeholders
the clause holds, so the generated code did not pass them. The table
reference has nowhere to hold an expression, so the times are now
analyzed with the query's WHERE clause, as the datetime2 SQL Server
takes them as.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The full-text predicates were left unconverted, which dropped a
placeholder for the search condition from the query's parameters, so
the generated code did not pass it. They are now calls of the functions
seeded under their names, whose search condition is the nvarchar(4000)
SQL Server takes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The converter dropped the WITH clause of an INSERT, and the core
analyzer bound CTEs for SELECT, UPDATE and DELETE but not INSERT, so an
INSERT ... SELECT reading a CTE failed with an unknown relation. Both
now carry it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CREATE FUNCTION was ignored, so a query calling the function failed. A
scalar function is now recorded with its parameters and result, which
SQL Server reports nullable whatever the arguments, so the converter
returns Nullable(T) and the schema records a T that may be NULL.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A table-valued function called in FROM was not converted, so the query
failed. A function in FROM is now called, and a table-valued function
returns rows of table(col type, ...): the columns a multi-statement
function declares, or those an inline function's query selects, which
the schema works out as it does a view's. A function returning such rows
binds as a relation of those columns.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A placeholder in VALUES took the type of the column it is inserted into,
but one selected by INSERT ... SELECT @A went untyped. A placeholder
selected as it is now takes its column's type too.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A Unicode string literal was typed as a varchar, as any string literal
is. N'...' is now an nvarchar as long as the string, as SQL Server
types it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A type the catalog holds no row for, as decimal(5,1) cast to before any
column has it, is looked up by its family. Looking the family up by its
bare name applied the dialect's default arguments again, as SQL Server's
decimal is decimal(18,0), which held no row either, so the expression
went untyped and its operators resolved against nothing. The family is
now found by name when its bare name is no row.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
EliSauder and others added 7 commits November 12, 2026 18:50
A literal such as 1.5 was typed as a float, and one too large for an
int, as 2147483648, as one too. SQL Server types each as the decimal
holding its digits, 1.5 as decimal(2,1) and 2147483648 as
decimal(10,0), and so now does sqlc. 1e5 is still a float.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TABLESAMPLE (@n PERCENT) and REPEATABLE (@seed) dropped their
placeholders. They are now analyzed with the query's WHERE clause, as
FOR SYSTEM_TIME's are: a percentage as a float, a count of rows and a
seed as a bigint.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…star

A placeholder selected by INSERT ... SELECT took the type of the column
at its position, but a star before it selects any number of columns, so
INSERT INTO t (id, name, at, n) SELECT t2.*, @n typed @n as name. Items
after a star are no longer paired.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sqlc.embed names a table, not a parameter, so sqlc.embed("order
details") was rejected as an invalid @name. Only sqlc.arg, narg and
slice are checked now.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SQL Server's names go on through #, $ and @ after the first character,
but the preprocessor stopped at them, so @A and @A#b were both read as
@A and given one number, and generated code bound only one of them.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A star qualified by a name that needs quoting was measured from the name
as sqlc quotes it for output, so "a""b".* read as [a"b] measured
short and left the star in the query, and "x]y".* measured long. Each
quoted form is now built from the name itself, doubling its quotes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e table's

sqlc.embed(b) on the optional side of an outer join folded b's columns
into the Book model, whose fields cannot hold NULL, so scanning a row
with no match failed. A table is now embedded only when each column is
as nullable as the table's, and otherwise its columns stay flat.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@EliSauder EliSauder left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions remain

Comment thread internal/sql/preprocess/scan.go Resolved
Comment thread internal/compiler/expand.go Resolved
Comment thread internal/compiler/parse_core.go Resolved
Comment thread internal/compiler/parse_core.go Resolved
Comment thread internal/core/analyzer/analyzer.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
EliSauder and others added 6 commits November 12, 2026 21:39
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant