Skip to content

Commit 114cd25

Browse files
committed
Add ClickHouse testgen module for recording ground-truth query analysis
Add a nested, dependency-free Go module at internal/engine/clickhouse/testgen that records what ClickHouse itself reports about a schema, fixture and sqlc query file, in the same JSON shape as `sqlc analyze`, so the two can be diffed. `testgen install` downloads the pinned clickhouse release for the running platform into the user cache directory. `testgen analyze` runs each query in its own `clickhouse local` process: result column types and nullability come from the executed query's result header, provenance from EXPLAIN QUERY TREE (followed through subqueries, CTEs and unions), and parameters from ordinal-carrying sentinel constants substituted for ?, sqlc.arg() and sqlc.narg(), or from DESCRIBE TABLE for INSERT ... VALUES. Golden tests under testdata/ cover the type lowering, expressions, subqueries and exec statements. The analyze_params case reproduces the existing sqlc analyze golden byte for byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
1 parent 8f23752 commit 114cd25

31 files changed

Lines changed: 2399 additions & 0 deletions

‎CLAUDE.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,9 @@ MYSQL_SERVER_URI="root:mysecretpassword@tcp(127.0.0.1:3306)/mysql?multiStatement
233233
- `/postgresql/` - PostgreSQL parser and converter
234234
- `/dolphin/` - MySQL parser (uses TiDB parser)
235235
- `/sqlite/` - SQLite parser
236+
- `/clickhouse/testgen/` - Nested module that records what a real
237+
ClickHouse reports about a schema, fixture and queries, in the `sqlc
238+
analyze` output format; see its README
236239
- `/duckdb/` - DuckDB 2.0 parser (uses darkwing, the pure Go port of
237240
DuckDB's PEG parser)
238241
- `<engine>/dialect/` - The engine's type system and standard library, as
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# ClickHouse testgen
2+
3+
`testgen` records what ClickHouse itself reports about a set of sqlc queries,
4+
so the answer can be committed as a golden file and compared with what sqlc's
5+
own analysis produces for the same schema and queries.
6+
7+
It is a nested Go module with no dependencies beyond the standard library.
8+
Run it from this directory:
9+
10+
```bash
11+
# Download the pinned clickhouse binary into the user cache directory.
12+
go run . install
13+
14+
# Analyze every query in query.sql against schema.sql with fixture.sql loaded.
15+
go run . analyze --schema schema.sql --fixture fixture.sql query.sql
16+
```
17+
18+
The binary is looked up in the `CLICKHOUSE` environment variable first, then
19+
in the cache populated by `install`. The version is pinned in `install.go`;
20+
bumping it can change the query tree format and type inference, so regenerate
21+
and review the goldens afterwards.
22+
23+
## How it works
24+
25+
Each query runs in its own `clickhouse local` process, which needs no server,
26+
no network and no configuration: the schema and fixture are loaded fresh, the
27+
query is explained and then executed, and the process exits.
28+
29+
- **Types and nullability** come from the executed query's result header,
30+
exactly as a driver would see them.
31+
- **Provenance** comes from `EXPLAIN QUERY TREE`, whose resolved columns point
32+
at the table expression they read from. References are followed through
33+
subqueries, CTEs and unions to the base table.
34+
- **Parameters** are invisible to ClickHouse, which substitutes `?` on the
35+
client. Each `?`, `sqlc.arg()` and `sqlc.narg()` is replaced by a constant
36+
expression that carries its ordinal, `(NULL + k)`, or `toUInt64(4294967295 + k)`
37+
after `LIMIT` and `OFFSET`. The query tree prints the expression each folded
38+
constant came from, so the placeholder is found again and described by the
39+
operand it is compared with. Parameters of `INSERT ... VALUES` map onto the
40+
target columns reported by `DESCRIBE TABLE`.
41+
42+
The output has the same shape as `sqlc analyze`, with ClickHouse types lowered
43+
the way sqlc's ClickHouse engine lowers them: the lowercased base name with
44+
parameters dropped, `Nullable` and `Array` folded into `not_null` and
45+
`is_array`, and `LowCardinality` discarded.
46+
47+
## Tests
48+
49+
`testdata/<case>/` holds a `schema.sql`, `query.sql`, an optional
50+
`fixture.sql` and the expected `stdout.txt`. The test skips unless a binary is
51+
available.
52+
53+
```bash
54+
go run . install
55+
go test .
56+
go test . -update # rewrite every stdout.txt
57+
```
Lines changed: 283 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,283 @@
1+
package main
2+
3+
import (
4+
"context"
5+
"encoding/json"
6+
"fmt"
7+
"regexp"
8+
"strconv"
9+
"strings"
10+
)
11+
12+
// The output mirrors the JSON `sqlc analyze` prints so a golden generated
13+
// here can be diffed against sqlc's own analysis of the same files.
14+
15+
type analyzedQuery struct {
16+
Name string `json:"name"`
17+
Cmd string `json:"cmd"`
18+
Columns []analyzedColumn `json:"columns"`
19+
Params []analyzedParam `json:"params"`
20+
}
21+
22+
type analyzedColumn struct {
23+
Name string `json:"name"`
24+
DataType string `json:"data_type"`
25+
NotNull bool `json:"not_null"`
26+
IsArray bool `json:"is_array"`
27+
Table string `json:"table,omitempty"`
28+
}
29+
30+
type analyzedParam struct {
31+
Number int `json:"number"`
32+
Column analyzedColumn `json:"column"`
33+
}
34+
35+
// analyze runs every query against the schema and fixture and records what
36+
// ClickHouse reports about each.
37+
func analyze(ctx context.Context, l local, schema, fixture string, queries []query) ([]analyzedQuery, error) {
38+
out := make([]analyzedQuery, 0, len(queries))
39+
for _, q := range queries {
40+
aq, err := analyzeQuery(ctx, l, schema, fixture, q)
41+
if err != nil {
42+
return nil, fmt.Errorf("%s: %w", q.Name, err)
43+
}
44+
out = append(out, aq)
45+
}
46+
return out, nil
47+
}
48+
49+
func analyzeQuery(ctx context.Context, l local, schema, fixture string, q query) (analyzedQuery, error) {
50+
sql, phs := bindPlaceholders(q.SQL)
51+
explain := returnsRows(sql)
52+
53+
var script strings.Builder
54+
for _, stmt := range []string{schema, fixture} {
55+
if s := strings.TrimRight(strings.TrimSpace(stmt), ";"); s != "" {
56+
script.WriteString(s)
57+
script.WriteString(";\n")
58+
}
59+
}
60+
if explain {
61+
script.WriteString("EXPLAIN QUERY TREE " + sql + ";\n")
62+
}
63+
script.WriteString(sql + ";\n")
64+
65+
results, err := l.run(ctx, script.String())
66+
if err != nil {
67+
return analyzedQuery{}, err
68+
}
69+
70+
aq := analyzedQuery{
71+
Name: q.Name,
72+
Cmd: q.Cmd,
73+
Columns: []analyzedColumn{},
74+
Params: []analyzedParam{},
75+
}
76+
if !explain {
77+
return analyzeExec(ctx, l, script.String(), sql, phs, aq)
78+
}
79+
if len(results) != 2 {
80+
return analyzedQuery{}, fmt.Errorf("expected the query tree and one result set, got %d results", len(results))
81+
}
82+
83+
var lines []string
84+
for _, row := range results[0].Data {
85+
var line string
86+
if err := json.Unmarshal(row["explain"], &line); err != nil {
87+
return analyzedQuery{}, fmt.Errorf("reading query tree: %w", err)
88+
}
89+
lines = append(lines, line)
90+
}
91+
tree, err := parseQueryTree(lines)
92+
if err != nil {
93+
return analyzedQuery{}, err
94+
}
95+
96+
// Names and types come from the block header of the executed query, the
97+
// same header a driver sees. The tree only adds where each came from.
98+
_, nodes := projection(firstNode(tree.root.children))
99+
for i, col := range results[1].Meta {
100+
ac := column(col.Name, col.Type)
101+
if i < len(nodes) && len(nodes) == len(results[1].Meta) {
102+
ac.Table = tree.sourceTable(nodes[i])
103+
}
104+
aq.Columns = append(aq.Columns, ac)
105+
}
106+
107+
sentinels := tree.sentinels()
108+
for i, ph := range phs {
109+
ac := analyzedColumn{}
110+
if sentinel := sentinels[i+1]; sentinel != nil {
111+
ac = tree.paramColumn(sentinel)
112+
}
113+
if ph.Name != "" {
114+
ac.Name = ph.Name
115+
}
116+
aq.Params = append(aq.Params, analyzedParam{Number: ph.Number, Column: ac})
117+
}
118+
return aq, nil
119+
}
120+
121+
func column(name, typ string) analyzedColumn {
122+
dataType, isArray, notNull := sqlcType(typ)
123+
return analyzedColumn{Name: name, DataType: dataType, NotNull: notNull, IsArray: isArray}
124+
}
125+
126+
// returnsRows reports whether a statement produces a result set and so can
127+
// be explained as a query tree.
128+
func returnsRows(sql string) bool {
129+
head := strings.ToLower(strings.TrimSpace(sql))
130+
if strings.HasPrefix(head, "(") {
131+
return true
132+
}
133+
for _, kw := range []string{"select", "with", "show", "describe", "desc", "exists"} {
134+
if strings.HasPrefix(head, kw) && (len(head) == len(kw) || !isWordByte(head[len(kw)])) {
135+
return true
136+
}
137+
}
138+
return false
139+
}
140+
141+
// sentinels finds the constants the placeholders were substituted with,
142+
// keyed by placeholder ordinal.
143+
func (t *queryTree) sentinels() map[int]*treeNode {
144+
found := map[int]*treeNode{}
145+
var walk func(n *treeNode)
146+
walk = func(n *treeNode) {
147+
if n.kind == "CONSTANT" {
148+
if k, ok := sentinelOrdinal(n); ok {
149+
found[k] = n
150+
return
151+
}
152+
}
153+
for _, c := range n.children {
154+
walk(c)
155+
}
156+
}
157+
walk(t.root)
158+
return found
159+
}
160+
161+
// sentinelOrdinal recognises a constant folded directly from `NULL + k` or
162+
// `4294967295 + k`, the shapes sentinelFor produces, and returns k. A
163+
// constant folded from a larger expression that merely contains a sentinel
164+
// does not match; the sentinel is found nested inside it instead.
165+
func sentinelOrdinal(c *treeNode) (int, bool) {
166+
fn := firstNode(c.section("EXPRESSION").childrenOrNil())
167+
if fn == nil || fn.kind != "FUNCTION" || fn.attrs["function_name"] != "plus" {
168+
return 0, false
169+
}
170+
args := fn.section("ARGUMENTS").list()
171+
if len(args) != 2 || args[0].kind != "CONSTANT" || args[1].kind != "CONSTANT" {
172+
return 0, false
173+
}
174+
base := args[0].attrs["constant_value"]
175+
if base != "NULL" && base != "UInt64_"+limitBase {
176+
return 0, false
177+
}
178+
k, err := strconv.Atoi(strings.TrimPrefix(args[1].attrs["constant_value"], "UInt64_"))
179+
if err != nil {
180+
return 0, false
181+
}
182+
return k, true
183+
}
184+
185+
// paramColumn describes what a placeholder is compared with or assigned to:
186+
// the other operand of the function it is an argument of, preferring a
187+
// column over an expression, or the projected column it stands for.
188+
func (t *queryTree) paramColumn(sentinel *treeNode) analyzedColumn {
189+
list := sentinel.parent
190+
if list != nil && list.kind == "LIST" && list.parent != nil {
191+
switch owner := list.parent; {
192+
case owner.kind == "" && owner.text == "ARGUMENTS":
193+
// Prefer a column operand, then an expression, then a constant.
194+
rank := map[string]int{"COLUMN": 0, "FUNCTION": 1, "CONSTANT": 2}
195+
var best *treeNode
196+
for _, sib := range list.children {
197+
if sib == sentinel {
198+
continue
199+
}
200+
if r, ok := rank[sib.kind]; ok && (best == nil || r < rank[best.kind]) {
201+
best = sib
202+
}
203+
}
204+
if best != nil {
205+
return t.describe(best)
206+
}
207+
case owner.kind == "" && owner.text == "PROJECTION":
208+
names, nodes := projection(owner.parent)
209+
for i, n := range nodes {
210+
if n == sentinel && i < len(names) {
211+
ac := column(names[i], sentinel.attrs["constant_value_type"])
212+
return ac
213+
}
214+
}
215+
}
216+
}
217+
return column("", sentinel.attrs["constant_value_type"])
218+
}
219+
220+
// describe turns a tree expression into a column description.
221+
func (t *queryTree) describe(n *treeNode) analyzedColumn {
222+
switch n.kind {
223+
case "COLUMN":
224+
ac := column(n.attrs["column_name"], n.attrs["result_type"])
225+
ac.Table = t.sourceTable(n)
226+
return ac
227+
case "FUNCTION":
228+
return column(n.attrs["function_name"], n.attrs["result_type"])
229+
case "CONSTANT":
230+
return column("", n.attrs["constant_value_type"])
231+
}
232+
return analyzedColumn{}
233+
}
234+
235+
var insertValuesRe = regexp.MustCompile(`(?is)^insert\s+into\s+(?:table\s+)?([\w.` + "`" + `"]+)\s*(?:\(([^)]*)\))?\s*(?:format\s+)?values\b`)
236+
237+
// analyzeExec runs a statement that returns no rows. The only parameters it
238+
// can describe are those of an INSERT ... VALUES, which map positionally
239+
// onto the target columns reported by DESCRIBE TABLE.
240+
func analyzeExec(ctx context.Context, l local, script, sql string, phs []placeholder, aq analyzedQuery) (analyzedQuery, error) {
241+
m := insertValuesRe.FindStringSubmatch(sql)
242+
if m != nil {
243+
script += "DESCRIBE TABLE " + m[1] + ";\n"
244+
}
245+
results, err := l.run(ctx, script)
246+
if err != nil {
247+
return analyzedQuery{}, err
248+
}
249+
250+
var targets []analyzedColumn
251+
if m != nil && len(results) == 1 {
252+
byName := map[string]analyzedColumn{}
253+
var all []analyzedColumn
254+
table := strings.Trim(m[1][strings.LastIndexByte(m[1], '.')+1:], "`\"")
255+
for _, row := range results[0].Data {
256+
var name, typ string
257+
json.Unmarshal(row["name"], &name)
258+
json.Unmarshal(row["type"], &typ)
259+
ac := column(name, typ)
260+
ac.Table = table
261+
byName[name] = ac
262+
all = append(all, ac)
263+
}
264+
if strings.TrimSpace(m[2]) == "" {
265+
targets = all
266+
} else {
267+
for _, name := range strings.Split(m[2], ",") {
268+
targets = append(targets, byName[strings.Trim(strings.TrimSpace(name), "`\"")])
269+
}
270+
}
271+
}
272+
for i, ph := range phs {
273+
ac := analyzedColumn{}
274+
if len(targets) > 0 {
275+
ac = targets[i%len(targets)]
276+
}
277+
if ph.Name != "" {
278+
ac.Name = ph.Name
279+
}
280+
aq.Params = append(aq.Params, analyzedParam{Number: ph.Number, Column: ac})
281+
}
282+
return aq, nil
283+
}
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
module github.com/sqlc-dev/sqlc/internal/engine/clickhouse/testgen
2+
3+
go 1.24.0

0 commit comments

Comments
 (0)