Skip to content

Commit 94078a0

Browse files
committed
Write testgen types as call expressions
Replace the data_type and is_array fields with one type expression per column: a lowercased name applied to arguments that are numbers, quoted strings, identifiers, other calls, or any of those with a label. Nullable, Array and LowCardinality are ordinary names in that grammar, so nested types such as Array(Nullable(String)), Map(String, Nullable(UInt8)) and Tuple(lat Float64, lon Float64) survive intact. An outer Nullable is lifted into the column's not_null flag; deeper ones stay in the expression. Resolving the names is left to the reader of the output. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
1 parent 114cd25 commit 94078a0

13 files changed

Lines changed: 280 additions & 283 deletions

File tree

‎CLAUDE.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -234,8 +234,8 @@ MYSQL_SERVER_URI="root:mysecretpassword@tcp(127.0.0.1:3306)/mysql?multiStatement
234234
- `/dolphin/` - MySQL parser (uses TiDB parser)
235235
- `/sqlite/` - SQLite parser
236236
- `/clickhouse/testgen/` - Nested module that records what a real
237-
ClickHouse reports about a schema, fixture and queries, in the `sqlc
238-
analyze` output format; see its README
237+
ClickHouse reports about a schema, fixture and queries, in the shape of
238+
`sqlc analyze` output with types as call expressions; see its README
239239
- `/duckdb/` - DuckDB 2.0 parser (uses darkwing, the pure Go port of
240240
DuckDB's PEG parser)
241241
- `<engine>/dialect/` - The engine's type system and standard library, as

‎internal/engine/clickhouse/testgen/README.md‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# ClickHouse testgen
22

33
`testgen` records what ClickHouse itself reports about a set of sqlc queries,
4-
so the answer can be committed as a golden file and compared with what sqlc's
4+
so the answer can be committed as a golden file and held against what sqlc's
55
own analysis produces for the same schema and queries.
66

77
It is a nested Go module with no dependencies beyond the standard library.
@@ -39,10 +39,25 @@ query is explained and then executed, and the process exits.
3939
operand it is compared with. Parameters of `INSERT ... VALUES` map onto the
4040
target columns reported by `DESCRIBE TABLE`.
4141

42-
The output has the same shape as `sqlc analyze`, with ClickHouse types lowered
43-
the way sqlc's ClickHouse engine lowers them: the lowercased base name with
44-
parameters dropped, `Nullable` and `Array` folded into `not_null` and
45-
`is_array`, and `LowCardinality` discarded.
42+
The output has the shape of `sqlc analyze`, except that each column's type is
43+
one expression rather than a name and flags. A type is written as a call: a
44+
lowercased name applied to arguments that are numbers, quoted strings,
45+
identifiers, other calls, or any of those with a label. `Nullable`, `Array`
46+
and `LowCardinality` are ordinary names in that grammar, so nothing about
47+
nesting is lost:
48+
49+
| ClickHouse | testgen |
50+
|-----------------------------------|------------------------------------|
51+
| `Array(Nullable(String))` | `array(nullable(string))` |
52+
| `Map(String, UInt32)` | `map(string, uint32)` |
53+
| `Tuple(lat Float64, lon Float64)` | `tuple(lat: float64, lon: float64)`|
54+
| `Enum8('a' = 1, 'b' = 2)` | `enum8('a': 1, 'b': 2)` |
55+
| `DateTime64(3, 'UTC')` | `datetime64(3, 'UTC')` |
56+
| `LowCardinality(Nullable(String))`| `lowcardinality(string)`, `not_null: false` |
57+
58+
An outer `Nullable` is the column's nullability rather than part of its type,
59+
so it is lifted into `not_null`; a `Nullable` at any greater depth stays in
60+
the expression. Resolving the names is left to whoever reads the output.
4661

4762
## Tests
4863

‎internal/engine/clickhouse/testgen/analyze.go‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,8 @@ import (
99
"strings"
1010
)
1111

12-
// The output mirrors the JSON `sqlc analyze` prints so a golden generated
13-
// here can be diffed against sqlc's own analysis of the same files.
12+
// The output follows the JSON `sqlc analyze` prints, with each type written
13+
// as a call expression (see types.go) instead of a name and flags.
1414

1515
type analyzedQuery struct {
1616
Name string `json:"name"`
@@ -20,11 +20,10 @@ type analyzedQuery struct {
2020
}
2121

2222
type analyzedColumn struct {
23-
Name string `json:"name"`
24-
DataType string `json:"data_type"`
25-
NotNull bool `json:"not_null"`
26-
IsArray bool `json:"is_array"`
27-
Table string `json:"table,omitempty"`
23+
Name string `json:"name"`
24+
Type string `json:"type"`
25+
NotNull bool `json:"not_null"`
26+
Table string `json:"table,omitempty"`
2827
}
2928

3029
type analyzedParam struct {
@@ -119,8 +118,8 @@ func analyzeQuery(ctx context.Context, l local, schema, fixture string, q query)
119118
}
120119

121120
func column(name, typ string) analyzedColumn {
122-
dataType, isArray, notNull := sqlcType(typ)
123-
return analyzedColumn{Name: name, DataType: dataType, NotNull: notNull, IsArray: isArray}
121+
expr, notNull := typeExpr(typ)
122+
return analyzedColumn{Name: name, Type: expr, NotNull: notNull}
124123
}
125124

126125
// returnsRows reports whether a statement produces a result set and so can

‎internal/engine/clickhouse/testgen/main.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@
55
// It loads a schema and a fixture into an ephemeral `clickhouse local`
66
// process, runs each query found in a sqlc query file against that data, and
77
// prints the result column types, nullability and source tables along with
8-
// the parameters each query binds, in the same JSON shape `sqlc analyze`
9-
// prints.
8+
// the parameters each query binds, in the JSON shape `sqlc analyze` prints
9+
// with each type written as a call expression.
1010
//
1111
// The clickhouse binary is downloaded once per pinned version with
1212
// `testgen install`, or supplied through the CLICKHOUSE environment variable.

‎internal/engine/clickhouse/testgen/testdata/analyze_params/stdout.txt‎

Lines changed: 7 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,14 @@
55
"columns": [
66
{
77
"name": "id",
8-
"data_type": "uint64",
8+
"type": "uint64",
99
"not_null": true,
10-
"is_array": false,
1110
"table": "events"
1211
},
1312
{
1413
"name": "name",
15-
"data_type": "string",
14+
"type": "string",
1615
"not_null": true,
17-
"is_array": false,
1816
"table": "events"
1917
}
2018
],
@@ -23,9 +21,8 @@
2321
"number": 1,
2422
"column": {
2523
"name": "id",
26-
"data_type": "uint64",
24+
"type": "uint64",
2725
"not_null": true,
28-
"is_array": false,
2926
"table": "events"
3027
}
3128
}
@@ -37,16 +34,14 @@
3734
"columns": [
3835
{
3936
"name": "id",
40-
"data_type": "uint64",
37+
"type": "uint64",
4138
"not_null": true,
42-
"is_array": false,
4339
"table": "events"
4440
},
4541
{
4642
"name": "name",
47-
"data_type": "string",
43+
"type": "string",
4844
"not_null": true,
49-
"is_array": false,
5045
"table": "events"
5146
}
5247
],
@@ -55,19 +50,17 @@
5550
"number": 1,
5651
"column": {
5752
"name": "name",
58-
"data_type": "string",
53+
"type": "string",
5954
"not_null": true,
60-
"is_array": false,
6155
"table": "events"
6256
}
6357
},
6458
{
6559
"number": 2,
6660
"column": {
6761
"name": "amount",
68-
"data_type": "float64",
62+
"type": "float64",
6963
"not_null": true,
70-
"is_array": false,
7164
"table": "events"
7265
}
7366
}

‎internal/engine/clickhouse/testgen/testdata/exec/stdout.txt‎

Lines changed: 10 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,17 @@
88
"number": 1,
99
"column": {
1010
"name": "id",
11-
"data_type": "uint64",
11+
"type": "uint64",
1212
"not_null": true,
13-
"is_array": false,
1413
"table": "users"
1514
}
1615
},
1716
{
1817
"number": 2,
1918
"column": {
2019
"name": "email",
21-
"data_type": "string",
20+
"type": "string",
2221
"not_null": true,
23-
"is_array": false,
2422
"table": "users"
2523
}
2624
}
@@ -35,39 +33,35 @@
3533
"number": 1,
3634
"column": {
3735
"name": "id",
38-
"data_type": "uint64",
36+
"type": "uint64",
3937
"not_null": true,
40-
"is_array": false,
4138
"table": "events"
4239
}
4340
},
4441
{
4542
"number": 2,
4643
"column": {
4744
"name": "name",
48-
"data_type": "string",
45+
"type": "string",
4946
"not_null": true,
50-
"is_array": false,
5147
"table": "events"
5248
}
5349
},
5450
{
5551
"number": 3,
5652
"column": {
5753
"name": "tag",
58-
"data_type": "string",
54+
"type": "string",
5955
"not_null": false,
60-
"is_array": false,
6156
"table": "events"
6257
}
6358
},
6459
{
6560
"number": 4,
6661
"column": {
6762
"name": "amount",
68-
"data_type": "float64",
63+
"type": "float64",
6964
"not_null": true,
70-
"is_array": false,
7165
"table": "events"
7266
}
7367
}
@@ -82,39 +76,35 @@
8276
"number": 1,
8377
"column": {
8478
"name": "id",
85-
"data_type": "uint64",
79+
"type": "uint64",
8680
"not_null": true,
87-
"is_array": false,
8881
"table": "events"
8982
}
9083
},
9184
{
9285
"number": 2,
9386
"column": {
9487
"name": "name",
95-
"data_type": "string",
88+
"type": "string",
9689
"not_null": true,
97-
"is_array": false,
9890
"table": "events"
9991
}
10092
},
10193
{
10294
"number": 3,
10395
"column": {
10496
"name": "id",
105-
"data_type": "uint64",
97+
"type": "uint64",
10698
"not_null": true,
107-
"is_array": false,
10899
"table": "events"
109100
}
110101
},
111102
{
112103
"number": 4,
113104
"column": {
114105
"name": "name",
115-
"data_type": "string",
106+
"type": "string",
116107
"not_null": true,
117-
"is_array": false,
118108
"table": "events"
119109
}
120110
}

0 commit comments

Comments
 (0)