Skip to content

Commit 8431c98

Browse files
committed
Emit testgen types as JSON call trees and drop not_null
A column's type is now a JSON expression: a lowercased name applied to arguments, each carrying an optional label and exactly one of type, int or string. The shape maps one to one onto a protobuf message with a oneof for the argument value. There is no separate nullability flag any more: a nullable column is one whose type is nullable(...), which keeps Nullable at every depth where ClickHouse put it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
1 parent 94078a0 commit 8431c98

8 files changed

Lines changed: 790 additions & 309 deletions

File tree

‎internal/engine/clickhouse/testgen/README.md‎

Lines changed: 24 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -40,24 +40,30 @@ query is explained and then executed, and the process exits.
4040
target columns reported by `DESCRIBE TABLE`.
4141

4242
The output has the shape of `sqlc analyze`, except that each column's type is
43-
one expression rather than a name and flags. A type is written as a call: a
44-
lowercased name applied to arguments that are numbers, quoted strings,
45-
identifiers, other calls, or any of those with a label. `Nullable`, `Array`
46-
and `LowCardinality` are ordinary names in that grammar, so nothing about
47-
nesting is lost:
48-
49-
| ClickHouse | testgen |
50-
|-----------------------------------|------------------------------------|
51-
| `Array(Nullable(String))` | `array(nullable(string))` |
52-
| `Map(String, UInt32)` | `map(string, uint32)` |
53-
| `Tuple(lat Float64, lon Float64)` | `tuple(lat: float64, lon: float64)`|
54-
| `Enum8('a' = 1, 'b' = 2)` | `enum8('a': 1, 'b': 2)` |
55-
| `DateTime64(3, 'UTC')` | `datetime64(3, 'UTC')` |
56-
| `LowCardinality(Nullable(String))`| `lowcardinality(string)`, `not_null: false` |
57-
58-
An outer `Nullable` is the column's nullability rather than part of its type,
59-
so it is lifted into `not_null`; a `Nullable` at any greater depth stays in
60-
the expression. Resolving the names is left to whoever reads the output.
43+
one expression rather than a name and flags. A type is a call: a lowercased
44+
`name` applied to `args`, each of which carries an optional `label` and
45+
exactly one of `type`, `int` or `string`. `Nullable`, `Array` and
46+
`LowCardinality` are ordinary names in that grammar, so nothing about nesting
47+
is lost, and there is no separate nullability flag: a nullable column is one
48+
whose type is `nullable(...)`.
49+
50+
```json
51+
{"name": "map", "args": [
52+
{"type": {"name": "string"}},
53+
{"type": {"name": "nullable", "args": [{"type": {"name": "uint8"}}]}}]}
54+
55+
{"name": "tuple", "args": [
56+
{"label": "lat", "type": {"name": "float64"}},
57+
{"label": "lon", "type": {"name": "float64"}}]}
58+
59+
{"name": "enum8", "args": [{"label": "active", "int": 1}, {"label": "deleted", "int": 2}]}
60+
61+
{"name": "datetime64", "args": [{"int": 3}, {"string": "UTC"}]}
62+
```
63+
64+
An identifier argument, such as the function in `AggregateFunction(uniq,
65+
String)`, is a type with no arguments. Resolving the names is left to
66+
whoever reads the output.
6167

6268
## Tests
6369

‎internal/engine/clickhouse/testgen/analyze.go‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,9 @@ type analyzedQuery struct {
2020
}
2121

2222
type analyzedColumn struct {
23-
Name string `json:"name"`
24-
Type string `json:"type"`
25-
NotNull bool `json:"not_null"`
26-
Table string `json:"table,omitempty"`
23+
Name string `json:"name"`
24+
Type *typeExpr `json:"type,omitempty"`
25+
Table string `json:"table,omitempty"`
2726
}
2827

2928
type analyzedParam struct {
@@ -118,8 +117,10 @@ func analyzeQuery(ctx context.Context, l local, schema, fixture string, q query)
118117
}
119118

120119
func column(name, typ string) analyzedColumn {
121-
expr, notNull := typeExpr(typ)
122-
return analyzedColumn{Name: name, Type: expr, NotNull: notNull}
120+
if typ == "" {
121+
return analyzedColumn{Name: name}
122+
}
123+
return analyzedColumn{Name: name, Type: parseType(typ)}
123124
}
124125

125126
// returnsRows reports whether a statement produces a result set and so can

‎internal/engine/clickhouse/testgen/testdata/analyze_params/stdout.txt‎

Lines changed: 21 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,16 @@
55
"columns": [
66
{
77
"name": "id",
8-
"type": "uint64",
9-
"not_null": true,
8+
"type": {
9+
"name": "uint64"
10+
},
1011
"table": "events"
1112
},
1213
{
1314
"name": "name",
14-
"type": "string",
15-
"not_null": true,
15+
"type": {
16+
"name": "string"
17+
},
1618
"table": "events"
1719
}
1820
],
@@ -21,8 +23,9 @@
2123
"number": 1,
2224
"column": {
2325
"name": "id",
24-
"type": "uint64",
25-
"not_null": true,
26+
"type": {
27+
"name": "uint64"
28+
},
2629
"table": "events"
2730
}
2831
}
@@ -34,14 +37,16 @@
3437
"columns": [
3538
{
3639
"name": "id",
37-
"type": "uint64",
38-
"not_null": true,
40+
"type": {
41+
"name": "uint64"
42+
},
3943
"table": "events"
4044
},
4145
{
4246
"name": "name",
43-
"type": "string",
44-
"not_null": true,
47+
"type": {
48+
"name": "string"
49+
},
4550
"table": "events"
4651
}
4752
],
@@ -50,17 +55,19 @@
5055
"number": 1,
5156
"column": {
5257
"name": "name",
53-
"type": "string",
54-
"not_null": true,
58+
"type": {
59+
"name": "string"
60+
},
5561
"table": "events"
5662
}
5763
},
5864
{
5965
"number": 2,
6066
"column": {
6167
"name": "amount",
62-
"type": "float64",
63-
"not_null": true,
68+
"type": {
69+
"name": "float64"
70+
},
6471
"table": "events"
6572
}
6673
}

‎internal/engine/clickhouse/testgen/testdata/exec/stdout.txt‎

Lines changed: 37 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -8,17 +8,19 @@
88
"number": 1,
99
"column": {
1010
"name": "id",
11-
"type": "uint64",
12-
"not_null": true,
11+
"type": {
12+
"name": "uint64"
13+
},
1314
"table": "users"
1415
}
1516
},
1617
{
1718
"number": 2,
1819
"column": {
1920
"name": "email",
20-
"type": "string",
21-
"not_null": true,
21+
"type": {
22+
"name": "string"
23+
},
2224
"table": "users"
2325
}
2426
}
@@ -33,35 +35,46 @@
3335
"number": 1,
3436
"column": {
3537
"name": "id",
36-
"type": "uint64",
37-
"not_null": true,
38+
"type": {
39+
"name": "uint64"
40+
},
3841
"table": "events"
3942
}
4043
},
4144
{
4245
"number": 2,
4346
"column": {
4447
"name": "name",
45-
"type": "string",
46-
"not_null": true,
48+
"type": {
49+
"name": "string"
50+
},
4751
"table": "events"
4852
}
4953
},
5054
{
5155
"number": 3,
5256
"column": {
5357
"name": "tag",
54-
"type": "string",
55-
"not_null": false,
58+
"type": {
59+
"name": "nullable",
60+
"args": [
61+
{
62+
"type": {
63+
"name": "string"
64+
}
65+
}
66+
]
67+
},
5668
"table": "events"
5769
}
5870
},
5971
{
6072
"number": 4,
6173
"column": {
6274
"name": "amount",
63-
"type": "float64",
64-
"not_null": true,
75+
"type": {
76+
"name": "float64"
77+
},
6578
"table": "events"
6679
}
6780
}
@@ -76,35 +89,39 @@
7689
"number": 1,
7790
"column": {
7891
"name": "id",
79-
"type": "uint64",
80-
"not_null": true,
92+
"type": {
93+
"name": "uint64"
94+
},
8195
"table": "events"
8296
}
8397
},
8498
{
8599
"number": 2,
86100
"column": {
87101
"name": "name",
88-
"type": "string",
89-
"not_null": true,
102+
"type": {
103+
"name": "string"
104+
},
90105
"table": "events"
91106
}
92107
},
93108
{
94109
"number": 3,
95110
"column": {
96111
"name": "id",
97-
"type": "uint64",
98-
"not_null": true,
112+
"type": {
113+
"name": "uint64"
114+
},
99115
"table": "events"
100116
}
101117
},
102118
{
103119
"number": 4,
104120
"column": {
105121
"name": "name",
106-
"type": "string",
107-
"not_null": true,
122+
"type": {
123+
"name": "string"
124+
},
108125
"table": "events"
109126
}
110127
}

0 commit comments

Comments
 (0)