Skip to content

Commit 8c81481

Browse files
committed
ast: tag every node with its type in JSON output
Node is an interface, so the JSON that parse and analyze --ast print carried no record of which node a given object was. Nodes with no fields of their own all encoded as "{}": a star in a RETURNING clause was indistinguishable from an untranslated clause, and an empty List was indistinguishable from both. Every node struct now declares a zero-sized marker as its first field: Tag NodeTag[T] `json:"tag"` with T the node's own type. encoding/json calls MarshalJSON on the field, and the generic type parameter carries the node's identity to it at compile time, so plain json.Marshal of any node emits its type name with no custom encoder walking the tree. The zero value is valid, so the engines' struct literals are unchanged, and the field is zero-sized, so nodes cost the same memory. The field's UnmarshalJSON accepts only the containing node's name, making a document decoded into the wrong node type an error instead of a silently misfielded tree. A test checks that every node struct declares the field, that its type parameter names the containing struct (the one mistake the compiler cannot catch, since a copy-pasted NodeTag[OtherNode] compiles), and that no node declares a colliding second field. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T9LCt1mwdY3mE14x3Zz5Vw
1 parent 2249e14 commit 8c81481

267 files changed

Lines changed: 769 additions & 8 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/howto/analyze.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,4 +106,6 @@ reports the result columns and parameters:
106106
]
107107
```
108108

109-
Pass `--ast` to also include each statement's parsed AST under an `ast` key.
109+
Pass `--ast` to also include each statement's parsed AST under an `ast` key. It
110+
has the same shape as the output of [`parse`](parse.md), with every node tagged
111+
by type.

‎docs/howto/parse.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ The output is a JSON array with one object per statement:
4848
"name": "GetAuthor",
4949
"cmd": ":one",
5050
"ast": {
51+
"tag": "RawStmt",
5152
"Stmt": {
5253
"...": "..."
5354
},
@@ -60,3 +61,29 @@ The output is a JSON array with one object per statement:
6061

6162
Statements without a `-- name:` annotation (for example schema DDL) omit the
6263
`name` and `cmd` fields.
64+
65+
## Node types
66+
67+
Every node in the AST carries a `tag` naming its type. Some nodes have no
68+
fields of their own, so without it a star, a null literal and an untranslated
69+
clause would all print as `{}`.
70+
71+
```json
72+
"Val": {
73+
"tag": "ColumnRef",
74+
"Name": "",
75+
"Fields": {
76+
"tag": "List",
77+
"Items": [
78+
{
79+
"tag": "A_Star"
80+
}
81+
]
82+
},
83+
"Location": 93
84+
}
85+
```
86+
87+
A `tag` of `TODO` marks a clause the dialect's converter does not translate
88+
yet. It means the clause was parsed but is not represented in the AST, not that
89+
the clause was absent from the query.

‎internal/endtoend/testdata/analyze_ast/postgresql/stdout.txt‎

Lines changed: 32 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,23 +24,32 @@
2424
}
2525
],
2626
"ast": {
27+
"tag": "RawStmt",
2728
"Stmt": {
29+
"tag": "SelectStmt",
2830
"DistinctClause": {
31+
"tag": "List",
2932
"Items": null
3033
},
3134
"IntoClause": null,
3235
"TargetList": {
36+
"tag": "List",
3337
"Items": [
3438
{
39+
"tag": "ResTarget",
3540
"Name": null,
3641
"Indirection": {
42+
"tag": "List",
3743
"Items": null
3844
},
3945
"Val": {
46+
"tag": "ColumnRef",
4047
"Name": "",
4148
"Fields": {
49+
"tag": "List",
4250
"Items": [
4351
{
52+
"tag": "String",
4453
"Str": "name"
4554
}
4655
]
@@ -52,8 +61,10 @@
5261
]
5362
},
5463
"FromClause": {
64+
"tag": "List",
5565
"Items": [
5666
{
67+
"tag": "RangeVar",
5768
"Catalogname": null,
5869
"Schemaname": null,
5970
"Relname": "authors",
@@ -65,48 +76,66 @@
6576
]
6677
},
6778
"WhereClause": {
79+
"tag": "A_Expr",
6880
"Kind": 1,
6981
"Name": {
82+
"tag": "List",
7083
"Items": [
7184
{
85+
"tag": "String",
7286
"Str": "="
7387
}
7488
]
7589
},
7690
"Lexpr": {
91+
"tag": "ColumnRef",
7792
"Name": "",
7893
"Fields": {
94+
"tag": "List",
7995
"Items": [
8096
{
97+
"tag": "String",
8198
"Str": "id"
8299
}
83100
]
84101
},
85102
"Location": 59
86103
},
87104
"Rexpr": {
105+
"tag": "ParamRef",
88106
"Number": 1,
89107
"Location": 64,
90108
"Dollar": true
91109
},
92110
"Location": 62
93111
},
94112
"GroupClause": {
113+
"tag": "List",
95114
"Items": null
96115
},
97-
"HavingClause": {},
116+
"HavingClause": {
117+
"tag": "TODO"
118+
},
98119
"WindowClause": {
120+
"tag": "List",
99121
"Items": null
100122
},
101123
"ValuesLists": {
124+
"tag": "List",
102125
"Items": null
103126
},
104127
"SortClause": {
128+
"tag": "List",
105129
"Items": null
106130
},
107-
"LimitOffset": {},
108-
"LimitCount": {},
131+
"LimitOffset": {
132+
"tag": "TODO"
133+
},
134+
"LimitCount": {
135+
"tag": "TODO"
136+
},
109137
"LockingClause": {
138+
"tag": "List",
110139
"Items": null
111140
},
112141
"WithClause": null,

‎internal/endtoend/testdata/parse_basic/clickhouse/stdout.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,22 @@
33
"name": "GetValue",
44
"cmd": ":one",
55
"ast": {
6+
"tag": "RawStmt",
67
"Stmt": {
8+
"tag": "SelectStmt",
79
"DistinctClause": null,
810
"IntoClause": null,
911
"TargetList": {
12+
"tag": "List",
1013
"Items": [
1114
{
15+
"tag": "ResTarget",
1216
"Name": null,
1317
"Indirection": null,
1418
"Val": {
19+
"tag": "A_Const",
1520
"Val": {
21+
"tag": "Integer",
1622
"Ival": 1
1723
},
1824
"Location": 31

‎internal/endtoend/testdata/parse_basic/duckdb/stdout.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,22 @@
33
"name": "GetValue",
44
"cmd": ":one",
55
"ast": {
6+
"tag": "RawStmt",
67
"Stmt": {
8+
"tag": "SelectStmt",
79
"DistinctClause": null,
810
"IntoClause": null,
911
"TargetList": {
12+
"tag": "List",
1013
"Items": [
1114
{
15+
"tag": "ResTarget",
1216
"Name": null,
1317
"Indirection": null,
1418
"Val": {
19+
"tag": "A_Const",
1520
"Val": {
21+
"tag": "Integer",
1622
"Ival": 1
1723
},
1824
"Location": 30

‎internal/endtoend/testdata/parse_basic/googlesql/stdout.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,22 @@
33
"name": "GetValue",
44
"cmd": ":one",
55
"ast": {
6+
"tag": "RawStmt",
67
"Stmt": {
8+
"tag": "SelectStmt",
79
"DistinctClause": null,
810
"IntoClause": null,
911
"TargetList": {
12+
"tag": "List",
1013
"Items": [
1114
{
15+
"tag": "ResTarget",
1216
"Name": null,
1317
"Indirection": null,
1418
"Val": {
19+
"tag": "A_Const",
1520
"Val": {
21+
"tag": "Integer",
1622
"Ival": 1
1723
},
1824
"Location": 30

‎internal/endtoend/testdata/parse_basic/mssql/stdout.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,22 @@
33
"name": "GetValue",
44
"cmd": ":one",
55
"ast": {
6+
"tag": "RawStmt",
67
"Stmt": {
8+
"tag": "SelectStmt",
79
"DistinctClause": null,
810
"IntoClause": null,
911
"TargetList": {
12+
"tag": "List",
1013
"Items": [
1114
{
15+
"tag": "ResTarget",
1216
"Name": null,
1317
"Indirection": null,
1418
"Val": {
19+
"tag": "A_Const",
1520
"Val": {
21+
"tag": "Integer",
1622
"Ival": 1
1723
},
1824
"Location": 30

‎internal/endtoend/testdata/parse_basic/mysql/stdout.txt‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,22 @@
33
"name": "GetValue",
44
"cmd": ":one",
55
"ast": {
6+
"tag": "RawStmt",
67
"Stmt": {
8+
"tag": "SelectStmt",
79
"DistinctClause": null,
810
"IntoClause": null,
911
"TargetList": {
12+
"tag": "List",
1013
"Items": [
1114
{
15+
"tag": "ResTarget",
1216
"Name": null,
1317
"Indirection": null,
1418
"Val": {
19+
"tag": "A_Const",
1520
"Val": {
21+
"tag": "Integer",
1622
"Ival": 1
1723
},
1824
"Location": 30
@@ -22,14 +28,17 @@
2228
]
2329
},
2430
"FromClause": {
31+
"tag": "List",
2532
"Items": null
2633
},
2734
"WhereClause": null,
2835
"GroupClause": {
36+
"tag": "List",
2937
"Items": null
3038
},
3139
"HavingClause": null,
3240
"WindowClause": {
41+
"tag": "List",
3342
"Items": []
3443
},
3544
"ValuesLists": null,

‎internal/endtoend/testdata/parse_basic/postgresql/stdout.txt‎

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,20 +3,28 @@
33
"name": "GetValue",
44
"cmd": ":one",
55
"ast": {
6+
"tag": "RawStmt",
67
"Stmt": {
8+
"tag": "SelectStmt",
79
"DistinctClause": {
10+
"tag": "List",
811
"Items": null
912
},
1013
"IntoClause": null,
1114
"TargetList": {
15+
"tag": "List",
1216
"Items": [
1317
{
18+
"tag": "ResTarget",
1419
"Name": null,
1520
"Indirection": {
21+
"tag": "List",
1622
"Items": null
1723
},
1824
"Val": {
25+
"tag": "A_Const",
1926
"Val": {
27+
"tag": "Integer",
2028
"Ival": 1
2129
},
2230
"Location": 30
@@ -26,25 +34,39 @@
2634
]
2735
},
2836
"FromClause": {
37+
"tag": "List",
2938
"Items": null
3039
},
31-
"WhereClause": {},
40+
"WhereClause": {
41+
"tag": "TODO"
42+
},
3243
"GroupClause": {
44+
"tag": "List",
3345
"Items": null
3446
},
35-
"HavingClause": {},
47+
"HavingClause": {
48+
"tag": "TODO"
49+
},
3650
"WindowClause": {
51+
"tag": "List",
3752
"Items": null
3853
},
3954
"ValuesLists": {
55+
"tag": "List",
4056
"Items": null
4157
},
4258
"SortClause": {
59+
"tag": "List",
4360
"Items": null
4461
},
45-
"LimitOffset": {},
46-
"LimitCount": {},
62+
"LimitOffset": {
63+
"tag": "TODO"
64+
},
65+
"LimitCount": {
66+
"tag": "TODO"
67+
},
4768
"LockingClause": {
69+
"tag": "List",
4870
"Items": null
4971
},
5072
"WithClause": null,

0 commit comments

Comments
 (0)