[Python] tsg-parser fails to extract generic base class relations

[maxfischer2781]

Working with typed base classes in Python (see #22291) I realised that the toolchain seems to drop generic base classes (e.g. list[int] in class Bar(list[int]):) at some point. That is, in the example code

class Foo(list): pass

class Bar(list[int]): pass

CodeQL knows the Foo -> list relation but not the Bar -> list[int] relation.

---

As far as I could trace it, the relation is already lost in the tsg-python parser. Using the v2.26.2 toolchain, the source code class Bar(list): pass produces the following tsg-python output (assignment nodes removed for brevity):

node 4
  _kind: "Name"
  _location: [0, 10, 0, 14]
  ctx: "load"
  variable: "list"
node 5
  _kind: "ClassExpr"
  _location: [0, 0, 0, 21]
  _location_end: [0, 16]
  inner_scope: [graph node 6]
  name: "Bar"
edge 5 -> 4
  bases: 0
node 6
  _kind: "Class"
  _location: [0, 0, 0, 21]
  _location_end: [0, 16]
  name: "Bar"
edge 6 -> 1
  body: 0

Now if the code is extended for a generic subscription as class Bar(list[int]): pass produces the following tsg-python output (assignment nodes removed for brevity):

node 4
  _kind: "Name"
  _location: [0, 10, 0, 14]
  ctx: "load"
  variable: "list"
node 5
  _kind: "Name"
  _location: [0, 15, 0, 18]
  ctx: "load"
  variable: "int"
node 6
  _kind: "Subscript"
  _location: [0, 10, 0, 19]
  ctx: "load"
  index: [graph node 5]
  value: [graph node 4]
node 7
  _kind: "ClassExpr"
  _location: [0, 0, 0, 26]
  _location_end: [0, 21]
  inner_scope: [graph node 8]
  name: "Bar"
node 8
  _kind: "Class"
  _location: [0, 0, 0, 26]
  _location_end: [0, 21]
  name: "Bar"
edge 8 -> 1
  body: 0

Notice that the type subscription is parsed (nodes 4, 5, 6) but the edge n -> m node for the bases: 0 relation is absent.

[maxfischer2781]

It seems that the issue is here in the Python tsg definition for base class nodes:

codeql/python/extractor/tsg-python/python.tsg

Lines 1285 to 1294 in c914268

	; Class.bases - using `(_ !value !name)` as a proxy for all non-keyword arguments.
	; In particular, `keyword_argument` nodes have a `name` field, and `dictionary_splat`
	; nodes have a `value` field.
	(class_definition
	superclasses: (argument_list element: (_ !value !name) @arg)
	) @class
	{
	edge @class.class_expr -> @arg.node
	attr (@class.class_expr -> @arg.node) bases = (named-child-index @arg)
	}

While !value excludes dictionary_splat nodes, it also excludes subscript nodes as they also have a value field:

codeql/python/extractor/tsg-python/python.tsg

Lines 2612 to 2618 in c914268

	;;;;;; Subscript (`a[b]`)
	(subscript
	value: (_) @value
	subscript: (_) @index
	) @subscript
	{

[maxfischer2781]

Syntactically, the parenthesised expressions after a class name are the same as function arguments. As such, the parser should use the same rules as for positional function arguments:

codeql/python/extractor/tsg-python/python.tsg

Lines 698 to 707 in c914268

	; Handle non-keyword arguments
	(call arguments: (argument_list element: (_) @arg)) @call
	{
	if (not (or
	(instance-of @arg "keyword_argument")
	(instance-of @arg "dictionary_splat"))) {
	edge @call.node -> @arg.node
	attr (@call.node -> @arg.node) positional_args = (named-child-index @arg)
	}
	}

If I manage to compile the entire toolchain I will provide a PR. Otherwise, it would be nice if someone else could do so: generic bases are a staple of Python code written for static code analysis, i.e. what CodeQL is made for.