Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
c0a0ba0
feat(code): a tree-less Thrift IDL lane
rahulsjha Nov 17, 2026
3665dc7
test: give the Thrift fixtures a test-inventory workflow
rahulsjha Nov 17, 2026
0d527c9
docs: format the language-support table with prettier
rahulsjha Nov 18, 2026
26ab218
Merge remote-tracking branch 'origin/main' into feat/thrift-lane
rahulsjha Nov 19, 2026
e78d117
fix(code): read Thrift includes from the comment-aware tokens
rahulsjha Nov 19, 2026
3ee5f74
test: format the Thrift include test with ruff
rahulsjha Nov 19, 2026
56d86c5
fix(thrift): read field types when the numeric field ID is absent
rahulsjha Nov 20, 2026
7c4f783
Merge remote-tracking branch 'origin/main' into feat/thrift-lane
rahulsjha Nov 20, 2026
2402850
test(thrift): cover struct, argument and throws fields without a nume…
rahulsjha Nov 20, 2026
920ce3d
Move the Thrift fixture out of the plain-text cases
rahulsjha Nov 20, 2026
7a6e02a
Merge remote-tracking branch 'origin/main' into feat/thrift-lane
rahulsjha Nov 20, 2026
96be03c
Replicated erasure fixtures provide isolated write leases
zlareb1 Nov 20, 2026
4a77a5c
Thrift definitions join code search and traversal
zlareb1 Nov 20, 2026
44dd513
Replicated erasure fixtures provide their required lease table
zlareb1 Nov 20, 2026
abf0ebc
Thrift references survive root files and nested field syntax
zlareb1 Nov 20, 2026
27c51f0
Preserve current documentation in the Thrift composition
zlareb1 Nov 20, 2026
74f72ee
Thrift type metadata stays out of dependency sites
zlareb1 Nov 20, 2026
9e25140
Use the replicated-erasure fixture supplied by main
zlareb1 Nov 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/development/adding-language-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,13 @@ Every module under `server/code/domain/extract/` is either in a stage of `STAGE_

## 1. Choose the lane's shape

| Shape | Examples | You provide |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Grammar lane | `lang_typescript.py`, `lang_rust.py`, `lang_go.py`, `lang_java.py` | A tree-sitter grammar dependency in `pyproject.toml`; symbols and call sites from the tree; chunks and edges come from the shared chunker and edge walker. |
| Tree-less lane | `lang_proto.py`, `lang_gotemplate.py`, `lang_hcl.py`, `lang_k8s.py`, `lang_jsonnet.py`, `lang_lua.py`, `lang_graphql.py` (`LANE_UNIT_LANGS`) | A hand-written scanner; `ParsedFile.tree` stays `None`; chunks from `lane_units.span_chunks`, edges from `reference_sites` + `SITE_RULES` bound by `lane_units.bind_sites`. |
| Include-only lane | `tla`, `flatbuffers`, `cmake`, `doc` (`_LANE_INCLUDE_LANGS`) | Symbols and includes resolved by the simple lane resolver. |
| Embedded language | SQL inside Go and Java (`go_sql_units`, `java_sql_units`), shell islands in Dockerfiles and Makefiles | Units joined into the host file's parse (`_join_island_symbols`), usually behind a kill switch setting. |
| Chunk-only | `.css`, text files (`lang_text.py`) | Nothing but detection: the file is chunked and searchable with no symbols. |
| Shape | Examples | You provide |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Grammar lane | `lang_typescript.py`, `lang_rust.py`, `lang_go.py`, `lang_java.py` | A tree-sitter grammar dependency in `pyproject.toml`; symbols and call sites from the tree; chunks and edges come from the shared chunker and edge walker. |
| Tree-less lane | `lang_proto.py`, `lang_thrift.py`, `lang_gotemplate.py`, `lang_hcl.py`, `lang_k8s.py`, `lang_jsonnet.py`, `lang_lua.py`, `lang_graphql.py` (`LANE_UNIT_LANGS`) | A hand-written scanner; `ParsedFile.tree` stays `None`; chunks from `lane_units.span_chunks`, edges from `reference_sites` + `SITE_RULES` bound by `lane_units.bind_sites`. |
| Include-only lane | `tla`, `flatbuffers`, `cmake`, `doc` (`_LANE_INCLUDE_LANGS`) | Symbols and includes resolved by the simple lane resolver. |
| Embedded language | SQL inside Go and Java (`go_sql_units`, `java_sql_units`), shell islands in Dockerfiles and Makefiles | Units joined into the host file's parse (`_join_island_symbols`), usually behind a kill switch setting. |
| Chunk-only | `.css`, text files (`lang_text.py`) | Nothing but detection: the file is chunked and searchable with no symbols. |

A grammar lane's grammar is **not** in `STAMPED_GRAMMARS`, so upgrading it moves no stamp. The lane owns re-deriving its rows.

Expand Down
1 change: 1 addition & 0 deletions docs/features/code-index/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ still gets file-level impact.
| Shell (`.sh`, `.bash`, `.bats`, shebang, rc files) | yes | calls (commands), `source` includes | — |
| Prisma (`.prisma`) | yes | model relations | — |
| Protocol Buffers (`.proto`), FlatBuffers (`.fbs`) | yes | imports, type uses | — |
| Thrift IDL (`.thrift`) | yes | includes, type uses, service `extends` | — |
| Go templates and Helm charts (`.tpl`, `.tmpl`, `.gotmpl`, templated manifests) | yes | template calls | — |
| Terraform / HCL (`.tf`, `.tfvars`, `.hcl`), Kubernetes manifests | yes | references (`reads`) | — |
| Jsonnet (`.jsonnet`, `.libsonnet`) | yes | calls, imports | — |
Expand Down
2 changes: 1 addition & 1 deletion docs/use-cases/code-intelligence-by-language.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ files ([switches](../features/code-index/overview.md#language-support)).
## Infrastructure as code

Terraform and HCL, Kubernetes manifests, Helm charts and Go templates, Jsonnet, CMake,
Protocol Buffers and shell scripts are indexed with their references, includes and calls,
Protocol Buffers, Thrift IDL and shell scripts are indexed with their references, includes and calls,
so an agent follows a Helm `include` to its `define`, or a Terraform reference to what it
names. Dockerfile, Makefile, Jenkinsfile and CI YAML jobs become symbols with
`LOOM_CODE_HOST_ISLANDS` on, after a backfill of those files when the namespace is already
Expand Down
13 changes: 12 additions & 1 deletion server/code/app/index/ingest_parse.py
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,12 @@ def _parse_java(path: str, source: bytes) -> ParsedFile:
return pf


def _parse_thrift(path: str, source: bytes) -> ParsedFile:
from server.code.domain.extract.lang_thrift import parse_thrift

return parse_thrift(path, source)


def _parse_proto(path: str, source: bytes) -> ParsedFile:
from server.code.domain.extract.lang_proto import parse_proto

Expand Down Expand Up @@ -404,6 +410,7 @@ def _parse_doc(path: str, source: bytes) -> ParsedFile:
"astro": _parse_astro,
"angular_template": _parse_angular_template,
"proto": _parse_proto,
"thrift": _parse_thrift,
"gotemplate": _parse_gotemplate,
"hcl": _parse_hcl,
"k8s": _parse_k8s,
Expand All @@ -423,7 +430,7 @@ def _parse_doc(path: str, source: bytes) -> ParsedFile:
# The lanes without a tree-sitter grammar — chunks from their symbol spans, edges from
# their reference sites (`lane_units`)
LANE_UNIT_LANGS: frozenset[str] = frozenset(
{"proto", "gotemplate", "hcl", "k8s", "jsonnet", "lua", "graphql"}
{"proto", "thrift", "gotemplate", "hcl", "k8s", "jsonnet", "lua", "graphql"}
)


Expand All @@ -445,6 +452,10 @@ def _lane_module(lang: str) -> Any:
from server.code.domain.extract import lang_lua

return lang_lua
if lang == "thrift":
from server.code.domain.extract import lang_thrift

return lang_thrift
if lang == "graphql":
from server.code.domain.extract import lang_graphql

Expand Down
1 change: 1 addition & 0 deletions server/code/domain/extract/lang_detect.py
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ def detect_shell(path: str, source: bytes) -> str:
".rego": "config",
".mdc": "doc",
".proto": "proto",
".thrift": "thrift",
Comment thread
zlareb1 marked this conversation as resolved.
Resolved
".graphql": "graphql",
".gql": "graphql",
".tpl": "gotemplate",
Expand Down
1 change: 0 additions & 1 deletion server/code/domain/extract/lang_text.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@
".purs",
".re",
".hx",
".thrift",
".avsc",
".capnp",
".asm",
Expand Down
Loading