Phase 5: Go, Rust, Python, JavaScript and Java

CLAUDE.md

@@ -7,8 +7,9 @@ contract is in `PRD.md` and the phase plan is in `PLAN.md`.
 
 An interactive visual map of a codebase. Clone a repo → tree-sitter extracts functions and call
 sites → resolve calls to definitions → store the graph in Postgres → explore it on a React Flow
-canvas (file → card → function mind-map → code block). Language: **TypeScript** through Phase 4;
-Go, Rust and Python are added in Phase 5, extraction only.
+canvas (file → card → function mind-map → code block). Languages: **TypeScript, TSX, JavaScript,
+JSX, Go, Rust, Python and Java**. Everything past the ECMAScript family is extraction plus
+same-file resolution only — see "Per-language extraction limits" in `docs/PARSING_STRATEGY.md`.
 
 The product is called **funcatlas** everywhere — repo, module path, npm scope, database, cookie.
 The old working name "CodeCanvas" is retired; do not reintroduce it.
@@ -21,7 +22,7 @@ The old working name "CodeCanvas" is retired; do not reintroduce it.
 - [x] Phase 3a — API and auth
 - [x] Phase 3b — Canvas and search
 - [x] Phase 4 — Webhooks, queue, hardening
-- [ ] Phase 5 — Go, Rust, Python (extraction only; per-language resolution stays cut) ← next
+- [x] Phase 5 — Go, Rust, Python, JavaScript, Java (extraction only; per-language resolution stays cut)
 
 Active task list: `TASKLIST.md`.
 
@@ -60,7 +61,8 @@ You implement, phase by phase, with tests. The user reviews at each phase gate.
 
 | Concern | Home |
 |---|---|
-| All parser constants — confidence tiers, node kinds, import kinds, limits | `services/parser/internal/utils/constants.go` |
+| All parser constants — language names, resolution groups, node kinds by language, confidence tiers, import kinds, limits | `services/parser/internal/utils/constants.go` |
+| One language's grammar, `.scm`, scope rules, receiver and imports | `services/parser/internal/extract/<language>.go`, registered in `spec.go` |
 | Tree-sitter node traversal | `services/parser/internal/utils/nodes.go` |
 | Repo-relative paths, module specifier resolution | `services/parser/internal/utils/paths.go` |
 | Qualified-name building and scope candidates | `services/parser/internal/utils/qualnames.go` |
@@ -91,8 +93,9 @@ You implement, phase by phase, with tests. The user reviews at each phase gate.
   lucide-react, Zustand + TanStack Query.
 - **API:** Fastify + Drizzle + postgres.js + Zod, arctic/oslo for GitHub OAuth, Redis sessions,
   `@fastify/rate-limit`.
-- **Parser:** Go + `tree-sitter/go-tree-sitter` v0.25.0 + `tree-sitter/tree-sitter-typescript`
-  v0.23.2 (both pinned in `go.mod`), pgx with explicit SQL, zap.
+- **Parser:** Go + `tree-sitter/go-tree-sitter` v0.25.0, plus one pinned grammar per language:
+  `tree-sitter-typescript` v0.23.2, `-javascript` v0.25.0, `-go` v0.25.0, `-rust` v0.24.2,
+  `-python` v0.25.0, `-java` v0.23.5. pgx with explicit SQL, zap.
 - **Database:** Postgres — edge tables and recursive CTEs. Neo4j deferred indefinitely.
 - **Queue:** Redis + BullMQ, consumed by a **Node worker that spawns the Go binary** (`pnpm worker`).
   The parser has no Redis dependency.
@@ -132,7 +135,17 @@ files over 1 MB skipped.
 - **One grammar per extension, never shared.** `.ts` uses `LanguageTypescript()`, `.tsx` uses
   `LanguageTSX()`. A mismatched grammar fails *silently*: the body becomes an `ERROR` node, the
   declaration still matches, and every call inside is dropped. Any new language needs a fixture that
-  pins the **calls** inside its hardest construct, not just the function names.
+  pins the **calls** inside its hardest construct, not just the function names. `tree-sitter-javascript`
+  is the exception that proves it: one grammar reads JSX in any file, so `.js` and `.jsx` share it.
+- **Adding a language is a `Spec` in `internal/extract/`, a `.scm` in `queries/`, and a fixture.**
+  The third is not optional. `internal/extract/spec_test.go` fails if a `.scm` is missing any of the
+  three captures, and the extension registry is driven off `registry` so a new language cannot leave
+  a test asserting last month's set.
+- **The resolver partitions by language group; it does not filter by it.** `byName` and `byPkgName`
+  are keyed on the group so there is no code path that can reach a foreign-language candidate at
+  all. `.ts`/`.tsx`/`.js`/`.jsx`/`.mjs`/`.cjs` share the one group with more than one language in
+  it. **Do not write a test that decides what to allow by calling `utils.ResolutionGroup`** — it
+  agrees with itself when broken. See R36.
 
 ## Known gaps
 
@@ -142,10 +155,10 @@ Phase 3b is closed. `TASKLIST.md` is the chunk-level truth; this is what outlive
   any change, so a `memo` keyed on reference re-rendered every card whenever one changed — and each
   card showing source re-ran its highlighted block, which the reader saw as untouched cards
   blinking. `sameNodeData` in `lib/graph.ts` is what both node memos use; keep it that way.
-- **Canvas state is persisted but never re-validated.** `store/ui.ts` restores the repository, file
-  and open branches through `zustand/persist`. A re-parse reinserts changed files' functions under
-  new ids, so a restored branch can point at rows that are gone — and a webhook now does that
-  without anyone touching the browser. See R34.
+- **Restored canvas state is checked late, not at rehydrate.** Nothing is loaded when
+  `zustand/persist` rehydrates, so `dropMissingRoots` runs when the open file's function list
+  arrives. That list is authoritative for branch roots only; an expanded id in another file is left
+  to its own query, which 404s. See R34.
 - **A webhook's HMAC covers the raw bytes, so the JSON parser is scoped, not global.** Fastify's
   default parser drops the text, and re-serialising `req.body` does not reproduce what GitHub sent —
   the digest then fails in a way that reads as a wrong secret. A test that signs *compact* JSON will
@@ -184,7 +197,11 @@ Phase 3b is closed. `TASKLIST.md` is the chunk-level truth; this is what outlive
   `node dist/index.js` cannot follow them. `pnpm dev` (tsx) works. Fix before containerising.
 - **Compiled test files land in `apps/api/dist`.** Harmless locally, wrong in an image.
 - **Resolution limits that are honest, not broken:** barrel re-export chains, default imports, and
-  `tsconfig` path aliases all resolve to `unresolved`. See `docs/PARSING_STRATEGY.md`.
+  `tsconfig` path aliases all resolve to `unresolved`. So does every per-language construct in
+  "Per-language extraction limits" — Go's single-type-argument generic call (ambiguous with a
+  conversion), anything inside a Rust macro (a `token_tree` is not parsed), and which Java overload
+  a call meant. Each is pinned by an assertion, because a parser that quietly produces *less* reads
+  as one that worked. See `docs/PARSING_STRATEGY.md`.
 
 ## Verify state before trusting this file
 
@@ -204,7 +221,7 @@ gh run list --branch $(git branch --show-current) --limit 2   # is CI green?
 Parser on its own, when the question is about extraction rather than the app:
 
 ```bash
-make go-run REPO=./services/parser/testdata/resolve
+make go-run REPO=./services/parser/testdata/polyglot
 cd services/parser && go run ./cmd/parser --repo ./testdata/resolve --format summary
 ```
 

DEVELOPMENT.md

@@ -55,7 +55,8 @@ on a reachable host was a session for the asking.
   /cmd/parser           entry point
   /internal/clone       local path or shallow git clone
   /internal/security    path containment, size and depth caps, symlink rejection
-  /internal/ts          tree-sitter extraction and the qualified-name scope walk
+  /internal/extract     tree-sitter extraction: the language-agnostic driver, the
+                        qualified-name scope walk, and one Spec per language
   /internal/ir          Go-native intermediate representation
   /internal/resolver    call resolution and confidence tagging
   /internal/db          Postgres writer

Makefile

@@ -106,7 +106,9 @@ parser-isolated: ## Run the parser in its container with no network at all (docs
 	  parser --repo /fixture --format summary --out -
 
 go-run: ## Run the parser against a local repo (usage: make go-run REPO=./path)
-	cd services/parser && go run ./cmd/parser --repo "$(REPO)"
+	# abspath, because the recipe cds into services/parser and REPO is written
+	# relative to the repo root -- which is where everything else in here is.
+	cd services/parser && go run ./cmd/parser --repo "$(abspath $(REPO))" --format summary
 
 clean: ## Clean up generated artifacts and caches
 	pnpm store prune

PLAN.md

@@ -38,9 +38,9 @@ Delivered:
 - `internal/clone` — local path, or `git clone --depth 1`; never runs the repo's install or build scripts.
 - `internal/security` — env-driven caps, `ContainsRoot` path containment, and a `Walk` that hard-fails
   on symlinks, sniffs for binary content, and enforces file-count, per-file-size, and depth limits.
-- `internal/ts` — tree-sitter TypeScript extraction: function declarations, methods, arrow and
+- `internal/ts` (now `internal/extract`) — tree-sitter TypeScript extraction: function declarations, methods, arrow and
   function expressions assigned to variables; call sites with their enclosing caller; imports.
-- `internal/ts/scope.go` — the dot-joined qualified-name walk (`Repo.sync`, `getUser.inner`).
+- `internal/ts/scope.go` (now `internal/extract/scope.go`) — the dot-joined qualified-name walk (`Repo.sync`, `getUser.inner`).
 - `internal/ir` — Go-native `File` / `Function` / `CallSite` / `Import` / `Graph`.
 - `queries/typescript.scm` — the query patterns, embedded at build time with `//go:embed`.
 - `Dockerfile` — multi-stage, non-root; `docker-compose.yml` runs the parser with
@@ -155,10 +155,11 @@ throttled; the parser still works with no network egress. — passed; see `TASKL
 
 **Known carry-over into Phase 5** — R34: `store/ui.ts` restores a file and its open branches through
 `zustand/persist`, and this is the first phase whose re-parse can delete the rows behind them.
+Closed in Phase 5.
 
 ---
 
-## Phase 5 — Go, Rust and Python · not started
+## Phase 5 — Go, Rust, Python, JavaScript and Java · done
 
 Extraction only. Each language gets functions, call sites and imports in the IR, and same-file
 resolution, which is language-agnostic. Cross-file calls resolve to `name_match` or `unresolved`,
@@ -167,7 +168,8 @@ never `exact`.
 **Why the split.** Extraction is a grammar, a `.scm`, and a set of node kinds — cheap and testable.
 Resolution is not, and it is the whole product. Go resolves through package clauses and
 capitalisation-based export, Rust through `mod`/`use`/crate paths and `impl` blocks, Python through
-`sys.path` and `__init__.py`. Sharing one resolver across them would emit confident wrong edges,
+`sys.path` and `__init__.py`, Java through the classpath and argument types. Sharing one resolver
+across them would emit confident wrong edges,
 which is worse than admitting ignorance — see [`PRD.md`](PRD.md#8-the-design-commitment). The
 confidence tiers already carry that admission to the user honestly.
 
@@ -180,13 +182,23 @@ parsed while three of four calls inside its JSX were dropped. One wrong grammar,
 edges gone, no error anywhere. Every language added multiplies that risk, which is why each one
 needs its own fixture pinning *calls*, not just functions.
 
-**Builds:** `tree-sitter-go`, `tree-sitter-rust`, `tree-sitter-python`; one `.scm` per language;
-per-language node-kind constants; `files.language` populated per file rather than hardcoded;
-extraction fixtures per language. The UI learns to show language per node.
+**Shipped:** `tree-sitter-javascript`, `-go`, `-rust`, `-python`, `-java`; one `.scm` per language;
+per-language node-kind constants; `files.language` populated per file rather than hardcoded, so
+`.tsx` now reports `tsx`; extraction fixtures per language. The UI badges a callee whose language
+differs from the file being read.
 
-**Exit test:** a polyglot fixture repo produces correct functions and call sites for all four
-languages, no edge crosses a language boundary, and `.tsx`-style silent grammar mismatch is
-impossible because every language's fixture pins the calls inside its hardest construct.
+JavaScript was added alongside the three, because the TypeScript spec already knew how to read it
+and `.js`/`.jsx` were being skipped entirely. It joins TypeScript's resolution group, so a `.js`
+file importing a `.ts` file still resolves `exact` -- the one cross-file case that survives.
+Java was added because it is the first language here with genuine overloads, which is what
+`overload_index` was always for.
+
+**Exit test:** `testdata/polyglot` produces functions *and* call sites for every language, no edge
+crosses a language boundary, and cross-file resolution is never `exact` outside the ECMAScript
+family. A `.tsx`-style silent mismatch is impossible because every language's fixture pins the
+calls inside its hardest construct -- and each of those turned out to be a real limit worth
+recording rather than a hypothetical: Go's single-type-argument generic call, Rust's macro bodies,
+Python's decorated definitions, Java's overloads.
 
 ---
 
@@ -198,7 +210,7 @@ Each of these was considered and deliberately deferred, not forgotten.
 |---|---|
 | LSP-based resolution | Accurate but slow to build and RPC-heavy on re-export chains. Name/scope resolution ships first, honestly tagged; LSP upgrades `name_match` to `exact` later. |
 | Freehand annotation layer | Pure nice-to-have. Zero bearing on whether the core graph is trustworthy. |
-| ~~Languages beyond TypeScript~~ | **Reinstated as Phase 5** (extraction only). Full per-language resolution is still cut. |
+| ~~Languages beyond TypeScript~~ | **Done in Phase 5** (extraction only). Full per-language resolution is still cut. |
 | Neo4j | Recursive CTEs over an edge table handle this scale. Revisit when traversal is measurably the bottleneck. |
 | Saved canvas layouts | Positions resetting on reload is a papercut, not a blocker. |
 | Multi-tenancy and RBAC | Single-user MVP. Each repo is already an isolated workspace, so this stays cheap to add. |