refactor(languages): step 2 — per-pack project readers

CLAUDE.md

@@ -75,12 +75,16 @@ IngestionPipeline (pipeline.py)
    (`parse(code, file_path) -> tuple[list[Node], list[Edge]]`; emit `CALLS` edges with
    `target="raw_call:<name>"`, set `Node.language = "<name>"`).
 2. In its `__init__.py`, define `PACK`, subclassing `cgis.core.extractor.LanguagePack` (mark methods `@override`)
-   (`name`, `extensions`, `code_fence`, `is_test`, `scope_of`, `extractor`).
+   (`name`, `extensions`, `code_fence`, `is_test`, `scope_of`, `extractor`, `project_reader`).
 3. Add one `Manifest` to `MANIFESTS` in `src/cgis/languages/__init__.py`, with a `load` function that
    imports the pack statically inside it (lazy, yet visible to the review-fingerprint walk). Nothing
    else names the pack: CLI, MCP server, guardian and scripts all go through the registry, and the
    `surface-packs` forbidden-edge gate fails if a surface imports a pack directly.
 4. A pack never imports another pack or `cgis.storage` (`cross-pack`, `pack-isolation` gates).
+5. A pack's `__init__` imports no grammar: import the extractor inside `extractor()`. Config reading
+   goes in a `ProjectReader` (`cgis.core.project`) whose facts are data keyed by `ingest_state`
+   key — the collector (`cgis.import_names`), not the pack, writes them. A key the collector does not
+   know (`_FACT_SHAPES`) is refused: a new fact needs store and resolver support first.
 
 ## Type Discipline
 

docs/plans/2026-10-14-language-packs-step1.md

@@ -23,6 +23,12 @@
 
 ## Rulings (where this plan departs from the spec's text, and why)
 
+> Superseded in places by the code that shipped (#597): `LanguagePack` lives in
+> `cgis.core.extractor`, manifests carry `load` functions instead of module paths,
+> packs subclass the protocol with `@override`, and (step 2) R7 gave way to light
+> pack modules that import their grammar inside `extractor()`. The spec's
+> implementation notes record each.
+
 - **R1 — Two "is test" rules, not one.** The spec says a pack's `is_test` replaces both `core.paths.is_test_path` and the pipeline's `_TEST_FILE_PATTERN`. They are different rules:
   - `_TEST_FILE_PATTERN` (`foo.test.ts`, `foo.spec.py`) makes the walk **skip** a file;
   - `is_test_path` (`tests/`, `test_*.py`) **flags** an ingested node.

docs/specs/2026-10-14-language-packs-design.md

@@ -100,6 +100,11 @@ Two planes over a shared core:
 - **Read plane.** `storage → query → surfaces`.
 - **Core.** Model types and FQN utilities. It imports nothing else from cgis.
 
+  > **Implementation note (steps 1–2):** the pack contract — `BaseExtractor`,
+  > `ModuleNamer`, `LanguagePack` (`cgis.core.extractor`), `ProjectReader` and
+  > `ProjectFacts` (`cgis.core.project`) — lives in core too. A protocol inside the
+  > registry formed a cycle with the packs that subclass it (#597).
+
 ### Registry
 
 `cgis/languages/__init__.py` holds one lightweight **manifest** per pack: its
@@ -123,6 +128,13 @@ The CLI, MCP server, health script and reviewer all use the registry; none
 names an extractor class. Surfaces and `query` may import
 `cgis.languages` (the registry) but not `cgis.languages.<pack>`.
 
+> **Implementation note (steps 1–2):** manifests load their pack through a
+> function holding a *static* import, never `importlib`, which the
+> review-fingerprint walk cannot follow (#597). Pack modules are imported up
+> front, because every project reader must see configs before the first file
+> of its language; what stays lazy is the grammar, which a pack imports only
+> inside `extractor()` (step 2, ruling S2).
+
 ### The `LanguagePack` contract
 
 ```python
@@ -154,6 +166,11 @@ class LanguagePack(Protocol):
 
 - `is_test` replaces the Python-only `is_test_path` and the pipeline's
   `_TEST_FILE_PATTERN` (pipeline.py:447). For Go it matches `_test.go`.
+
+  > **Implementation note (step 1):** those are two rules, not one — the walk's
+  > *skip* (`foo.test.ts` is never read) became `Manifest.never_ingested` and the
+  > registry's `skips()`; the node *flag* is the pack's `is_test`. Merging them
+  > would have dropped every `tests/` directory from the graph (#597, ruling R1).
 - `scope_of` returns the namespace unit a file's definitions live in:
   - Python and TS return today's module FQN for the file, so nothing changes;
   - Go returns the package directory, so `func (s *Server) Run()` in
@@ -182,6 +199,12 @@ class ProjectFacts(Protocol):                        # frozen, JSON-serialisable
     def differs_from(self, store: SQLiteStore) -> bool: ... # a change forces a rebuild
 ```
 
+> **Implementation note (step 2):** `record`/`differs_from` contradict "a pack
+> never touches storage" below, which the `pack-isolation` gate enforces. As
+> built, `ProjectFacts.state()` returns the facts keyed by their `ingest_state`
+> key, and the collector (`cgis.import_names.ImportNames`) records and compares
+> them through the store — same keys, no pack-to-storage edge (ruling S1).
+
 The Python pack's facts are today's dependency roots. The TS pack's are npm
 dependencies, workspace packages (package.json and pnpm) and tsconfig path
 aliases. `import_names.py` becomes the collector that asks each pack.

src/cgis/core/extractor.py

@@ -5,9 +5,12 @@
 """
 
 from abc import ABC, abstractmethod
+from collections.abc import Mapping
+from pathlib import Path
 from typing import Protocol, runtime_checkable
 
 from cgis.core.models import Edge, Node
+from cgis.core.project import ProjectReader
 
 
 @runtime_checkable
@@ -83,3 +86,7 @@ def scope_of(self, path: str, source_root: str | None) -> str:
     def extractor(self, extension: str, source_roots: list[str]) -> BaseExtractor:
         """A new extractor for files of `extension`."""
         ...
+
+    def project_reader(self, root: Path, extractors: Mapping[str, BaseExtractor]) -> ProjectReader:
+        """A reader for this pack's configs around `root` (`extractors`: the walk's own)."""
+        ...

src/cgis/core/project.py

@@ -0,0 +1,38 @@
+"""What a language pack reads from a project's configs (spec: Project facts).
+
+Data, not a writer: a pack never touches storage (the `pack-isolation` gate), so
+facts say *what* to store under which `ingest_state` key and the build plane's
+collector (`cgis.import_names`) records and compares them (step 2, ruling S1).
+"""
+
+from collections.abc import Mapping
+from pathlib import Path
+from typing import Protocol
+
+
+class ProjectFacts(Protocol):
+    """Frozen, JSON-serialisable facts one pack read from a project."""
+
+    def state(self) -> Mapping[str, object]:
+        """Each fact under the `ingest_state` key it is stored as."""
+        ...
+
+
+class ProjectReader(Protocol):
+    """Collects one pack's configs during a walk, and from the directories above it."""
+
+    def note(self, path: Path) -> bool:
+        """Take a file the walk passed, if it is one of this pack's configs."""
+        ...
+
+    def note_above(self, directories: list[Path], bounded: bool) -> None:
+        """Read configs above the ingest root, nearest first (#555).
+
+        `bounded` says a repository boundary ends the list; without one the list
+        runs to `/`, and only the nearest find should be trusted.
+        """
+        ...
+
+    def facts(self) -> ProjectFacts:
+        """What the walk's configs say, once it is done."""
+        ...

src/cgis/import_names.py

@@ -1,41 +1,36 @@
 """The names imports resolve through, collected from the configs around a walk.
 
-Four sources, read the same way and handed to the resolver together: workspace
-packages and declared npm dependencies from `package.json` (#504, #556) and
-`compilerOptions.paths` aliases from `tsconfig.json` (#508), which the walk passes
-or which sit above the ingest root (#555), and the dependencies a Python project
-declares (#495), read from the manifests at or above the ingest root.
-Kept out of `IngestionPipeline`, which only says which files it walked past and
-when the result must be stored.
+Each language pack reads its own (`cgis.core.project`); this module only routes
+the walk's files and the directories above the root to every pack's reader, and
+merges what they read (language-packs step 2). Kept out of `IngestionPipeline`,
+which only says which files it walked past and when the result must be stored.
 """
 
-from collections.abc import Mapping
+from collections.abc import Iterable, Mapping
 from dataclasses import dataclass
 from pathlib import Path
-from typing import TYPE_CHECKING
-
-import structlog
+from typing import TYPE_CHECKING, cast
 
 from cgis.core.extractor import BaseExtractor
-from cgis.npm_dependencies import NpmDependencies
-from cgis.python_dependencies import declared_import_roots
-from cgis.tsconfig_paths import TSCONFIG, TsconfigPaths
-from cgis.workspaces import (
-    PACKAGE_MANIFEST,
-    PNPM_WORKSPACE,
-    WorkspacePackages,
-    pnpm_member_names,
-    workspace_member_names,
-)
+from cgis.core.project import ProjectFacts
+from cgis.languages import project_readers
 
 if TYPE_CHECKING:
     from cgis.storage.sqlite_store import SQLiteStore
 
-logger = structlog.getLogger(__name__)
-
 #: What marks a repository boundary the upward search never crosses.
 _REPOSITORY_MARKER = ".git"
 
+#: The facts the collector can record, compare and hand the resolver, with the
+#: JSON shape each reads back as. A pack contributing any other key needs store
+#: and resolver support first; until then its facts would vanish silently.
+_FACT_SHAPES: dict[str, type] = {
+    "workspace_packages": dict,
+    "tsconfig_paths": dict,
+    "python_dependencies": list,
+    "npm_dependencies": list,
+}
+
 
 @dataclass(frozen=True)
 class ImportNames:
@@ -67,6 +62,29 @@ def differ_from(self, store: "SQLiteStore") -> bool:
             or (store.get_npm_dependencies() or []) != self.npm_dependencies
         )
 
+    @classmethod
+    def from_facts(cls, facts: Iterable[ProjectFacts]) -> "ImportNames":
+        """Merge every pack's facts; a pack that read nothing contributes empty values.
+
+        A key two packs both claim is an error, not a last-writer-wins: one pack's
+        facts would vanish from resolution without a trace.
+        """
+        state: dict[str, object] = {}
+        for fact in facts:
+            contribution = fact.state()
+            shared = state.keys() & contribution.keys()
+            if shared:
+                msg = f"two packs record the same ingest_state key(s): {sorted(shared)}"
+                raise ValueError(msg)
+            state.update(contribution)
+        _check_shapes(state)
+        return cls(
+            workspace_packages=cast("dict[str, str]", state.get("workspace_packages", {})),
+            path_aliases=cast("dict[str, dict[str, list[str]]]", state.get("tsconfig_paths", {})),
+            python_dependencies=cast("list[str]", state.get("python_dependencies", [])),
+            npm_dependencies=cast("list[str]", state.get("npm_dependencies", [])),
+        )
+
     def record(self, store: "SQLiteStore") -> None:
         """Record these names on the stored graph, for the next run's `differ_from`."""
         store.record_workspace_packages(self.workspace_packages)
@@ -75,6 +93,18 @@ def record(self, store: "SQLiteStore") -> None:
         store.record_npm_dependencies(self.npm_dependencies)
 
 
+def _check_shapes(state: Mapping[str, object]) -> None:
+    """Refuse a fact the collector cannot store, or one that would never read back equal."""
+    unknown = sorted(state.keys() - _FACT_SHAPES.keys())
+    if unknown:
+        msg = f"no collector support for ingest_state key(s): {unknown}"
+        raise ValueError(msg)
+    for key, value in state.items():
+        if not isinstance(value, _FACT_SHAPES[key]):
+            msg = f"{key} must be a {_FACT_SHAPES[key].__name__}, not {type(value).__name__}"
+            raise TypeError(msg)
+
+
 def _enclosing_directories(root: Path) -> tuple[list[Path], bool]:
     """The directories above `root`, nearest first, and whether a repository bounds them.
 
@@ -94,69 +124,26 @@ def _enclosing_directories(root: Path) -> tuple[list[Path], bool]:
 
 
 class ImportNameCollector:
-    """Takes the `package.json` and `tsconfig.json` files a walk passes."""
+    """Routes the configs a walk passes to every language pack's reader."""
 
     def __init__(self, workspace_root: Path, extractors: Mapping[str, BaseExtractor]) -> None:
-        """Collect for `workspace_root`, naming modules with the TypeScript extractor's rule.
+        """Collect for `workspace_root`; `extractors` reach the readers that name modules (S4).
 
         The root is resolved first: the search above it walks `parents`, which a
         relative path does not have past `.` (#560 review). The pipeline already
         passes a resolved root, so for it nothing changes.
         """
         self._root = workspace_root.resolve()
-        self._packages = WorkspacePackages(self._root, extractors)
-        self._tsconfigs = TsconfigPaths(self._root, extractors)
-        self._npm = NpmDependencies()
-        # Workspace packages named by a manifest above the root, outside the walk.
-        self._siblings: set[str] = set()
-        self._note_configs_above()
-
-    def _note_configs_above(self) -> None:
-        """Read the configs above the ingest root that its files still belong to (#555).
-
-        `cgis ingest app/src` never walks past `app/tsconfig.json` or
-        `app/package.json`, yet the compiler gives each file the nearest tsconfig
-        above it and Node finds packages in every `node_modules` up the tree. So
-        the nearest tsconfig above counts — unless the root has its own, which is
-        nearer — and every package.json above declares dependencies, less the
-        workspace packages a monorepo manifest there names. The search stops at
-        the repository boundary, as `python_dependencies` does (#495).
-        """
+        self._readers = project_readers(self._root, extractors)
         directories, bounded = _enclosing_directories(self._root)
-        manifests = [d / PACKAGE_MANIFEST for d in directories if (d / PACKAGE_MANIFEST).is_file()]
-        # Without a repository the walk up reaches `$HOME` or `/`: a stray manifest
-        # there would add packages, so only the nearest is trusted (#560 review).
-        for manifest in manifests if bounded else manifests[:1]:
-            self._npm.note(manifest)
-            self._siblings |= workspace_member_names(manifest)
-        # pnpm names its workspace in its own file, beside the root manifest (#562).
-        pnpm = [d / PNPM_WORKSPACE for d in directories if (d / PNPM_WORKSPACE).is_file()]
-        for workspace_file in pnpm if bounded else pnpm[:1]:
-            self._siblings |= pnpm_member_names(workspace_file)
-        if (self._root / TSCONFIG).is_file():
-            return
-        config = next((d / TSCONFIG for d in directories if (d / TSCONFIG).is_file()), None)
-        if config is not None:
-            logger.info("Reading tsconfig.json above the ingest root", path=str(config))
-            self._tsconfigs.note(config)
+        for reader in self._readers:
+            reader.note_above(directories, bounded)
 
     def note(self, path: Path) -> None:
-        """Hand a file the walk passed to whichever reader takes it; any other is ignored."""
-        if path.name == PACKAGE_MANIFEST:
-            self._packages.note(path)
-            self._npm.note(path)
-        elif path.name == TSCONFIG:
-            self._tsconfigs.note(path)
+        """Hand a file the walk passed to every reader; each takes only its own."""
+        for reader in self._readers:
+            reader.note(path)
 
     def collect(self) -> ImportNames:
-        """The names, once the walk is done.
-
-        Aliases are read last: a tsconfig can extend a shared config that lives in
-        a workspace package, which only the finished package map can locate.
-        """
-        return ImportNames(
-            workspace_packages=self._packages.unambiguous(),
-            path_aliases=self._tsconfigs.aliases(self._packages.directories()),
-            python_dependencies=sorted(declared_import_roots(self._root)),
-            npm_dependencies=self._npm.declared(excluding=self._packages.names() | self._siblings),
-        )
+        """The names, once the walk is done."""
+        return ImportNames.from_facts(reader.facts() for reader in self._readers)

src/cgis/languages/__init__.py

@@ -8,7 +8,8 @@
 Adding a language is a manifest here plus a package beside this file that
 exposes `PACK`. Each manifest loads its pack with a static import inside a
 function — lazy, yet visible to the review-fingerprint walk, which cannot follow
-`importlib` (`test_no_dynamic_imports_in_the_review_path`).
+`importlib` (`test_no_dynamic_imports_in_the_review_path`). A pack's own module
+imports no grammar; its `extractor()` does (step 2, ruling S2).
 
 Case-sensitive on purpose: `IngestionPipeline._get_extractor` matches
 `filename.endswith(ext)` against these lowercase extensions, so `Foo.TS` is
@@ -20,10 +21,11 @@
 from collections.abc import Callable, Iterator, Mapping
 from dataclasses import dataclass
 from functools import cache
-from pathlib import PurePosixPath
+from pathlib import Path, PurePosixPath
 
 from cgis.core.extractor import BaseExtractor, LanguagePack
 from cgis.core.models import Node
+from cgis.core.project import ProjectReader
 
 
 @dataclass(frozen=True)
@@ -111,6 +113,11 @@ def scope_of(path: str, source_root: str | None) -> str:
     return pack.scope_of(path, source_root)
 
 
+def project_readers(root: Path, extractors: Mapping[str, BaseExtractor]) -> list[ProjectReader]:
+    """Every pack's project reader, in manifest order — imports each pack, loads no grammar."""
+    return [_load(m.name).project_reader(root, extractors) for m in MANIFESTS]
+
+
 def skips(filename: str) -> bool:
     """Whether the walk never reads this file (`foo.test.ts`, `foo.spec.py`)."""
     return any(m.never_ingested.search(filename) for m in MANIFESTS)

src/cgis/languages/python/__init__.py

@@ -1,11 +1,14 @@
 """The Python language pack: tree-sitter-python, module FQNs from file paths."""
 
+from collections.abc import Mapping
+from pathlib import Path
 from typing import override
 
 from cgis.core.extractor import BaseExtractor, LanguagePack
 from cgis.core.paths import is_test_path
-from cgis.languages.python._syntax import PYTHON_LANG
-from cgis.languages.python.extractor import PythonExtractor, file_path_to_module_fqn
+from cgis.core.project import ProjectReader
+from cgis.languages.python._syntax import PYTHON_LANG, file_path_to_module_fqn
+from cgis.languages.python.project import PythonProjectReader
 
 
 class PythonPack(LanguagePack):
@@ -31,8 +34,15 @@ def scope_of(self, path: str, source_root: str | None) -> str:
 
     @override
     def extractor(self, extension: str, source_roots: list[str]) -> BaseExtractor:
-        """One grammar for the one extension."""
+        """One grammar for the one extension — loaded here, on first use."""
+        from cgis.languages.python.extractor import PythonExtractor  # noqa: PLC0415
+
         return PythonExtractor(source_roots=source_roots)
 
+    @override
+    def project_reader(self, root: Path, extractors: Mapping[str, BaseExtractor]) -> ProjectReader:
+        """This pack's config reader for the project at `root`."""
+        return PythonProjectReader(root)
+
 
 PACK = PythonPack()