From e0637a8b434ce45ac6cde2cbb51f0cfbef28bb26 Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 13:37:36 +0000 Subject: [PATCH 01/10] =?UTF-8?q?docs(plan):=20language-packs=20step=204?= =?UTF-8?q?=20=E2=80=94=20resolution=20rules=20per=20pack?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- docs/plans/2026-10-14-language-packs-step4.md | 681 ++++++++++++++++++ 1 file changed, 681 insertions(+) create mode 100644 docs/plans/2026-10-14-language-packs-step4.md diff --git a/docs/plans/2026-10-14-language-packs-step4.md b/docs/plans/2026-10-14-language-packs-step4.md new file mode 100644 index 00000000..6c147018 --- /dev/null +++ b/docs/plans/2026-10-14-language-packs-step4.md @@ -0,0 +1,681 @@ +# Language Packs — Step 4: Resolution Rules — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Every language branch of the resolver becomes a rule that a pack contributes. After this step `cgis.resolver` names no language, every stored graph stays byte-identical, and the `engine-packs` allowlist is empty. + +**Architecture:** +- `cgis.core.resolution` defines the contract: + - `ResolutionView`: what a rule may read. `SymbolIndex` satisfies it structurally, so packs never import the resolver. + - `ResolutionRules`: type normalizers, positioned steps, a fallback, and an import reconciler. +- `LanguagePack.resolution()` returns a pack's rules. The registry answers `rules_for(source_file)` and `type_normalizers()`. +- The engine keeps the generic chain and calls the source file's rules at fixed points: + - local type, with the normalizers applied; + - import map; + - `after:import_map` steps; + - global symbols; + - fallback. +- **TypeScript contributes:** + - `reconcile_import` (tsconfig aliases, workspace packages, `.d` declarations); + - the `after:import_map` step `_package_call`; + - the JS-builtins fallback at confidence 0.8. + + Its resolution facts (`workspace_packages`, `path_aliases`, `file_bare_bindings`, `file_shadowed_globals`) move into `TypeScriptIndexFacts`. +- **Python contributes:** + - `reconcile_import` (layout-prefix stripping); + - the `Annotated`-alias type normalizer. + + Its star-import expansion moves into `PythonIndexFacts.reexports`. +- `cross_file_metadata` moves into the manifests. The pipeline unions them. + +**Tech Stack:** Python 3.12, tree-sitter, SQLite, pytest, ruff, mypy strict, interrogate. + +**Spec:** `docs/specs/2026-10-14-language-packs-design.md` (rev 3), sections "Resolution", "Incremental ingest" and Migration row 4. Outline: `docs/plans/2026-10-14-language-packs.md` "Step 4". Steps 1–3: #597, #599, #600 and the spec's implementation notes. + +## Global Constraints + +- `make format && make lint && make type-check && make pytest && make doc-coverage` passes before every commit. Gate commands with `&&`, never with `;`, a bare newline after a heredoc, or a pipe on the gating command. +- mypy strict on `src/`. Docstring coverage ≥ 90%. Sonar cognitive complexity ≤ 15 per function. +- **Behaviour unchanged:** + - `tests/fixtures/graph_identity/snapshot.json` is not modified. + - The main-vs-branch `graph_identity diff` reads `identical` on all three trees: cgis `src` @715cd9a, owner-api @8de2245, and `ui/`. +- The resolver's behaviour tests keep every assertion. They may change only where they reach into a moved private (`_through_annotated_alias`, `js_builtin_target`, `SymbolIndex.workspace_packages`, …). They construct `ResolverEngine(..., workspace_packages=..., tsconfig_paths=..., dependency_roots=..., npm_dependencies=...)`, and that signature does not change. +- **One language branch per commit** (spec, step 4). After each, the snapshot test and the full suite pass. +- **Ingest time:** the owner-api median of three, branch vs main, must stay within +5% (spec acceptance). Over budget: **stop and report**. +- **Drift:** `resolution` stands at 0.386 against a lock of 0.40, and unresolved at 0.442 against 0.45. The `symbols.py` DISPATCHES comment records that a module split once pushed it past tolerance. Measure after Tasks 4, 5 and 6. Over either bound: **stop and report**. Never raise a tolerance. +- `SymbolResolver` is at the god-object threshold, so new helpers are module-level, never methods on it. +- Test-count floor: raise `.github/test-count-baseline` (now 3095) if the suite passes 3245. +- **Disk:** the root disk is ~97% full. Keep scratch databases under the scratchpad (`/tmp`, tmpfs) and delete them after each measurement. +- Commits end with `Co-Authored-By: Claude Opus 5.5 `. The PR body ends with `🤖 Generated with [Claude Code](https://claude.com/claude-code)`. +- Worktree `/home/zaebee/projects/ownima/cgb-step4`, branch `feat/language-packs-step4`. + +## Rulings (where this plan departs from the spec's or outline's text) + +- **D1 — `layout_prefixes`, `reexports` and `suffix_map` stay on `SymbolIndex`, filled from pack facts.** + - Step 3's C3 said these resolution facts would leave `SymbolIndex`, but `map_to_node_fqn`/`resolve_layout`/`follow_reexport` read them for every language today, and step 3's C4 kept those generic. + - What moves to the packs is the *computation*: Python's star expansion becomes `PythonIndexFacts.reexports`. + - The TS-only facts do leave, because only TS rules read them: `workspace_packages`, `path_aliases`, `file_bare_bindings` and `file_shadowed_globals`. +- **D2 — the engine applies every pack's type normalizers, in manifest order, not only the source pack's.** + - `receiver_type_fqn` has no source file. + - The only normalizer, Python's `Annotated` alias, is a no-op for a type that is not a Python DI alias. + - *Cost if wrong:* a language whose normalizer rewrites another language's type. That is not possible with today's one normalizer. +- **D3 — positions are validated.** `ResolutionRules.steps` accepts only the positions the engine runs (`after:import_map`). An unknown position raises at pack load, so a typo cannot silently drop a rule. +- **D4 — a rule receives a `ResolutionView` (a core protocol), never `SymbolIndex`.** This keeps `cgis.languages` → `cgis.resolver` edges at zero; the `engine-packs` gate already forbids the reverse direction. + +## Review Focus + +- **A TS file importing through a tsconfig alias, a workspace package and a `.d.ts`.** These resolve exactly as before. Covered by the existing `test_resolver_ts_workspaces.py` / `test_resolver_ts_declarations.py` / `test_tsconfig_paths.py` and the `ui/` identity diff. +- **A Python reference with no source file (`receiver_type_fqn`).** It still gets the `Annotated` normalizer (Task 6, `test_a_normalizer_applies_without_a_source`). +- **A call in a TS file to a global the file shadows (`const confirm = …`).** It stays unresolved, never `js_builtins.confirm` (existing `test_resolver_js_builtins.py`). +- **An unknown step position in a pack's rules** fails at load (Task 2, `test_an_unknown_step_position_is_refused`). +- **Incremental ingest.** The cross-file signature is identical to before for every node type (Task 1, `test_cross_file_metadata_is_the_old_table`). + +--- + +### Task 1: `cross_file_metadata` per manifest + +**Files:** +- Modify: `src/cgis/languages/__init__.py` (`Manifest.cross_file_metadata` and `cross_file_metadata()`) +- Modify: `src/cgis/pipeline.py:32-37` (replace `_CROSS_FILE_METADATA`) +- Test: `tests/unit/test_languages.py` (append) + +**Interfaces:** +- Produces: + - `Manifest.cross_file_metadata: Mapping[NodeType, tuple[str, ...]]` + - `cgis.languages.cross_file_metadata() -> dict[NodeType, tuple[str, ...]]` + +- [ ] **Step 1: Write the failing test.** Append to `tests/unit/test_languages.py`: + +```python +from cgis.languages import cross_file_metadata + + +def test_cross_file_metadata_is_the_old_table() -> None: + """The pipeline's incremental signature reads exactly what it read before step 4.""" + assert cross_file_metadata() == { + NodeType.FILE: ("import_map", "reexports", "star_imports"), + NodeType.CLASS: ("self_types",), + NodeType.VARIABLE: ("annotated_type",), + } + + +def test_cross_file_metadata_imports_no_pack() -> None: + """Spec: declared in the manifest, so collecting it imports nothing.""" + script = ( + "import sys\n" + "from cgis.languages import cross_file_metadata\n" + "cross_file_metadata()\n" + "print(sorted(m for m in sys.modules if m.startswith('cgis.languages.')))\n" + ) + out = subprocess.run([sys.executable, "-c", script], capture_output=True, text=True, check=False) + assert out.returncode == 0, out.stderr + assert out.stdout.strip() == "[]" +``` + +Move the import to the import block. + +- [ ] **Step 2: Run it and watch it fail.** + Run: `uv run pytest tests/unit/test_languages.py -q -k cross_file` + Expected: `ImportError: cannot import name 'cross_file_metadata'`. + +- [ ] **Step 3: Implement.** + 1. In `src/cgis/languages/__init__.py`, import `NodeType` from `cgis.core.models`, and `field` from `dataclasses`. + 2. Add a field to `Manifest`: + +```python + #: Per node type, the metadata keys other files resolve against; a change to + #: one forces a rebuild of an incremental run. Declared here so collecting the + #: union imports no pack (spec, Incremental ingest). + cross_file_metadata: Mapping[NodeType, tuple[str, ...]] = field(default_factory=dict) +``` + + 3. Give the Python manifest the three entries from `pipeline._CROSS_FILE_METADATA`, verbatim, comment included. The TypeScript manifest declares none. + 4. Add: + +```python +def cross_file_metadata() -> dict[NodeType, tuple[str, ...]]: + """The union over every manifest, keys in manifest order; imports no pack.""" + merged: dict[NodeType, tuple[str, ...]] = {} + for manifest in MANIFESTS: + for node_type, keys in manifest.cross_file_metadata.items(): + merged[node_type] = merged.get(node_type, ()) + tuple( + k for k in keys if k not in merged.get(node_type, ()) + ) + return merged +``` + + 5. In `src/cgis/pipeline.py`, delete `_CROSS_FILE_METADATA`. At its single use, write `cross_file_metadata().get(node.type, ())`. Bind `_CROSS_FILE_KEYS = cross_file_metadata()` once at module level, next to the old constant's place, and use `_CROSS_FILE_KEYS.get(...)`. Import from `cgis.languages`. + +- [ ] **Step 4: Run.** + Run: `uv run pytest tests/unit/test_languages.py tests/unit/test_pipeline.py tests/unit/test_pipeline_incremental_symbols.py -q` + Expected: all pass. + +- [ ] **Step 5: Gate and commit.** `refactor(languages): cross_file_metadata is declared per manifest`. + +--- + +### Task 2: The resolution contract + +**Files:** +- Create: `src/cgis/core/resolution.py`, `tests/unit/test_resolution_rules.py` +- Modify: + - `src/cgis/core/extractor.py` (`LanguagePack.resolution`) + - both pack `__init__.py` (return empty rules for now) + - `src/cgis/languages/__init__.py` (`rules_for`, `type_normalizers`) + - `src/cgis/resolver/indices.py` (rename `_strip_layout_prefixes` → `strip_layout_prefixes`, so `SymbolIndex` satisfies `ResolutionView`) + +**Interfaces:** +- Produces: + - **`cgis.core.resolution`:** + - `ResolutionView` (Protocol): `nodes`, `facts`, `map_to_node_fqn(fqn) -> str | None`, `strip_layout_prefixes(fqn) -> str | None`, `is_variable_node(fqn) -> bool` + - `TypeNormalizer = Callable[[str, ResolutionView], str]` + - `CallStep = Callable[[str, str, ResolutionView], str | None]`, taking `(name, file_path, view)` + - `Fallback = Callable[[str, str, ResolutionView], tuple[str, float] | None]`, taking `(name, file_path, view)` + - `ImportReconciler = Callable[[str, str | None, ResolutionView], str | None]`, taking `(fqn, source_file, view)` + - `AFTER_IMPORT_MAP = "after:import_map"` + - `ResolutionRules` (frozen dataclass): `type_normalizers: tuple[TypeNormalizer, ...] = ()`, `steps: tuple[tuple[str, CallStep], ...] = ()`, `fallback: Fallback | None = None`, `reconcile_import: ImportReconciler | None = None`. It validates `steps` in `__post_init__`. + - **`cgis.languages`:** + - `rules_for(source_file: str | None) -> ResolutionRules | None`, meaning `classifying_pack(source_file)`'s rules, or None + - `type_normalizers() -> tuple[TypeNormalizer, ...]`, every pack's, in manifest order + +- [ ] **Step 1: Write the failing tests.** Create `tests/unit/test_resolution_rules.py`: + +```python +"""The resolution contract between the engine and the language packs (step 4).""" + +import pytest + +from cgis.core.resolution import AFTER_IMPORT_MAP, ResolutionRules, ResolutionView +from cgis.languages import rules_for, type_normalizers +from cgis.resolver.indices import IndexBuilder + + +def _step(name: str, file_path: str, view: ResolutionView) -> str | None: + return None + + +def test_an_unknown_step_position_is_refused() -> None: + """A typo would otherwise drop the rule silently.""" + with pytest.raises(ValueError, match="after:imports"): + ResolutionRules(steps=(("after:imports", _step),)) + + +def test_the_known_position_is_accepted() -> None: + assert ResolutionRules(steps=((AFTER_IMPORT_MAP, _step),)).steps[0][0] == AFTER_IMPORT_MAP + + +def test_the_index_is_a_resolution_view() -> None: + """Rules read the index through the core protocol, never by importing the resolver.""" + view: ResolutionView = IndexBuilder().build([]) + assert view.strip_layout_prefixes("x.y") is None + + +def test_rules_follow_the_classifying_pack() -> None: + """No source is Python's (the preserved quirk); a source no pack reads has none.""" + assert rules_for(None) is not None + assert rules_for("a.ts") is not None + assert rules_for("a.js") is None + + +def test_normalizers_come_in_manifest_order() -> None: + assert isinstance(type_normalizers(), tuple) +``` + +- [ ] **Step 2: Run them and watch them fail.** + Run: `uv run pytest tests/unit/test_resolution_rules.py -q` + Expected: `ModuleNotFoundError: cgis.core.resolution`. + +- [ ] **Step 3: The contract.** Create `src/cgis/core/resolution.py`: + +```python +"""What a language pack contributes to resolution (spec: Resolution). + +The engine runs a fixed chain — local type (with type normalizers), import map, +steps positioned `after:import_map`, global symbols, fallback — and asks the +source file's pack at each of those points. A rule reads the index through +`ResolutionView`, never the resolver itself, so `cgis.languages` stays clear of +`cgis.resolver` (language-packs step 4). +""" + +from collections.abc import Callable, Mapping +from dataclasses import dataclass +from typing import Protocol + +from cgis.core.index import IndexFacts +from cgis.core.models import Node + +#: The only position the engine runs pack steps at today: after the import map, +#: before the global symbol index (TS `_package_call`). +AFTER_IMPORT_MAP = "after:import_map" +_POSITIONS = frozenset({AFTER_IMPORT_MAP}) + + +class ResolutionView(Protocol): + """What a resolution rule may read; `SymbolIndex` satisfies it.""" + + @property + def nodes(self) -> Mapping[str, Node]: + """Every node, by id.""" + ... + + @property + def facts(self) -> Mapping[str, IndexFacts]: + """Each pack's index facts, by pack name.""" + ... + + def map_to_node_fqn(self, imported_fqn: str) -> str | None: + """An imported FQN reconciled to a node id, or None.""" + ... + + def strip_layout_prefixes(self, fqn: str) -> str | None: + """The node `fqn` names once corroborated layout prefixes are dropped, or None.""" + ... + + def is_variable_node(self, fqn: str) -> bool: + """Whether `fqn` is a VARIABLE node.""" + ... + + +TypeNormalizer = Callable[[str, ResolutionView], str] +CallStep = Callable[[str, str, ResolutionView], str | None] +Fallback = Callable[[str, str, ResolutionView], tuple[str, float] | None] +ImportReconciler = Callable[[str, str | None, ResolutionView], str | None] + + +@dataclass(frozen=True) +class ResolutionRules: + """One pack's contribution to the resolver's chain.""" + + #: Applied to a declared type before its methods are looked up (e.g. Annotated alias → T). + type_normalizers: tuple[TypeNormalizer, ...] = () + #: (position, step): a call step run at that point of the chain. + steps: tuple[tuple[str, CallStep], ...] = () + #: When nothing resolved a call: (target, confidence), or None. + fallback: Fallback | None = None + #: An import target with no node of that name: the node it means, or None. + reconcile_import: ImportReconciler | None = None + + def __post_init__(self) -> None: + """Refuse a step at a position the engine never runs.""" + unknown = sorted({position for position, _ in self.steps} - _POSITIONS) + if unknown: + msg = f"unknown resolution step position(s): {unknown}" + raise ValueError(msg) +``` + + - In `cgis.core.extractor.LanguagePack`, add `def resolution(self) -> ResolutionRules: ...`, with the docstring `"""This pack's resolution rules (spec: Resolution)."""`. + - Implement it in both packs with `@override`, returning `ResolutionRules()` for now. Tasks 3–6 fill them in. + +- [ ] **Step 4: The registry and the view.** + 1. In `src/cgis/resolver/indices.py`, rename `_strip_layout_prefixes` to `strip_layout_prefixes` (definition plus its two call sites) and drop "private" from its docstring if it says so. + 2. In `src/cgis/languages/__init__.py`: + +```python +@cache +def _rules(name: str) -> ResolutionRules: + return _load(name).resolution() + + +def rules_for(source_file: str | None) -> ResolutionRules | None: + """The resolution rules of the pack that classifies `source_file`, or None.""" + pack = classifying_pack(source_file) + return None if pack is None else _rules(pack.name) + + +def type_normalizers() -> tuple[TypeNormalizer, ...]: + """Every pack's type normalizers, in manifest order (step 4, ruling D2).""" + return tuple(n for m in MANIFESTS for n in _rules(m.name).type_normalizers) +``` + +- [ ] **Step 5: Run.** + Run: `uv run pytest tests/unit/test_resolution_rules.py tests/unit/test_resolver_indices.py -q && uv run mypy src` + Expected: all pass, and mypy accepts `IndexBuilder().build([])` as a `ResolutionView`. + +- [ ] **Step 6: Gate and commit.** `feat(languages): the resolution contract (ResolutionView, ResolutionRules)`. + +--- + +### Task 3: TypeScript import reconciliation (and Python's) + +**Files:** +- Modify: + - `src/cgis/languages/typescript/index.py` (facts gain `workspace_packages`, `path_aliases`) + - new `src/cgis/languages/typescript/resolution.py` + - TS pack `__init__.py` + - new `src/cgis/languages/python/resolution.py` + - Python pack `__init__.py` + - `src/cgis/resolver/indices.py` (`resolve_import_target`; delete `PathAlias`, `_resolve_typescript_import`, `_workspace_candidates`, `_alias_candidates`, `_ordered_aliases`, `_DECLARATION_SUFFIX`, `_ALIAS_WILDCARD`, `_PYTHON_SUFFIXES`, `_TYPESCRIPT_SUFFIXES` and the `workspace_packages`/`path_aliases` fields; the builder's `state` gains `workspace_packages` and `tsconfig_paths`) +- Test: `tests/unit/test_typescript_index.py`, `tests/unit/test_resolution_rules.py` (append) + +- [ ] **Step 1: Write the failing tests.** Append to `tests/unit/test_resolution_rules.py`: + +```python +from cgis.core.models import Node, NodeType + + +def _module(fqn: str, path: str) -> Node: + return Node(id=fqn, type=NodeType.FILE, name=path.rsplit("/", 1)[-1], file_path=path, + start_line=1, end_line=1, language="typescript") # fmt: skip + + +def test_typescript_reconciles_through_its_own_pack() -> None: + """Aliases and workspace packages are the TS pack's facts; the index no longer carries them.""" + nodes = [_module("packages.lib.util", "packages/lib/util.ts")] + index = IndexBuilder().build(nodes, workspace_packages={"@acme.lib": "packages.lib"}) + assert not hasattr(index, "workspace_packages") + assert index.resolve_import_target("@acme.lib.util", "apps/web/a.ts") == "packages.lib.util" + assert index.resolve_import_target("@acme.lib.util", "apps/web/a.py") is None + + +def test_python_reconciles_by_layout_prefixes() -> None: + rules = rules_for("app/x.py") + assert rules is not None and rules.reconcile_import is not None +``` + + Also append to `tests/unit/test_typescript_index.py` a test that `index_facts(..., {"workspace_packages": {"a": "x", "a.b": "y"}, "npm_dependencies": []}).workspace_packages` lists `"a.b"` before `"a"` (longest first), and that `path_aliases` holds `_ordered_aliases` output for a `{"apps/web": {"@/*": ["apps.web.src.*"]}}` input. + +- [ ] **Step 2: Run them and watch them fail.** + Run: `uv run pytest tests/unit/test_resolution_rules.py tests/unit/test_typescript_index.py -q` + Expected: `test_typescript_reconciles_through_its_own_pack` fails on `hasattr`; the facts tests fail on a missing attribute. + +- [ ] **Step 3: Move the TS half.** + 1. Into `languages/typescript/index.py`, move verbatim, with docstrings, `PathAlias`, `_ordered_aliases` and `_ALIAS_WILDCARD`. + 2. Add to `TypeScriptIndexFacts`: + - `workspace_packages: Mapping[str, str]` (longest name first, `MappingProxyType` of the sorted dict, exactly as the builder built it); + - `path_aliases: Mapping[str, tuple[PathAlias, ...]]`. + 3. In `index_facts`, read `state.get("workspace_packages", {})` and `state.get("tsconfig_paths", {})`, building them exactly as `IndexBuilder.build` did. + 4. Create `languages/typescript/resolution.py`: + - move `_resolve_typescript_import`, `_workspace_candidates`, `_alias_candidates` and `_DECLARATION_SUFFIX` verbatim; + - add: + +```python +def reconcile_import(fqn: str, source_file: str | None, view: ResolutionView) -> str | None: + """A TS import's node through tsconfig aliases, then workspace packages (#504, #507, #508).""" + facts = cast("TypeScriptIndexFacts", view.facts["typescript"]) + aliased = _alias_candidates(fqn, source_file or "", facts.path_aliases) + return _resolve_typescript_import(fqn, view.nodes, facts.workspace_packages, aliased) + + +RULES = ResolutionRules(reconcile_import=reconcile_import) +``` + + Check `_alias_candidates`' signature at `indices.py:368` before writing the call. Keep the argument order it has. + 5. In the TS pack, `resolution()` returns `resolution.RULES`. + 6. Create `languages/python/resolution.py` with + +```python +def reconcile_import(fqn: str, source_file: str | None, view: ResolutionView) -> str | None: + """A Python import's node once corroborated layout prefixes are stripped (#319, #494).""" + return view.strip_layout_prefixes(fqn) + + +RULES = ResolutionRules(reconcile_import=reconcile_import) +``` + + The Python pack returns it. + +- [ ] **Step 4: The engine side.** In `SymbolIndex.resolve_import_target`, keep the docstring and replace the body after `if fqn in self.nodes: return fqn` with: + +```python + rules = rules_for(source_file) + if rules is None or rules.reconcile_import is None: + return None + return rules.reconcile_import(fqn, source_file, self) +``` + + - This preserves today's three branches: a TS suffix goes to the TS rule; a non-`.py` suffix goes to `None`; `.py` or no source goes to stripping (`rules_for(None)` is Python's). + - In `IndexBuilder.build`, add `"workspace_packages": dict(workspace_packages or {})` and `"tsconfig_paths": dict(tsconfig_paths or {})` to `state`, and delete the two fields and the deleted helpers listed under **Files**. + +- [ ] **Step 5: Run.** + Run: `uv run pytest tests/unit/test_resolution_rules.py tests/unit/test_typescript_index.py tests/unit/test_resolver_ts_workspaces.py tests/unit/test_resolver_ts_declarations.py tests/unit/test_tsconfig_paths.py tests/unit/test_configs_above_root.py tests/unit/test_resolver_indices.py tests/unit/test_graph_identity.py -q` + Expected: all pass. + +- [ ] **Step 6: Gate and commit.** `refactor(languages): import reconciliation is each pack's rule`. + +--- + +### Task 4: TypeScript `_package_call` becomes the TS pack's positioned step + +**Files:** +- Modify: + - `src/cgis/languages/typescript/index.py` (facts gain `file_bare_bindings`) + - `src/cgis/languages/typescript/resolution.py` (the step) + - `src/cgis/resolver/symbols.py` (run positioned steps; delete `_package_call` and the TS import) + - `src/cgis/resolver/indices.py` (delete the `file_bare_bindings` field and its accumulation in `_index_metadata`) + - `tests/self_parsing/forbidden_edges_allowlist.json` (`engine-packs` → `{}`) + +- [ ] **Step 1: Write the failing test.** Append to `tests/unit/test_resolution_rules.py`: + +```python +from cgis.core.resolution import AFTER_IMPORT_MAP + + +def test_package_call_is_the_typescript_packs_step() -> None: + rules = rules_for("app/x.ts") + assert rules is not None + assert [position for position, _ in rules.steps] == [AFTER_IMPORT_MAP] +``` + +- [ ] **Step 2: Run it and watch it fail.** + Run: `uv run pytest tests/unit/test_resolution_rules.py -q -k package_call` + Expected: FAIL. The rules have no steps yet. + +- [ ] **Step 3: Move the rule.** + 1. Add `file_bare_bindings: Mapping[str, dict[str, str]]` to `TypeScriptIndexFacts`. Build it in `index_facts` under the same condition as `file_bare_imports`, keyed by normalized path: `dict(n.metadata.get("bare_bindings") or {})` when `bare_imports` is truthy. + 2. In `languages/typescript/resolution.py`, move `_package_call`'s body from `symbols.py`, with its docstring: + +```python +def package_call(name: str, file_path: str, view: ResolutionView) -> str | None: + """""" + facts = cast("TypeScriptIndexFacts", view.facts["typescript"]) + bindings = facts.file_bare_bindings.get(file_path) + # ... the rest of the old body, with `index.` reads replaced by `facts.` ... + if package_namespace(candidate, file_path, facts) is None: + return None + return candidate +``` + + Then `RULES = ResolutionRules(steps=((AFTER_IMPORT_MAP, package_call),), reconcile_import=reconcile_import)`. + 3. In `symbols.py`, add a module-level helper (not a method: `SymbolResolver` is at the god-object threshold): + +```python +def _positioned(index: SymbolIndex, position: str, name: str, file_path: str) -> str | None: + """The first hit among the source pack's steps at `position` (spec: Resolution).""" + rules = rules_for(file_path) + if rules is None: + return None + for step_position, step in rules.steps: + if step_position == position: + found = step(name, file_path, index) + if found: + return found + return None +``` + + 4. In `resolve_global_call`, replace `or _package_call(self.index, name, file_path)` with `or _positioned(self.index, AFTER_IMPORT_MAP, name, file_path)`. + 5. Delete `_package_call` and the `cgis.languages.typescript.index` import. Import `rules_for` from `cgis.languages` (the exempt registry) and `AFTER_IMPORT_MAP` from `cgis.core.resolution`. + 6. Set `"engine-packs": {}` in the allowlist. + +- [ ] **Step 4: Run.** + Run: `uv run pytest tests/unit/test_resolution_rules.py tests/unit/test_resolver.py tests/unit/test_resolver_language_scope.py tests/self_parsing/test_architecture.py tests/unit/test_graph_identity.py -q` + Expected: all pass, including `test_forbidden_edges_match_the_allowlist_exactly` with `engine-packs` empty. + +- [ ] **Step 5: Measure `resolution` drift** with the commands from step 3's plan, Task 4 Step 1. Stop if it exceeds the bounds. Record the result in the ledger. + +- [ ] **Step 6: Gate and commit.** `refactor(languages): _package_call is the TypeScript pack's after:import_map step`. + +--- + +### Task 5: The JS-builtins fallback becomes the TS pack's + +**Files:** +- Move: `src/cgis/resolver/js_builtins.py` → `src/cgis/languages/typescript/builtins.py` (with `git mv`; drop `is_js_source`, since the pack dispatch replaces it) +- Modify: + - `src/cgis/languages/typescript/index.py` (facts gain `file_shadowed_globals`) + - `src/cgis/languages/typescript/resolution.py` (fallback) + - `src/cgis/resolver/engine.py` (call `rules_for(file_path).fallback`) + - `src/cgis/resolver/indices.py` (delete the `file_shadowed_globals` field and its build) + - importers of `js_builtins` in tests + +- [ ] **Step 1: Write the failing test.** Append: + +```python +def test_js_builtins_are_the_typescript_packs_fallback() -> None: + """Spec: the fallback carries its confidence — 0.8 for a JS runtime global, not 1.0.""" + ts, py = rules_for("app/x.ts"), rules_for("app/x.py") + assert ts is not None and ts.fallback is not None + assert py is not None and py.fallback is None + view = IndexBuilder().build([]) + assert ts.fallback("Math.max", "app/x.ts", view) == ("js_builtins.Math.max", 0.8) +``` + +- [ ] **Step 2: Run it and watch it fail.** + Run: `uv run pytest tests/unit/test_resolution_rules.py -q -k fallback` + Expected: FAIL, because `fallback is None`. + +- [ ] **Step 3: Move it.** + 1. Run `git mv` on the module. Keep `js_builtin_target`, with the `is_js_source` guard removed because only the TS pack calls it now. Keep its docstring and add `(Only the TypeScript pack calls this, so the source is already JS.)`. + 2. Add `file_shadowed_globals: Mapping[str, frozenset[str]]` to the TS facts. Build it exactly as `IndexBuilder.build` did: FILE nodes whose `metadata["shadowed_globals"]` is truthy, keyed by normalized path. + 3. In `resolution.py`: + +```python +_FALLBACK_CONFIDENCE = 0.8 + + +def fallback(name: str, file_path: str, view: ResolutionView) -> tuple[str, float] | None: + """A call on a JS runtime global the file does not shadow: `js_builtins.` at 0.8 (#111).""" + facts = cast("TypeScriptIndexFacts", view.facts["typescript"]) + target = js_builtin_target(name, file_path, facts.file_shadowed_globals.get(file_path, frozenset())) + return None if target is None else (target, _FALLBACK_CONFIDENCE) +``` + + Add `fallback=fallback` to `RULES`. + 4. In `engine._resolved_call_edge`, replace the shadowed or `js_builtin_target` block with: + +```python + file_path = self._index.normalized_file_path(edge.source, edge.file_path) + rules = rules_for(file_path) if file_path else None + hit = rules.fallback(raw_name, file_path, self._index) if rules and rules.fallback else None + if hit is not None: + return edge.model_copy(update={"target": hit[0], "confidence": hit[1]}) +``` + + Today a `None` file path rewrote nothing (`is_js_source(None)` is False), and the `if file_path` guard keeps that. + 5. Re-point test imports with `git grep -n "resolver.js_builtins\|is_js_source" -- tests src`. + 6. `JS_BUILTINS_ROOT` and `JS_GLOBALS` stay in `cgis.core.js_globals`. + +- [ ] **Step 4: Run.** + Run: `uv run pytest tests/unit/test_resolver_js_builtins.py tests/self_parsing/test_self_parse_ts.py tests/unit/test_resolution_rules.py tests/unit/test_graph_identity.py -q` + Expected: all pass. + +- [ ] **Step 5: Measure drift** as in Task 4 Step 5. + +- [ ] **Step 6: Gate and commit.** `refactor(languages): the JS-builtins fallback is the TypeScript pack's`. + +--- + +### Task 6: Python's type normalizer and star-import expansion + +**Files:** +- Modify: + - `src/cgis/languages/python/resolution.py` (normalizer) + - `src/cgis/languages/python/index.py` (facts gain `reexports`; move `_public_names_under`, `_close_over_sources` and `_expand_star_imports` from `IndexBuilder`) + - `src/cgis/core/index.py` (`IndexFacts.reexports`) + - `src/cgis/languages/typescript/index.py` (empty `reexports`) + - `src/cgis/resolver/symbols.py` (`_through_annotated_alias` → registry normalizers) + - `src/cgis/resolver/indices.py` (builder merges facts' `reexports`; delete the star code and the `reexports`/`star_imports` accumulation in `_index_metadata`) + +- [ ] **Step 1: Write the failing tests.** Append: + +```python +def test_the_annotated_alias_is_pythons_normalizer() -> None: + rules = rules_for("app/x.py") + assert rules is not None + assert len(rules.type_normalizers) == 1 + assert type_normalizers() == rules.type_normalizers + + +def test_a_normalizer_applies_without_a_source() -> None: + """receiver_type_fqn has no source file: ServiceDep still reads as Service (#550, D2).""" + alias = Node(id="app.deps.ServiceDep", type=NodeType.VARIABLE, name="ServiceDep", + file_path="app/deps.py", start_line=1, end_line=1, + metadata={"annotated_type": "app.svc.Service"}) # fmt: skip + service = Node(id="app.svc.Service", type=NodeType.CLASS, name="Service", + file_path="app/svc.py", start_line=1, end_line=1) # fmt: skip + index = IndexBuilder().build([alias, service]) + (normalize,) = type_normalizers() + assert normalize("app.deps.ServiceDep", index) == "app.svc.Service" +``` + + Add to `tests/unit/test_python_index.py` a test that `index_facts` turns a FILE with `metadata={"star_imports": ["pkg.mod"]}` over nodes `pkg.mod.A` and `pkg.mod._b` into `reexports == {: {"A": "pkg.mod.A"}}`. That is the `_expand_star_imports` contract. + +- [ ] **Step 2: Run them and watch them fail.** + Run: `uv run pytest tests/unit/test_resolution_rules.py tests/unit/test_python_index.py -q` + Expected: FAIL. There are no normalizers, and the facts have no `reexports`. + +- [ ] **Step 3: Move the normalizer.** + 1. Move `_through_annotated_alias` from `symbols.py` to `languages/python/resolution.py` as `annotated_alias(type_fqn, view)`, docstring included. Replace `index.` with `view.`. + 2. Set `RULES = ResolutionRules(type_normalizers=(annotated_alias,), reconcile_import=reconcile_import)`. + 3. In `symbols.py`, add a module-level helper: + +```python +def _normalized(index: SymbolIndex, type_fqn: str) -> str: + """`type_fqn` through every pack's type normalizers, in manifest order (D2).""" + for normalize in type_normalizers(): + type_fqn = normalize(type_fqn, index) + return type_fqn +``` + + 4. Replace both `_through_annotated_alias(self.index, x)` / `(index, x)` call sites with `_normalized(index, x)`. + +- [ ] **Step 4: Move the star expansion.** + 1. Add `reexports: Mapping[str, Mapping[str, str]]` as a property to `IndexFacts`. + 2. TS facts get `reexports=MappingProxyType({})`. + 3. In `languages/python/index.py`, move `_public_names_under`, `_close_over_sources` and `_expand_star_imports` from `IndexBuilder` as module functions. In `index_facts`, collect `reexports`/`star_imports` from FILE metadata exactly as `_index_metadata` did, run the expansion, and store the result. + 4. In `IndexBuilder.build`, set `reexports` to the merge of every pack's `reexports` (a key two packs both produce is a `ValueError`, as in step 2's `from_facts`), and delete the moved code. + +- [ ] **Step 5: Run.** + Run: `uv run pytest tests/unit/test_resolution_rules.py tests/unit/test_python_index.py tests/unit/test_resolver.py tests/unit/test_resolver_indices.py tests/self_parsing tests/unit/test_graph_identity.py -q` + Expected: all pass. + +- [ ] **Step 6: Measure drift** as in Task 4 Step 5. + +- [ ] **Step 7: Gate and commit.** `refactor(languages): Python's Annotated normalizer and star expansion are the Python pack's`. + +--- + +### Task 7: Measure, record, document + +- [ ] **Step 1: Ingest timing.** Run owner-api, median of three, main vs branch, using the Python `perf_counter` wrapper from step 3, Task 4 (there is no `/usr/bin/time` on this machine). Expected: within +5%. Delete the databases afterwards. +- [ ] **Step 2: Inventory.** Regenerate it. Expected: nothing gone, and the new rule modules added. +- [ ] **Step 3: Grep for leftovers.** + Run: `git grep -nE "\.ts\b|\.tsx\b|\.py\b|typescript|python|js_builtins" -- src/cgis/resolver` + Expected: no language rule remains. Docstrings that name a language as history are fine. A suffix check or pack name in code is a finding: fix it or ledger it. +- [ ] **Step 4: Docs.** + - Spec "Resolution": add an `> **Implementation note (step 4):**` covering: + - D1 (generic `layout_prefixes`/`reexports`/`suffix_map` stay, filled by packs); + - D2 (normalizers from every pack); + - D3 (validated positions); + - D4 (`ResolutionView`); + - the empty `engine-packs` allowlist; + - the timing. + - CLAUDE.md "Adding a Language Pack": add `resolution()` (rules: normalizers, positioned steps, fallback with confidence, `reconcile_import`) and `cross_file_metadata` in the manifest. +- [ ] **Step 5: Gate and commit.** `docs: step 4 implementation notes; inventory`, with the measurements in the body. + +--- + +### Task 8: Identity report, review, PR + +- [ ] **Step 1: Identity.** Same as step 3's plan, Task 5 Step 1, with `step4` paths, `BR=/home/zaebee/projects/ownima/cgb-step4`, and scratch under the scratchpad. + - Expected: `identical` three times. + - Then delete the identity databases. +- [ ] **Step 2: Independent review.** Dispatch a reviewer with execution tools on the most capable model. Give it the branch diff, this plan, the spec, the ledger rulings and the Review Focus. Ask for an old-vs-new probe of `resolve()` output over the same captured nodes and edges, as in step 3's review, and warn it about the disk. Fix Critical and Important findings test-first, then rerun the gate and identity. +- [ ] **Step 3: PR.** Push, then open `refactor(languages): step 4 — resolution rules per pack`. The body carries: + - the module map; + - rulings D1–D4; + - three identity results; + - timing and drift; + - the `engine-packs` allowlist emptied; + - deferred minors. + + Then check the Gemini and Guardian comments and Sonar, and reply in every thread. From c6d4ee8ff85b9cf27287094b499be020efd6a6c7 Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 13:41:37 +0000 Subject: [PATCH 02/10] refactor(languages): cross_file_metadata is declared per manifest MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pipeline's incremental signature reads the union over the manifests; the Python manifest carries today's table, TypeScript none, so the union is the old table exactly — and collecting it imports no pack. Co-Authored-By: Claude Opus 5.5 --- src/cgis/languages/__init__.py | 24 ++++++++++++++++++++++-- src/cgis/pipeline.py | 12 ++++-------- tests/unit/test_languages.py | 25 +++++++++++++++++++++++++ 3 files changed, 51 insertions(+), 10 deletions(-) diff --git a/src/cgis/languages/__init__.py b/src/cgis/languages/__init__.py index 82ec4d6c..892aa23b 100644 --- a/src/cgis/languages/__init__.py +++ b/src/cgis/languages/__init__.py @@ -19,13 +19,13 @@ import re from collections.abc import Callable, Iterator, Mapping -from dataclasses import dataclass +from dataclasses import dataclass, field from functools import cache, lru_cache from pathlib import Path, PurePosixPath from cgis.core.extractor import BaseExtractor, LanguagePack from cgis.core.index import IndexFacts -from cgis.core.models import Node +from cgis.core.models import Node, NodeType from cgis.core.project import ProjectReader @@ -37,6 +37,10 @@ class Manifest: load: Callable[[], LanguagePack] #: imports the pack; called once, on first use extensions: tuple[str, ...] #: matched against the path suffix, dot included never_ingested: re.Pattern[str] #: this language's test files the walk skips outright + #: Per node type, the metadata keys other files resolve against; a change to + #: one forces a rebuild of an incremental run. Declared here so collecting the + #: union imports no pack (spec, Incremental ingest). + cross_file_metadata: Mapping[NodeType, tuple[str, ...]] = field(default_factory=dict) def _python() -> LanguagePack: @@ -57,6 +61,12 @@ def _typescript() -> LanguagePack: load=_python, extensions=(".py",), never_ingested=re.compile(r"\.(test|spec)\.py$", re.IGNORECASE), + cross_file_metadata={ + NodeType.FILE: ("import_map", "reexports", "star_imports"), + NodeType.CLASS: ("self_types",), + # Another file's parameter typed with the alias reads the type behind it (#550). + NodeType.VARIABLE: ("annotated_type",), + }, ), Manifest( name="typescript", @@ -136,6 +146,16 @@ def classifying_pack(source_file: str | None) -> LanguagePack | None: return _load("python") if source_file is None else pack_for_path(source_file) +def cross_file_metadata() -> dict[NodeType, tuple[str, ...]]: + """The union over every manifest, keys in manifest order; imports no pack.""" + merged: dict[NodeType, tuple[str, ...]] = {} + for manifest in MANIFESTS: + for node_type, keys in manifest.cross_file_metadata.items(): + known = merged.get(node_type, ()) + merged[node_type] = known + tuple(k for k in keys if k not in known) + return merged + + def skips(filename: str) -> bool: """Whether the walk never reads this file (`foo.test.ts`, `foo.spec.py`).""" return any(m.never_ingested.search(filename) for m in MANIFESTS) diff --git a/src/cgis/pipeline.py b/src/cgis/pipeline.py index 22b6639e..2064be45 100644 --- a/src/cgis/pipeline.py +++ b/src/cgis/pipeline.py @@ -16,7 +16,7 @@ from cgis.core.models import Edge, EdgeType, Node, NodeType from cgis.core.paths import EXCLUDED_DIRS from cgis.import_names import ImportNameCollector -from cgis.languages import skips +from cgis.languages import cross_file_metadata, skips from cgis.resolver.engine import ResolverEngine from cgis.resolver.symbols import dispatch_edges from cgis.resolver.uplift import SemanticUpliftEngine @@ -29,12 +29,8 @@ #: The node metadata another file's resolution reads, by node type. Everything #: else a file carries (`local_types`, `shadowed_globals`, decorators) only #: steers the resolution of that file's own edges, which are re-resolved anyway. -_CROSS_FILE_METADATA: dict[NodeType, tuple[str, ...]] = { - NodeType.FILE: ("import_map", "reexports", "star_imports"), - NodeType.CLASS: ("self_types",), - # Another file's parameter typed with the alias reads the type behind it (#550). - NodeType.VARIABLE: ("annotated_type",), -} +#: Per node type, the metadata other files resolve against — every manifest's, unioned. +_CROSS_FILE_KEYS = cross_file_metadata() def refresh_dispatch(store: "SQLiteStore") -> None: @@ -64,7 +60,7 @@ def _resolution_signature(nodes: list[Node], edges: list[Edge]) -> frozenset[str """ signature: set[str] = set() for node in nodes: - keys = _CROSS_FILE_METADATA.get(node.type, ()) + keys = _CROSS_FILE_KEYS.get(node.type, ()) meta = {key: node.metadata.get(key) for key in keys} signature.add(json.dumps([node.id, node.type.value, node.name, meta], sort_keys=True)) signature.update( diff --git a/tests/unit/test_languages.py b/tests/unit/test_languages.py index 19e79364..7a715519 100644 --- a/tests/unit/test_languages.py +++ b/tests/unit/test_languages.py @@ -20,6 +20,7 @@ MANIFESTS, Manifest, classifying_pack, + cross_file_metadata, extractors, is_supported, pack_for_node, @@ -338,3 +339,27 @@ def test_a_source_no_pack_reads_gets_the_generic_tail(source: str) -> None: assert index.classify_fqn("app.g", source) is NodeNamespace.INTERNAL assert index.classify_fqn("js_builtins.Math", source) is NodeNamespace.STDLIB assert index.classify_fqn("os.path", source) is NodeNamespace.UNKNOWN + + +def test_cross_file_metadata_is_the_old_table() -> None: + """The pipeline's incremental signature reads exactly what it read before step 4.""" + assert cross_file_metadata() == { + NodeType.FILE: ("import_map", "reexports", "star_imports"), + NodeType.CLASS: ("self_types",), + NodeType.VARIABLE: ("annotated_type",), + } + + +def test_cross_file_metadata_imports_no_pack() -> None: + """Spec: declared in the manifest, so collecting it imports nothing.""" + script = ( + "import sys\n" + "from cgis.languages import cross_file_metadata\n" + "cross_file_metadata()\n" + "print(sorted(m for m in sys.modules if m.startswith('cgis.languages.')))\n" + ) + out = subprocess.run( + [sys.executable, "-c", script], capture_output=True, text=True, check=False + ) + assert out.returncode == 0, out.stderr + assert out.stdout.strip() == "[]" From b4f83a6f14596418aa446851ffb688e2c79cb289 Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 13:44:41 +0000 Subject: [PATCH 03/10] feat(languages): the resolution contract (ResolutionView, ResolutionRules) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit cgis.core.resolution: the view a rule reads the index through (SymbolIndex satisfies it; strip_layout_prefixes becomes public for it), and the rules a pack contributes — type normalizers, steps at validated positions, a fallback with its confidence, an import reconciler. Packs return empty rules until the next commits move one language branch each. Registry: rules_for, type_normalizers. Co-Authored-By: Claude Opus 5.5 --- src/cgis/core/extractor.py | 5 ++ src/cgis/core/resolution.py | 73 +++++++++++++++++++++++ src/cgis/languages/__init__.py | 17 ++++++ src/cgis/languages/python/__init__.py | 6 ++ src/cgis/languages/typescript/__init__.py | 6 ++ src/cgis/resolver/indices.py | 6 +- tests/unit/test_resolution_rules.py | 38 ++++++++++++ 7 files changed, 148 insertions(+), 3 deletions(-) create mode 100644 src/cgis/core/resolution.py create mode 100644 tests/unit/test_resolution_rules.py diff --git a/src/cgis/core/extractor.py b/src/cgis/core/extractor.py index 9b150116..60d0284c 100644 --- a/src/cgis/core/extractor.py +++ b/src/cgis/core/extractor.py @@ -12,6 +12,7 @@ from cgis.core.index import ClassifyContext, IndexFacts from cgis.core.models import Edge, Node, NodeNamespace from cgis.core.project import ProjectReader +from cgis.core.resolution import ResolutionRules @runtime_checkable @@ -99,3 +100,7 @@ def index_facts(self, nodes: Mapping[str, Node], state: Mapping[str, object]) -> def classify(self, fqn: str, ctx: ClassifyContext) -> NodeNamespace: """The namespace of `fqn` referenced from one of this pack's files: the whole chain (C1).""" ... + + def resolution(self) -> ResolutionRules: + """This pack's resolution rules (spec: Resolution).""" + ... diff --git a/src/cgis/core/resolution.py b/src/cgis/core/resolution.py new file mode 100644 index 00000000..767e70b0 --- /dev/null +++ b/src/cgis/core/resolution.py @@ -0,0 +1,73 @@ +"""What a language pack contributes to resolution (spec: Resolution). + +The engine runs a fixed chain — local type (with type normalizers), import map, +steps positioned `after:import_map`, global symbols, fallback — and asks the +source file's pack at each of those points. A rule reads the index through +`ResolutionView`, never the resolver itself, so `cgis.languages` stays clear of +`cgis.resolver` (language-packs step 4). +""" + +from collections.abc import Callable, Mapping +from dataclasses import dataclass +from typing import Protocol + +from cgis.core.index import IndexFacts +from cgis.core.models import Node + +#: The only position the engine runs pack steps at today: after the import map, +#: before the global symbol index (TS `_package_call`). +AFTER_IMPORT_MAP = "after:import_map" +_POSITIONS = frozenset({AFTER_IMPORT_MAP}) + + +class ResolutionView(Protocol): + """What a resolution rule may read; `SymbolIndex` satisfies it.""" + + @property + def nodes(self) -> Mapping[str, Node]: + """Every node, by id.""" + ... + + @property + def facts(self) -> Mapping[str, IndexFacts]: + """Each pack's index facts, by pack name.""" + ... + + def map_to_node_fqn(self, imported_fqn: str) -> str | None: + """An imported FQN reconciled to a node id, or None.""" + ... + + def strip_layout_prefixes(self, fqn: str) -> str | None: + """The node `fqn` names once corroborated layout prefixes are dropped, or None.""" + ... + + def is_variable_node(self, fqn: str) -> bool: + """Whether `fqn` is a VARIABLE node.""" + ... + + +TypeNormalizer = Callable[[str, ResolutionView], str] +CallStep = Callable[[str, str, ResolutionView], str | None] +Fallback = Callable[[str, str, ResolutionView], tuple[str, float] | None] +ImportReconciler = Callable[[str, str | None, ResolutionView], str | None] + + +@dataclass(frozen=True) +class ResolutionRules: + """One pack's contribution to the resolver's chain.""" + + #: Applied to a declared type before its methods are looked up (e.g. Annotated alias → T). + type_normalizers: tuple[TypeNormalizer, ...] = () + #: (position, step): a call step run at that point of the chain. + steps: tuple[tuple[str, CallStep], ...] = () + #: When nothing resolved a call: (target, confidence), or None. + fallback: Fallback | None = None + #: An import target with no node of that name: the node it means, or None. + reconcile_import: ImportReconciler | None = None + + def __post_init__(self) -> None: + """Refuse a step at a position the engine never runs.""" + unknown = sorted({position for position, _ in self.steps} - _POSITIONS) + if unknown: + msg = f"unknown resolution step position(s): {unknown}" + raise ValueError(msg) diff --git a/src/cgis/languages/__init__.py b/src/cgis/languages/__init__.py index 892aa23b..2be99d1e 100644 --- a/src/cgis/languages/__init__.py +++ b/src/cgis/languages/__init__.py @@ -27,6 +27,7 @@ from cgis.core.index import IndexFacts from cgis.core.models import Node, NodeType from cgis.core.project import ProjectReader +from cgis.core.resolution import ResolutionRules, TypeNormalizer @dataclass(frozen=True) @@ -156,6 +157,22 @@ def cross_file_metadata() -> dict[NodeType, tuple[str, ...]]: return merged +@cache +def _rules(name: str) -> ResolutionRules: + return _load(name).resolution() + + +def rules_for(source_file: str | None) -> ResolutionRules | None: + """The resolution rules of the pack that classifies `source_file`, or None.""" + pack = classifying_pack(source_file) + return None if pack is None else _rules(pack.name) + + +def type_normalizers() -> tuple[TypeNormalizer, ...]: + """Every pack's type normalizers, in manifest order (step 4, ruling D2).""" + return tuple(n for m in MANIFESTS for n in _rules(m.name).type_normalizers) + + def skips(filename: str) -> bool: """Whether the walk never reads this file (`foo.test.ts`, `foo.spec.py`).""" return any(m.never_ingested.search(filename) for m in MANIFESTS) diff --git a/src/cgis/languages/python/__init__.py b/src/cgis/languages/python/__init__.py index 87d08dc0..fb8b94e1 100644 --- a/src/cgis/languages/python/__init__.py +++ b/src/cgis/languages/python/__init__.py @@ -9,6 +9,7 @@ from cgis.core.models import Node, NodeNamespace from cgis.core.paths import is_test_path from cgis.core.project import ProjectReader +from cgis.core.resolution import ResolutionRules from cgis.languages.python import index from cgis.languages.python._syntax import PYTHON_LANG, file_path_to_module_fqn from cgis.languages.python.project import PythonProjectReader @@ -57,5 +58,10 @@ def classify(self, fqn: str, ctx: ClassifyContext) -> NodeNamespace: """See `cgis.languages.python.index.classify`.""" return index.classify(fqn, ctx) + @override + def resolution(self) -> ResolutionRules: + """This pack's rules — filled in by language-packs step 4, one branch per commit.""" + return ResolutionRules() + PACK = PythonPack() diff --git a/src/cgis/languages/typescript/__init__.py b/src/cgis/languages/typescript/__init__.py index 0176e007..029fc103 100644 --- a/src/cgis/languages/typescript/__init__.py +++ b/src/cgis/languages/typescript/__init__.py @@ -9,6 +9,7 @@ from cgis.core.models import Node, NodeNamespace from cgis.core.paths import is_test_path from cgis.core.project import ProjectReader +from cgis.core.resolution import ResolutionRules from cgis.languages.typescript import index from cgis.languages.typescript.naming import TYPESCRIPT_LANG, file_path_to_module_fqn from cgis.languages.typescript.project import TypeScriptProjectReader @@ -57,5 +58,10 @@ def classify(self, fqn: str, ctx: ClassifyContext) -> NodeNamespace: """See `cgis.languages.typescript.index.classify`.""" return index.classify(fqn, ctx) + @override + def resolution(self) -> ResolutionRules: + """This pack's rules — filled in by language-packs step 4, one branch per commit.""" + return ResolutionRules() + PACK = TypeScriptPack() diff --git a/src/cgis/resolver/indices.py b/src/cgis/resolver/indices.py index c7439f46..e8d3f63d 100644 --- a/src/cgis/resolver/indices.py +++ b/src/cgis/resolver/indices.py @@ -130,9 +130,9 @@ def resolve_import_target(self, fqn: str, source_file: str | None = None) -> str return _resolve_typescript_import(fqn, self.nodes, self.workspace_packages, aliased) if source_file is not None and not source_file.endswith(_PYTHON_SUFFIXES): return None - return self._strip_layout_prefixes(fqn) + return self.strip_layout_prefixes(fqn) - def _strip_layout_prefixes(self, fqn: str) -> str | None: + def strip_layout_prefixes(self, fqn: str) -> str | None: """The node `fqn` names once known layout prefixes are dropped from its head. Strips one leading segment at a time, and only while each is a corroborated @@ -220,7 +220,7 @@ def resolve_layout(self, fqn: str) -> str | None: return candidates[0] if self.classify_fqn(fqn) in (NodeNamespace.EXTERNAL, NodeNamespace.STDLIB): return None - return self._strip_layout_prefixes(fqn) + return self.strip_layout_prefixes(fqn) def follow_reexport(self, fqn: str) -> str | None: """Resolve `module.name` through every forwarding hop, or None if not forwarded. diff --git a/tests/unit/test_resolution_rules.py b/tests/unit/test_resolution_rules.py new file mode 100644 index 00000000..17551737 --- /dev/null +++ b/tests/unit/test_resolution_rules.py @@ -0,0 +1,38 @@ +"""The resolution contract between the engine and the language packs (step 4).""" + +import pytest + +from cgis.core.resolution import AFTER_IMPORT_MAP, ResolutionRules, ResolutionView +from cgis.languages import rules_for, type_normalizers +from cgis.resolver.indices import IndexBuilder + + +def _step(_name: str, _file_path: str, _view: ResolutionView) -> str | None: + return None + + +def test_an_unknown_step_position_is_refused() -> None: + """A typo would otherwise drop the rule silently.""" + with pytest.raises(ValueError, match="after:imports"): + ResolutionRules(steps=(("after:imports", _step),)) + + +def test_the_known_position_is_accepted() -> None: + assert ResolutionRules(steps=((AFTER_IMPORT_MAP, _step),)).steps[0][0] == AFTER_IMPORT_MAP + + +def test_the_index_is_a_resolution_view() -> None: + """Rules read the index through the core protocol, never by importing the resolver.""" + view: ResolutionView = IndexBuilder().build([]) + assert view.strip_layout_prefixes("x.y") is None + + +def test_rules_follow_the_classifying_pack() -> None: + """No source is Python's (the preserved quirk); a source no pack reads has none.""" + assert rules_for(None) is not None + assert rules_for("a.ts") is not None + assert rules_for("a.js") is None + + +def test_normalizers_come_in_manifest_order() -> None: + assert isinstance(type_normalizers(), tuple) From 962d067dd1399142fef88c776f42ad58027895f6 Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 13:59:17 +0000 Subject: [PATCH 04/10] refactor(languages): import reconciliation is each pack's rule resolve_import_target keeps the exact-id hit and hands the rest to the source pack's reconcile_import: TypeScript's reads tsconfig aliases, workspace packages and `.d` declarations from its own index facts; Python's strips corroborated layout prefixes. SymbolIndex loses workspace_packages and path_aliases, and the resolver its last suffix checks (_PYTHON_SUFFIXES, _TYPESCRIPT_SUFFIXES). Self-drift: resolution hygiene unresolved_ratio 0.444 -> 0.459 (161 UNKNOWN of 351 calls, from 169 of 381): the moved TS helpers took more resolved calls than unresolved ones out of the domain, and the rule call `rules.reconcile_import` is one new dynamic dispatch. Re-baselined once at the end of step 4 (user decision). Co-Authored-By: Claude Opus 5.5 --- src/cgis/languages/python/__init__.py | 6 +- src/cgis/languages/python/resolution.py | 11 ++ src/cgis/languages/typescript/__init__.py | 6 +- src/cgis/languages/typescript/index.py | 42 +++++- src/cgis/languages/typescript/resolution.py | 111 +++++++++++++++ src/cgis/resolver/indices.py | 147 +------------------- tests/unit/test_resolution_rules.py | 28 ++++ tests/unit/test_typescript_index.py | 18 +++ 8 files changed, 222 insertions(+), 147 deletions(-) create mode 100644 src/cgis/languages/python/resolution.py create mode 100644 src/cgis/languages/typescript/resolution.py diff --git a/src/cgis/languages/python/__init__.py b/src/cgis/languages/python/__init__.py index fb8b94e1..990f8601 100644 --- a/src/cgis/languages/python/__init__.py +++ b/src/cgis/languages/python/__init__.py @@ -10,7 +10,7 @@ from cgis.core.paths import is_test_path from cgis.core.project import ProjectReader from cgis.core.resolution import ResolutionRules -from cgis.languages.python import index +from cgis.languages.python import index, resolution from cgis.languages.python._syntax import PYTHON_LANG, file_path_to_module_fqn from cgis.languages.python.project import PythonProjectReader @@ -60,8 +60,8 @@ def classify(self, fqn: str, ctx: ClassifyContext) -> NodeNamespace: @override def resolution(self) -> ResolutionRules: - """This pack's rules — filled in by language-packs step 4, one branch per commit.""" - return ResolutionRules() + """See `cgis.languages.python.resolution`.""" + return resolution.RULES PACK = PythonPack() diff --git a/src/cgis/languages/python/resolution.py b/src/cgis/languages/python/resolution.py new file mode 100644 index 00000000..ac485764 --- /dev/null +++ b/src/cgis/languages/python/resolution.py @@ -0,0 +1,11 @@ +"""The Python pack's resolution rules (spec: Resolution; language-packs step 4).""" + +from cgis.core.resolution import ResolutionRules, ResolutionView + + +def reconcile_import(fqn: str, _source_file: str | None, view: ResolutionView) -> str | None: + """A Python import's node once corroborated layout prefixes are stripped (#319, #494).""" + return view.strip_layout_prefixes(fqn) + + +RULES = ResolutionRules(reconcile_import=reconcile_import) diff --git a/src/cgis/languages/typescript/__init__.py b/src/cgis/languages/typescript/__init__.py index 029fc103..d7562a5d 100644 --- a/src/cgis/languages/typescript/__init__.py +++ b/src/cgis/languages/typescript/__init__.py @@ -10,7 +10,7 @@ from cgis.core.paths import is_test_path from cgis.core.project import ProjectReader from cgis.core.resolution import ResolutionRules -from cgis.languages.typescript import index +from cgis.languages.typescript import index, resolution from cgis.languages.typescript.naming import TYPESCRIPT_LANG, file_path_to_module_fqn from cgis.languages.typescript.project import TypeScriptProjectReader @@ -60,8 +60,8 @@ def classify(self, fqn: str, ctx: ClassifyContext) -> NodeNamespace: @override def resolution(self) -> ResolutionRules: - """This pack's rules — filled in by language-packs step 4, one branch per commit.""" - return ResolutionRules() + """See `cgis.languages.typescript.resolution`.""" + return resolution.RULES PACK = TypeScriptPack() diff --git a/src/cgis/languages/typescript/index.py b/src/cgis/languages/typescript/index.py index 70451ff0..fffeb97c 100644 --- a/src/cgis/languages/typescript/index.py +++ b/src/cgis/languages/typescript/index.py @@ -8,7 +8,7 @@ from collections.abc import Mapping from dataclasses import dataclass from types import MappingProxyType -from typing import cast +from typing import NamedTuple, cast from cgis.core.index import ClassifyContext from cgis.core.js_globals import ( @@ -19,6 +19,20 @@ ) from cgis.core.models import Node, NodeNamespace, NodeType +#: The wildcard of a tsconfig `paths` alias, kept in its dotted spelling (#508). +_ALIAS_WILDCARD = "*" + + +class PathAlias(NamedTuple): + """One tsconfig `paths` entry, dotted: `@lib.*` -> (`apps.web.lib.*`,) (#508).""" + + #: The pattern without its trailing `*`: what a matching import starts with. + prefix: str + #: Whether the pattern ended in `*`; otherwise it must equal the import. + wildcard: bool + #: Target FQNs in the order TypeScript tries them; a `*` takes the remainder. + targets: tuple[str, ...] + @dataclass(frozen=True) class TypeScriptIndexFacts: @@ -30,6 +44,13 @@ class TypeScriptIndexFacts: npm_dependencies: frozenset[str] #: normalized file_path -> the bare specifiers that file imports, as written. file_bare_imports: Mapping[str, frozenset[str]] + #: dotted workspace package name -> the directory FQN it lives at, longest name + #: first: npm names may contain dots, so `a.b` is both "package a, subpath b" and + #: "package a.b" once `/` is dotted (#504). + workspace_packages: Mapping[str, str] + #: project directory -> its tsconfig `paths` aliases, exact patterns first and then + #: wildcards longest prefix first — the order TypeScript picks one in (#508). + path_aliases: Mapping[str, tuple[PathAlias, ...]] def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> TypeScriptIndexFacts: @@ -39,11 +60,17 @@ def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> TypeS for n in nodes.values() if n.type == NodeType.FILE and n.metadata.get("bare_imports") } + packages = cast("dict[str, str]", state.get("workspace_packages", {})) + aliases = cast("dict[str, dict[str, list[str]]]", state.get("tsconfig_paths", {})) return TypeScriptIndexFacts( internal_roots=frozenset(), layout_prefixes=frozenset(), npm_dependencies=frozenset(cast("list[str]", state.get("npm_dependencies", []))), file_bare_imports=MappingProxyType(bare), + workspace_packages=MappingProxyType( + dict(sorted(packages.items(), key=lambda i: -len(i[0]))) + ), + path_aliases=MappingProxyType({scope: _ordered_aliases(r) for scope, r in aliases.items()}), ) @@ -99,3 +126,16 @@ def classify(fqn: str, ctx: ClassifyContext) -> NodeNamespace: if root == JS_BUILTINS_ROOT: return NodeNamespace.STDLIB return NodeNamespace.UNKNOWN + + +def _ordered_aliases(rules: Mapping[str, list[str]]) -> tuple[PathAlias, ...]: + """One project's aliases as `PathAlias`es, in the order TypeScript tries patterns.""" + aliases = [ + PathAlias( + prefix=pattern.removesuffix(_ALIAS_WILDCARD), + wildcard=pattern.endswith(_ALIAS_WILDCARD), + targets=tuple(targets), + ) + for pattern, targets in rules.items() + ] + return tuple(sorted(aliases, key=lambda a: (a.wildcard, -len(a.prefix), a.prefix))) diff --git a/src/cgis/languages/typescript/resolution.py b/src/cgis/languages/typescript/resolution.py new file mode 100644 index 00000000..84f996af --- /dev/null +++ b/src/cgis/languages/typescript/resolution.py @@ -0,0 +1,111 @@ +"""The TypeScript pack's resolution rules (spec: Resolution; language-packs step 4). + +Moved from `cgis.resolver.indices` with their docstrings: what a TypeScript import +means once tsconfig aliases, workspace packages and declaration files are read. +""" + +import os +import posixpath +from collections.abc import Mapping +from typing import cast + +from cgis.core.models import Node +from cgis.core.resolution import ResolutionRules, ResolutionView +from cgis.languages.typescript.index import _ALIAS_WILDCARD, PathAlias, TypeScriptIndexFacts + +#: What a declaration file's module FQN ends in: `Calendar.d.ts` -> `Calendar.d` (#507). +_DECLARATION_SUFFIX = ".d" + + +def _resolve_typescript_import( + fqn: str, + nodes: Mapping[str, Node], + packages: Mapping[str, str], + aliased: tuple[str, ...] = (), +) -> str | None: + """Reconcile a TypeScript import against the node ids, or None to leave it alone. + + Each candidate is tried as written and then as a declaration file: + `packages/types/Calendar.d.ts` is the module `packages.types.Calendar.d`, + while code imports it with no `.d` (#507). That is the order TypeScript + resolves in, so an implementation beside its declaration wins. The node ids + are not changed instead: that would rename every declaration node in + existing graphs and give `foo.ts` and `foo.d.ts` one id. + + The candidates are the target itself — a relative import, already made + absolute by the extractor — then what a tsconfig `paths` alias of the + importing project rewrites it to (#508), and then, for a workspace package, + the module it names (#504). Aliases come before packages because TypeScript + consults `paths` before `node_modules`, where a workspace package is linked. + """ + for candidate in (fqn, *aliased, *_workspace_candidates(fqn, packages)): + for spelled in (candidate, f"{candidate}{_DECLARATION_SUFFIX}"): + if spelled in nodes: + return spelled + return None + + +def _workspace_candidates(fqn: str, packages: Mapping[str, str]) -> tuple[str, ...]: + """The module FQNs a TypeScript import of a workspace package may name. + + `@calcom.lib.hooks.useLocale` becomes `packages.lib.hooks.useLocale` when the + pipeline found a `package.json` named `@calcom/lib` in `packages/lib`; the + package root is tried before `src/`, a common layout for a package's sources. + + The name must match on a segment boundary, so `@x/a` never claims + `@x/a-b`. The first — longest — package that matches decides: falling back + to a shorter one would read `a.b.util` as package `a` when package `a.b` + was meant. No candidates for a name the map does not hold. + """ + for name, directory in packages.items(): + if fqn != name and not fqn.startswith(name + "."): + continue + subpath = fqn[len(name) + 1 :] + return tuple( + f"{base}.{subpath}" if subpath else base for base in (directory, f"{directory}.src") + ) + return () + + +def _alias_candidates( + fqn: str, source_file: str, scopes: Mapping[str, tuple[PathAlias, ...]] +) -> tuple[str, ...]: + """The module FQNs the importing project's tsconfig `paths` rewrite `fqn` to. + + The project is the nearest directory above `source_file` that has a + tsconfig; a nearer one without aliases shadows a farther one with them, as + it does for the compiler. Within it the first pattern that matches decides, + exact before wildcard and longer prefix before shorter — and, as in + TypeScript, a pattern whose targets miss does not fall back to another. + """ + if not scopes: + return () + directory = posixpath.dirname(source_file.replace(os.sep, "/")) + while directory not in scopes: + if not directory: + return () + directory = posixpath.dirname(directory) + for alias in scopes[directory]: + if not alias.wildcard: + if fqn == alias.prefix: + return alias.targets + continue + if fqn.startswith(alias.prefix): + # The remainder may be empty, as in TypeScript (`jquery*` matches + # `jquery`): the target is then its directory, so `lib.*` -> `lib`. + rest = fqn[len(alias.prefix) :] + substituted = ( + t.replace(_ALIAS_WILDCARD, rest).removesuffix(".") for t in alias.targets + ) + return tuple(candidate for candidate in substituted if candidate) + return () + + +def reconcile_import(fqn: str, source_file: str | None, view: ResolutionView) -> str | None: + """A TS import's node through tsconfig aliases, then workspace packages (#504, #507, #508).""" + facts = cast("TypeScriptIndexFacts", view.facts["typescript"]) + aliased = _alias_candidates(fqn, source_file or "", facts.path_aliases) + return _resolve_typescript_import(fqn, view.nodes, facts.workspace_packages, aliased) + + +RULES = ResolutionRules(reconcile_import=reconcile_import) diff --git a/src/cgis/resolver/indices.py b/src/cgis/resolver/indices.py index e8d3f63d..5b93ea8b 100644 --- a/src/cgis/resolver/indices.py +++ b/src/cgis/resolver/indices.py @@ -1,39 +1,19 @@ """Symbol indices for FQN resolution: built once by IndexBuilder, read-only after.""" import os -import posixpath from collections.abc import Iterable, Mapping from dataclasses import dataclass from types import MappingProxyType -from typing import NamedTuple from cgis.core.index import ClassifyContext, IndexFacts from cgis.core.js_globals import JS_BUILTINS_ROOT from cgis.core.models import SELF_PREFIX, Node, NodeNamespace, NodeType -from cgis.languages import classifying_pack +from cgis.languages import classifying_pack, rules_for from cgis.languages import index_facts as pack_index_facts + #: Sources whose references read against Python's stdlib, builtins and import roots. -_PYTHON_SUFFIXES: tuple[str, ...] = (".py",) #: The suffixes the TypeScript extractor handles; `.js`/`.jsx` are not ingested (registry.py). -_TYPESCRIPT_SUFFIXES = (".ts", ".tsx") -#: What a declaration file's module FQN ends in: `Calendar.d.ts` -> `Calendar.d` (#507). -_DECLARATION_SUFFIX = ".d" -#: The wildcard of a tsconfig `paths` alias, kept in its dotted spelling (#508). -_ALIAS_WILDCARD = "*" - - -class PathAlias(NamedTuple): - """One tsconfig `paths` entry, dotted: `@lib.*` -> (`apps.web.lib.*`,) (#508).""" - - #: The pattern without its trailing `*`: what a matching import starts with. - prefix: str - #: Whether the pattern ended in `*`; otherwise it must equal the import. - wildcard: bool - #: Target FQNs in the order TypeScript tries them; a `*` takes the remainder. - targets: tuple[str, ...] - - @dataclass(frozen=True) class SymbolIndex: """Immutable lookup indices over the extracted node set. @@ -86,16 +66,6 @@ class SymbolIndex: # import maps, never guessed — kept apart from `internal_roots` so a target can # be reconciled by stripping *these* and nothing else (#494). layout_prefixes: frozenset[str] = frozenset() - # dotted workspace package name -> the directory FQN it lives at, longest name - # first: `@calcom.lib` -> `packages.lib`. Read from the `package.json` files the - # pipeline walked, never guessed, and consulted only for TypeScript sources - # (#504). Longest first because npm names may contain dots, so once `/` is - # dotted `a.b` is both "package a, subpath b" and "package a.b". - workspace_packages: Mapping[str, str] = MappingProxyType({}) - # project directory -> its tsconfig `paths` aliases, exact patterns first and - # then wildcards longest prefix first — the order TypeScript picks one in. - # A file uses the nearest project above it (#508). - path_aliases: Mapping[str, tuple[PathAlias, ...]] = MappingProxyType({}) # normalized file_path -> {local name: package symbol FQN} for a TS file's imports: # `useState` -> `react.useState`, a namespace import `Icons` -> `lucide-react`. file_bare_bindings: Mapping[str, dict[str, str]] = MappingProxyType({}) @@ -125,12 +95,10 @@ def resolve_import_target(self, fqn: str, source_file: str | None = None) -> str """ if fqn in self.nodes: return fqn - if source_file is not None and source_file.endswith(_TYPESCRIPT_SUFFIXES): - aliased = _alias_candidates(fqn, source_file, self.path_aliases) - return _resolve_typescript_import(fqn, self.nodes, self.workspace_packages, aliased) - if source_file is not None and not source_file.endswith(_PYTHON_SUFFIXES): + rules = rules_for(source_file) + if rules is None or rules.reconcile_import is None: return None - return self.strip_layout_prefixes(fqn) + return rules.reconcile_import(fqn, source_file, self) def strip_layout_prefixes(self, fqn: str) -> str | None: """The node `fqn` names once known layout prefixes are dropped from its head. @@ -315,103 +283,6 @@ def normalized_file_path(self, source_fqn: str, edge_file_path: str | None) -> s _ROOT_PACKAGE = "__init__" -def _resolve_typescript_import( - fqn: str, - nodes: Mapping[str, Node], - packages: Mapping[str, str], - aliased: tuple[str, ...] = (), -) -> str | None: - """Reconcile a TypeScript import against the node ids, or None to leave it alone. - - Each candidate is tried as written and then as a declaration file: - `packages/types/Calendar.d.ts` is the module `packages.types.Calendar.d`, - while code imports it with no `.d` (#507). That is the order TypeScript - resolves in, so an implementation beside its declaration wins. The node ids - are not changed instead: that would rename every declaration node in - existing graphs and give `foo.ts` and `foo.d.ts` one id. - - The candidates are the target itself — a relative import, already made - absolute by the extractor — then what a tsconfig `paths` alias of the - importing project rewrites it to (#508), and then, for a workspace package, - the module it names (#504). Aliases come before packages because TypeScript - consults `paths` before `node_modules`, where a workspace package is linked. - """ - for candidate in (fqn, *aliased, *_workspace_candidates(fqn, packages)): - for spelled in (candidate, f"{candidate}{_DECLARATION_SUFFIX}"): - if spelled in nodes: - return spelled - return None - - -def _workspace_candidates(fqn: str, packages: Mapping[str, str]) -> tuple[str, ...]: - """The module FQNs a TypeScript import of a workspace package may name. - - `@calcom.lib.hooks.useLocale` becomes `packages.lib.hooks.useLocale` when the - pipeline found a `package.json` named `@calcom/lib` in `packages/lib`; the - package root is tried before `src/`, a common layout for a package's sources. - - The name must match on a segment boundary, so `@x/a` never claims - `@x/a-b`. The first — longest — package that matches decides: falling back - to a shorter one would read `a.b.util` as package `a` when package `a.b` - was meant. No candidates for a name the map does not hold. - """ - for name, directory in packages.items(): - if fqn != name and not fqn.startswith(name + "."): - continue - subpath = fqn[len(name) + 1 :] - return tuple( - f"{base}.{subpath}" if subpath else base for base in (directory, f"{directory}.src") - ) - return () - - -def _alias_candidates( - fqn: str, source_file: str, scopes: Mapping[str, tuple[PathAlias, ...]] -) -> tuple[str, ...]: - """The module FQNs the importing project's tsconfig `paths` rewrite `fqn` to. - - The project is the nearest directory above `source_file` that has a - tsconfig; a nearer one without aliases shadows a farther one with them, as - it does for the compiler. Within it the first pattern that matches decides, - exact before wildcard and longer prefix before shorter — and, as in - TypeScript, a pattern whose targets miss does not fall back to another. - """ - if not scopes: - return () - directory = posixpath.dirname(source_file.replace(os.sep, "/")) - while directory not in scopes: - if not directory: - return () - directory = posixpath.dirname(directory) - for alias in scopes[directory]: - if not alias.wildcard: - if fqn == alias.prefix: - return alias.targets - continue - if fqn.startswith(alias.prefix): - # The remainder may be empty, as in TypeScript (`jquery*` matches - # `jquery`): the target is then its directory, so `lib.*` -> `lib`. - rest = fqn[len(alias.prefix) :] - substituted = ( - t.replace(_ALIAS_WILDCARD, rest).removesuffix(".") for t in alias.targets - ) - return tuple(candidate for candidate in substituted if candidate) - return () - - -def _ordered_aliases(rules: Mapping[str, list[str]]) -> tuple[PathAlias, ...]: - """One project's aliases as `PathAlias`es, in the order TypeScript tries patterns.""" - aliases = [ - PathAlias( - prefix=pattern.removesuffix(_ALIAS_WILDCARD), - wildcard=pattern.endswith(_ALIAS_WILDCARD), - targets=tuple(targets), - ) - for pattern, targets in rules.items() - ] - return tuple(sorted(aliases, key=lambda a: (a.wildcard, -len(a.prefix), a.prefix))) - - class IndexBuilder: """Builds a SymbolIndex from extracted nodes (Phase 1: indexing). @@ -487,6 +358,8 @@ def build( state = { "python_dependencies": list(dependency_roots), "npm_dependencies": list(npm_dependencies), + "workspace_packages": dict(workspace_packages or {}), + "tsconfig_paths": dict(tsconfig_paths or {}), } facts = pack_index_facts(MappingProxyType(nodes_by_id), state) roots = frozenset(internal_roots.union(*(f.internal_roots for f in facts.values()))) @@ -510,12 +383,6 @@ def build( internal_roots=roots, layout_prefixes=prefixes, facts=MappingProxyType(facts), - workspace_packages=MappingProxyType( - dict(sorted((workspace_packages or {}).items(), key=lambda item: -len(item[0]))) - ), - path_aliases=MappingProxyType( - {scope: _ordered_aliases(rules) for scope, rules in (tsconfig_paths or {}).items()} - ), file_bare_bindings=MappingProxyType(file_bare_bindings), ) diff --git a/tests/unit/test_resolution_rules.py b/tests/unit/test_resolution_rules.py index 17551737..5390923b 100644 --- a/tests/unit/test_resolution_rules.py +++ b/tests/unit/test_resolution_rules.py @@ -2,6 +2,7 @@ import pytest +from cgis.core.models import Node, NodeType from cgis.core.resolution import AFTER_IMPORT_MAP, ResolutionRules, ResolutionView from cgis.languages import rules_for, type_normalizers from cgis.resolver.indices import IndexBuilder @@ -36,3 +37,30 @@ def test_rules_follow_the_classifying_pack() -> None: def test_normalizers_come_in_manifest_order() -> None: assert isinstance(type_normalizers(), tuple) + + +def _module(fqn: str, path: str) -> Node: + return Node( + id=fqn, + type=NodeType.FILE, + name=path.rsplit("/", 1)[-1], + file_path=path, + start_line=1, + end_line=1, + language="typescript", + ) + + +def test_typescript_reconciles_through_its_own_pack() -> None: + """Aliases and workspace packages are the TS pack's facts; the index no longer carries them.""" + nodes = [_module("packages.lib.util", "packages/lib/util.ts")] + index = IndexBuilder().build(nodes, workspace_packages={"@acme.lib": "packages.lib"}) + assert not hasattr(index, "workspace_packages") + assert index.resolve_import_target("@acme.lib.util", "apps/web/a.ts") == "packages.lib.util" + assert index.resolve_import_target("@acme.lib.util", "apps/web/a.py") is None + + +def test_python_reconciles_by_layout_prefixes() -> None: + rules = rules_for("app/x.py") + assert rules is not None + assert rules.reconcile_import is not None diff --git a/tests/unit/test_typescript_index.py b/tests/unit/test_typescript_index.py index 706ed2c5..6798ef13 100644 --- a/tests/unit/test_typescript_index.py +++ b/tests/unit/test_typescript_index.py @@ -62,3 +62,21 @@ def test_classification_chain() -> None: classify("json.parse", ctx) is NodeNamespace.INTERNAL ) # an internal root, not Python's json assert classify("os.path", ctx) is NodeNamespace.UNKNOWN # #454: Python names mean nothing here + + +def test_workspace_packages_are_kept_longest_name_first() -> None: + """`a.b` is both "package a, subpath b" and "package a.b": the longer name wins (#504).""" + facts = index_facts( + MappingProxyType({}), + {"workspace_packages": {"a": "x", "a.b": "y"}, "npm_dependencies": []}, + ) + assert list(facts.workspace_packages) == ["a.b", "a"] + + +def test_path_aliases_are_ordered_per_project() -> None: + facts = index_facts( + MappingProxyType({}), + {"tsconfig_paths": {"apps/web": {"@.*": ["apps.web.src.*"]}}, "npm_dependencies": []}, + ) + (alias,) = facts.path_aliases["apps/web"] + assert (alias.prefix, alias.wildcard, alias.targets) == ("@.", True, ("apps.web.src.*",)) From 2d2399b7b07e56a1cb485fe19953271cd2cdbe2f Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 14:02:30 +0000 Subject: [PATCH 05/10] refactor(languages): _package_call is the TypeScript pack's after:import_map step resolve_global_call runs the source pack's steps at `after:import_map` through a module-level helper (SymbolResolver is at the god-object threshold). The TS step reads the file's bare bindings from the TS pack's index facts; SymbolIndex loses file_bare_bindings. The resolver no longer imports a pack: the engine-packs allowlist is empty. Co-Authored-By: Claude Opus 5.5 --- src/cgis/languages/typescript/index.py | 10 +++++ src/cgis/languages/typescript/resolution.py | 40 +++++++++++++++-- src/cgis/resolver/indices.py | 18 ++------ src/cgis/resolver/symbols.py | 43 ++++++------------- .../forbidden_edges_allowlist.json | 4 +- tests/unit/test_resolution_rules.py | 6 +++ 6 files changed, 72 insertions(+), 49 deletions(-) diff --git a/src/cgis/languages/typescript/index.py b/src/cgis/languages/typescript/index.py index fffeb97c..9e6e50cc 100644 --- a/src/cgis/languages/typescript/index.py +++ b/src/cgis/languages/typescript/index.py @@ -48,6 +48,9 @@ class TypeScriptIndexFacts: #: first: npm names may contain dots, so `a.b` is both "package a, subpath b" and #: "package a.b" once `/` is dotted (#504). workspace_packages: Mapping[str, str] + #: normalized file_path -> {local name: package symbol FQN} for that file's bare + #: imports: `useState` -> `react.useState`, a namespace import `Icons` -> `lucide-react`. + file_bare_bindings: Mapping[str, dict[str, str]] #: project directory -> its tsconfig `paths` aliases, exact patterns first and then #: wildcards longest prefix first — the order TypeScript picks one in (#508). path_aliases: Mapping[str, tuple[PathAlias, ...]] @@ -67,6 +70,13 @@ def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> TypeS layout_prefixes=frozenset(), npm_dependencies=frozenset(cast("list[str]", state.get("npm_dependencies", []))), file_bare_imports=MappingProxyType(bare), + file_bare_bindings=MappingProxyType( + { + os.path.normpath(n.file_path): dict(n.metadata.get("bare_bindings") or {}) + for n in nodes.values() + if n.type == NodeType.FILE and n.metadata.get("bare_imports") + } + ), workspace_packages=MappingProxyType( dict(sorted(packages.items(), key=lambda i: -len(i[0]))) ), diff --git a/src/cgis/languages/typescript/resolution.py b/src/cgis/languages/typescript/resolution.py index 84f996af..4a5dfd20 100644 --- a/src/cgis/languages/typescript/resolution.py +++ b/src/cgis/languages/typescript/resolution.py @@ -10,8 +10,13 @@ from typing import cast from cgis.core.models import Node -from cgis.core.resolution import ResolutionRules, ResolutionView -from cgis.languages.typescript.index import _ALIAS_WILDCARD, PathAlias, TypeScriptIndexFacts +from cgis.core.resolution import AFTER_IMPORT_MAP, ResolutionRules, ResolutionView +from cgis.languages.typescript.index import ( + _ALIAS_WILDCARD, + PathAlias, + TypeScriptIndexFacts, + package_namespace, +) #: What a declaration file's module FQN ends in: `Calendar.d.ts` -> `Calendar.d` (#507). _DECLARATION_SUFFIX = ".d" @@ -108,4 +113,33 @@ def reconcile_import(fqn: str, source_file: str | None, view: ResolutionView) -> return _resolve_typescript_import(fqn, view.nodes, facts.workspace_packages, aliased) -RULES = ResolutionRules(reconcile_import=reconcile_import) +def package_call(name: str, file_path: str, view: ResolutionView) -> str | None: + """A TypeScript call on a name imported from a package: `useState` -> `react.useState`. + + Only when the binding lands in a package (#556). A binding into one of our + own modules through an alias or workspace name is left to the lookups that + already resolve those, so internal resolution is unchanged. + + Bindings are per file, not per scope: a parameter that shadows an imported + name (`function f(map) { map() }` beside `import { map } from "lodash"`) is + read as the import. Accepted — shadowing an import is rare, and scope + tracking is a separate piece of work. + """ + facts = cast("TypeScriptIndexFacts", view.facts["typescript"]) + bindings = facts.file_bare_bindings.get(file_path) + if not bindings: + return None + head, _, rest = name.partition(".") + bound = bindings.get(head) + if bound is None: + return None + candidate = f"{bound}.{rest}" if rest else bound + if package_namespace(candidate, file_path, facts) is None: + return None + return candidate + + +RULES = ResolutionRules( + steps=((AFTER_IMPORT_MAP, package_call),), + reconcile_import=reconcile_import, +) diff --git a/src/cgis/resolver/indices.py b/src/cgis/resolver/indices.py index 5b93ea8b..9d51dd3d 100644 --- a/src/cgis/resolver/indices.py +++ b/src/cgis/resolver/indices.py @@ -66,9 +66,6 @@ class SymbolIndex: # import maps, never guessed — kept apart from `internal_roots` so a target can # be reconciled by stripping *these* and nothing else (#494). layout_prefixes: frozenset[str] = frozenset() - # normalized file_path -> {local name: package symbol FQN} for a TS file's imports: - # `useState` -> `react.useState`, a namespace import `Icons` -> `lucide-react`. - file_bare_bindings: Mapping[str, dict[str, str]] = MappingProxyType({}) def resolve_import_target(self, fqn: str, source_file: str | None = None) -> str | None: """Reconcile a module import against the node ids, or None to leave it alone. @@ -316,7 +313,6 @@ def build( file_variable_symbols: dict[tuple[str, str], list[str]] = {} file_imports: dict[str, dict[str, str]] = {} file_shadowed_globals: dict[str, frozenset[str]] = {} - file_bare_bindings: dict[str, dict[str, str]] = {} reexports: dict[str, dict[str, str]] = {} star_imports: dict[str, list[str]] = {} suffix_map: dict[str, list[str]] = {} @@ -330,7 +326,6 @@ def build( self_types, reexports, star_imports, - file_bare_bindings, ) if node.type == NodeType.FILE and node.metadata.get("shadowed_globals"): file_shadowed_globals[os.path.normpath(node.file_path)] = frozenset( @@ -383,7 +378,6 @@ def build( internal_roots=roots, layout_prefixes=prefixes, facts=MappingProxyType(facts), - file_bare_bindings=MappingProxyType(file_bare_bindings), ) @staticmethod @@ -483,21 +477,17 @@ def _index_metadata( self_types: dict[str, dict[str, str]], reexports: dict[str, dict[str, str]], star_imports: dict[str, list[str]], - bare_bindings: dict[str, dict[str, str]], ) -> None: """Read the maps the extractor attached to a FILE or CLASS node. - A FILE carries its import map, and a TypeScript one its bare imports and - their bindings (#556); a CLASS carries the attribute types recorded from - annotations (spec D1). All are extractor output the resolver consumes - rather than anything derived here. + A FILE carries its import map, re-exports and star imports; a CLASS the + attribute types recorded from annotations (spec D1). A TypeScript FILE's + bare imports and bindings are the TypeScript pack's index facts (step 4). + All are extractor output the resolver consumes rather than derives. """ if node.type == NodeType.FILE: path = os.path.normpath(node.file_path) file_imports[path] = node.metadata.get("import_map") or {} - specifiers = node.metadata.get("bare_imports") - if specifiers: - bare_bindings[path] = dict(node.metadata.get("bare_bindings") or {}) forwarded = node.metadata.get("reexports") if forwarded: reexports[node.id] = dict(forwarded) diff --git a/src/cgis/resolver/symbols.py b/src/cgis/resolver/symbols.py index 16fa1b55..55064ea1 100644 --- a/src/cgis/resolver/symbols.py +++ b/src/cgis/resolver/symbols.py @@ -3,7 +3,8 @@ from collections.abc import Iterable, Mapping from cgis.core.models import RAW_CLASS_PREFIX, Edge, EdgeType, Node, NodeNamespace -from cgis.languages.typescript.index import TypeScriptIndexFacts, package_namespace +from cgis.core.resolution import AFTER_IMPORT_MAP +from cgis.languages import rules_for from cgis.resolver.indices import IndexBuilder, SymbolIndex @@ -21,33 +22,17 @@ def _owning_class(index: SymbolIndex, source_fqn: str) -> str | None: return None -def _package_call(index: SymbolIndex, name: str, file_path: str) -> str | None: - """A TypeScript call on a name imported from a package: `useState` -> `react.useState`. - - Only when the binding lands in a package (#556). A binding into one of our - own modules through an alias or workspace name is left to the lookups that - already resolve those, so internal resolution is unchanged. - - Bindings are per file, not per scope: a parameter that shadows an imported - name (`function f(map) { map() }` beside `import { map } from "lodash"`) is - read as the import. Accepted — shadowing an import is rare, and scope - tracking is a separate piece of work. - """ - bindings = index.file_bare_bindings.get(file_path) - if not bindings: - return None - head, _, rest = name.partition(".") - bound = bindings.get(head) - if bound is None: +def _positioned(index: SymbolIndex, position: str, name: str, file_path: str) -> str | None: + """The first hit among the source pack's steps at `position` (spec: Resolution).""" + rules = rules_for(file_path) + if rules is None: return None - candidate = f"{bound}.{rest}" if rest else bound - # A TypeScript rule inside the engine until step 4 makes it the TS pack's step (C5). - facts = index.facts.get("typescript") - if not isinstance(facts, TypeScriptIndexFacts): - return None - if package_namespace(candidate, file_path, facts) is None: - return None - return candidate + for step_position, step in rules.steps: + if step_position == position: + found = step(name, file_path, index) + if found: + return found + return None def receiver_type_fqn(index: SymbolIndex, owner_fqn: str, attr: str) -> str | None: @@ -207,8 +192,8 @@ def resolve_global_call( return result if file_path: - result = self._resolve_via_import_map(name, file_path) or _package_call( - self.index, name, file_path + result = self._resolve_via_import_map(name, file_path) or _positioned( + self.index, AFTER_IMPORT_MAP, name, file_path ) if result: return result diff --git a/tests/self_parsing/forbidden_edges_allowlist.json b/tests/self_parsing/forbidden_edges_allowlist.json index c39a6b7d..4fff686f 100644 --- a/tests/self_parsing/forbidden_edges_allowlist.json +++ b/tests/self_parsing/forbidden_edges_allowlist.json @@ -1,9 +1,7 @@ { "core-isolation": {}, "cross-pack": {}, - "engine-packs": { - "cgis.resolver.symbols -> cgis.languages.typescript.index": 4 - }, + "engine-packs": {}, "pack-isolation": {}, "resolver-storage": { "cgis.resolver.uplift -> cgis.storage.sqlite_store": 8 diff --git a/tests/unit/test_resolution_rules.py b/tests/unit/test_resolution_rules.py index 5390923b..0ba81be9 100644 --- a/tests/unit/test_resolution_rules.py +++ b/tests/unit/test_resolution_rules.py @@ -64,3 +64,9 @@ def test_python_reconciles_by_layout_prefixes() -> None: rules = rules_for("app/x.py") assert rules is not None assert rules.reconcile_import is not None + + +def test_package_call_is_the_typescript_packs_step() -> None: + rules = rules_for("app/x.ts") + assert rules is not None + assert [position for position, _ in rules.steps] == [AFTER_IMPORT_MAP] From 16a994d3860cb4a44cfcc5ed9a77349f7e59b731 Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 14:08:39 +0000 Subject: [PATCH 06/10] refactor(languages): the JS-builtins fallback is the TypeScript pack's MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit js_builtins.py moves into the TypeScript pack; the engine asks the source pack's fallback for a call nothing resolved and takes its (target, confidence) — 0.8 for a JS runtime global, as before. A missing file still rewrites nothing. The file's shadowed globals are TS index facts; SymbolIndex loses file_shadowed_globals, and js_builtin_target its now-redundant language check. The engine's fallback helper reads the index as core's ResolutionView, not as SymbolIndex: importing cgis.resolver.indices into engine made an engine→symbols→indices triangle that took resolution's self-drift from 0.384 to 0.594. Through the protocol the domain's imports are a single chain and the drift is 0.177. Co-Authored-By: Claude Opus 5.5 --- .../typescript/builtins.py} | 23 ++++--------------- src/cgis/languages/typescript/index.py | 9 ++++++++ src/cgis/languages/typescript/resolution.py | 15 ++++++++++++ src/cgis/resolver/engine.py | 23 +++++++++++++++---- src/cgis/resolver/indices.py | 9 -------- tests/unit/test_resolution_rules.py | 11 +++++++++ 6 files changed, 58 insertions(+), 32 deletions(-) rename src/cgis/{resolver/js_builtins.py => languages/typescript/builtins.py} (54%) diff --git a/src/cgis/resolver/js_builtins.py b/src/cgis/languages/typescript/builtins.py similarity index 54% rename from src/cgis/resolver/js_builtins.py rename to src/cgis/languages/typescript/builtins.py index 21fb4cb3..1c4901b0 100644 --- a/src/cgis/resolver/js_builtins.py +++ b/src/cgis/languages/typescript/builtins.py @@ -14,28 +14,15 @@ from cgis.core.js_globals import JS_BUILTINS_ROOT, JS_GLOBALS -#: Extensions whose calls are JavaScript at runtime. Matches the extractors -#: registered in `cgis.languages` for `.ts`/`.tsx`. -_JS_SUFFIXES: tuple[str, ...] = (".ts", ".tsx") - -def is_js_source(file_path: str | None) -> bool: - """True when `file_path` is a JavaScript/TypeScript source cgis extracts.""" - return file_path is not None and file_path.endswith(_JS_SUFFIXES) - - -def js_builtin_target( - raw_name: str, file_path: str | None, shadowed: frozenset[str] = frozenset() -) -> str | None: +def js_builtin_target(raw_name: str, shadowed: frozenset[str] = frozenset()) -> str | None: """The `js_builtins.*` FQN for a call on a JS global, or None when it is not one. - `file_path` is the calling source's file; without it the language is unknown - and nothing is rewritten. `shadowed` holds the globals that file imports or - declares itself — `import history from './history'`, `const confirm = ...`, a - parameter named `process` — which are the file's own values, not the runtime's. + `shadowed` holds the globals the calling file imports or declares itself — + `import history from './history'`, `const confirm = ...`, a parameter named + `process` — which are the file's own values, not the runtime's. + Only the TypeScript pack calls this, so the source is already JavaScript. """ - if not is_js_source(file_path): - return None root = raw_name.split(".", maxsplit=1)[0] if root not in JS_GLOBALS or root in shadowed: return None diff --git a/src/cgis/languages/typescript/index.py b/src/cgis/languages/typescript/index.py index 9e6e50cc..c9fe8080 100644 --- a/src/cgis/languages/typescript/index.py +++ b/src/cgis/languages/typescript/index.py @@ -51,6 +51,8 @@ class TypeScriptIndexFacts: #: normalized file_path -> {local name: package symbol FQN} for that file's bare #: imports: `useState` -> `react.useState`, a namespace import `Icons` -> `lucide-react`. file_bare_bindings: Mapping[str, dict[str, str]] + #: normalized file_path -> JS globals the file imports or declares itself (#111). + file_shadowed_globals: Mapping[str, frozenset[str]] #: project directory -> its tsconfig `paths` aliases, exact patterns first and then #: wildcards longest prefix first — the order TypeScript picks one in (#508). path_aliases: Mapping[str, tuple[PathAlias, ...]] @@ -70,6 +72,13 @@ def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> TypeS layout_prefixes=frozenset(), npm_dependencies=frozenset(cast("list[str]", state.get("npm_dependencies", []))), file_bare_imports=MappingProxyType(bare), + file_shadowed_globals=MappingProxyType( + { + os.path.normpath(n.file_path): frozenset(n.metadata["shadowed_globals"]) + for n in nodes.values() + if n.type == NodeType.FILE and n.metadata.get("shadowed_globals") + } + ), file_bare_bindings=MappingProxyType( { os.path.normpath(n.file_path): dict(n.metadata.get("bare_bindings") or {}) diff --git a/src/cgis/languages/typescript/resolution.py b/src/cgis/languages/typescript/resolution.py index 4a5dfd20..43ae85f9 100644 --- a/src/cgis/languages/typescript/resolution.py +++ b/src/cgis/languages/typescript/resolution.py @@ -11,6 +11,7 @@ from cgis.core.models import Node from cgis.core.resolution import AFTER_IMPORT_MAP, ResolutionRules, ResolutionView +from cgis.languages.typescript.builtins import js_builtin_target from cgis.languages.typescript.index import ( _ALIAS_WILDCARD, PathAlias, @@ -139,7 +140,21 @@ def package_call(name: str, file_path: str, view: ResolutionView) -> str | None: return candidate +#: What a call rewritten to the JS runtime scores: the unresolved value a Python +#: builtin gets, not the 1.0 of a resolved call (#111). +_FALLBACK_CONFIDENCE = 0.8 + + +def fallback(name: str, file_path: str, view: ResolutionView) -> tuple[str, float] | None: + """A call on a JS runtime global the file does not shadow: `js_builtins.` (#111).""" + facts = cast("TypeScriptIndexFacts", view.facts["typescript"]) + shadowed = facts.file_shadowed_globals.get(file_path, frozenset()) + target = js_builtin_target(name, shadowed) + return None if target is None else (target, _FALLBACK_CONFIDENCE) + + RULES = ResolutionRules( steps=((AFTER_IMPORT_MAP, package_call),), + fallback=fallback, reconcile_import=reconcile_import, ) diff --git a/src/cgis/resolver/engine.py b/src/cgis/resolver/engine.py index 2d7b01b8..65fabb3c 100644 --- a/src/cgis/resolver/engine.py +++ b/src/cgis/resolver/engine.py @@ -12,7 +12,8 @@ NodeNamespace, NodeType, ) -from cgis.resolver.js_builtins import js_builtin_target +from cgis.core.resolution import ResolutionView +from cgis.languages import rules_for from cgis.resolver.symbols import SymbolResolver, dispatch_edges RAW_DEP_PREFIX = "raw_dep:" @@ -46,6 +47,19 @@ def _agree_on_confidence(edges: list[Edge], virtual_nodes: dict[str, Node]) -> l ] +def _fallback(view: ResolutionView, name: str, file_path: str | None) -> tuple[str, float] | None: + """The source pack's fallback for a call nothing resolved: (target, confidence), or None. + + No file means no language, so nothing is rewritten — as before step 4. + """ + if not file_path: + return None + rules = rules_for(file_path) + if rules is None or rules.fallback is None: + return None + return rules.fallback(name, file_path, view) + + class ResolverEngine: """ The 'Brain' of the CGIS. @@ -170,10 +184,9 @@ def _resolved_call_edge(self, edge: Edge) -> Edge: # Fallback only: a project symbol named like a global has already won above. # Confidence stays at the unresolved 0.8, the value a Python builtin gets. file_path = self._index.normalized_file_path(edge.source, edge.file_path) - shadowed = self._index.file_shadowed_globals.get(file_path or "", frozenset()) - new_target = js_builtin_target(raw_name, file_path, shadowed) - if new_target is not None: - return edge.model_copy(update={"target": new_target, "confidence": 0.8}) + hit = _fallback(self._index, raw_name, file_path) + if hit is not None: + return edge.model_copy(update={"target": hit[0], "confidence": hit[1]}) final_target = new_target or raw_name confidence = min(edge.confidence + 0.5, 1.0) if new_target else 0.8 return edge.model_copy(update={"target": final_target, "confidence": confidence}) diff --git a/src/cgis/resolver/indices.py b/src/cgis/resolver/indices.py index 9d51dd3d..2d54918c 100644 --- a/src/cgis/resolver/indices.py +++ b/src/cgis/resolver/indices.py @@ -47,9 +47,6 @@ class SymbolIndex: file_variable_symbols: Mapping[tuple[str, str], list[str]] # normalized file_path -> {local_alias: target_fqn} (from FILE node import_map) file_imports: Mapping[str, dict[str, str]] - # normalized file_path -> JS globals the file imports or declares itself - # (from TypeScript FILE node metadata; read by the js_builtins fallback, #111). - file_shadowed_globals: Mapping[str, frozenset[str]] # module_fqn -> {name: definer_fqn}: names a module forwards rather than defines, # from explicit re-exports and expanded star imports (#417). reexports: Mapping[str, dict[str, str]] @@ -312,7 +309,6 @@ def build( variable_symbols: dict[str, list[str]] = {} file_variable_symbols: dict[tuple[str, str], list[str]] = {} file_imports: dict[str, dict[str, str]] = {} - file_shadowed_globals: dict[str, frozenset[str]] = {} reexports: dict[str, dict[str, str]] = {} star_imports: dict[str, list[str]] = {} suffix_map: dict[str, list[str]] = {} @@ -327,10 +323,6 @@ def build( reexports, star_imports, ) - if node.type == NodeType.FILE and node.metadata.get("shadowed_globals"): - file_shadowed_globals[os.path.normpath(node.file_path)] = frozenset( - node.metadata["shadowed_globals"] - ) # Index global functions/symbols if node.type in (NodeType.FUNCTION, NodeType.CLASS): @@ -372,7 +364,6 @@ def build( variable_symbols=MappingProxyType(variable_symbols), file_variable_symbols=MappingProxyType(file_variable_symbols), file_imports=MappingProxyType(file_imports), - file_shadowed_globals=MappingProxyType(file_shadowed_globals), reexports=MappingProxyType(reexports), suffix_map=MappingProxyType(suffix_map), internal_roots=roots, diff --git a/tests/unit/test_resolution_rules.py b/tests/unit/test_resolution_rules.py index 0ba81be9..1fa5d5f3 100644 --- a/tests/unit/test_resolution_rules.py +++ b/tests/unit/test_resolution_rules.py @@ -70,3 +70,14 @@ def test_package_call_is_the_typescript_packs_step() -> None: rules = rules_for("app/x.ts") assert rules is not None assert [position for position, _ in rules.steps] == [AFTER_IMPORT_MAP] + + +def test_js_builtins_are_the_typescript_packs_fallback() -> None: + """Spec: the fallback carries its confidence — 0.8 for a JS runtime global, not 1.0.""" + ts, py = rules_for("app/x.ts"), rules_for("app/x.py") + assert ts is not None + assert ts.fallback is not None + assert py is not None + assert py.fallback is None + view = IndexBuilder().build([]) + assert ts.fallback("Math.max", "app/x.ts", view) == ("js_builtins.Math.max", 0.8) From b0cf760040c9fc55024b17e9f7d0117cd51f5dfa Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 14:11:58 +0000 Subject: [PATCH 07/10] refactor(languages): Python's Annotated normalizer and star expansion are the Python pack's _through_annotated_alias becomes the Python pack's type normalizer; the engine applies every pack's normalizers in manifest order (receiver_type_fqn has no source file; ruling D2). Explicit re-exports and star-import expansion move into PythonIndexFacts.reexports; SymbolIndex.reexports is the merge of every pack's (a module two packs both forward is an error) and stays generic, since follow_reexport serves every language (D1). Co-Authored-By: Claude Opus 5.5 --- src/cgis/core/index.py | 5 + src/cgis/languages/python/index.py | 101 +++++++++++++++++++ src/cgis/languages/python/resolution.py | 23 ++++- src/cgis/languages/typescript/index.py | 3 + src/cgis/resolver/indices.py | 123 ++++-------------------- src/cgis/resolver/symbols.py | 34 ++----- tests/unit/test_python_index.py | 9 ++ tests/unit/test_resolution_rules.py | 31 ++++++ 8 files changed, 198 insertions(+), 131 deletions(-) diff --git a/src/cgis/core/index.py b/src/cgis/core/index.py index 5545a4ac..1c620a4a 100644 --- a/src/cgis/core/index.py +++ b/src/cgis/core/index.py @@ -20,6 +20,11 @@ def internal_roots(self) -> frozenset[str]: """Roots this pack adds to the language-free set (first segment of every node id).""" ... + @property + def reexports(self) -> Mapping[str, Mapping[str, str]]: + """module_fqn -> {name: definer_fqn}: the names a module forwards (#417).""" + ... + @property def layout_prefixes(self) -> frozenset[str]: """Prefixes sources write that node ids do not carry (#494): stripped off a target.""" diff --git a/src/cgis/languages/python/index.py b/src/cgis/languages/python/index.py index 8262b981..3faafbfe 100644 --- a/src/cgis/languages/python/index.py +++ b/src/cgis/languages/python/index.py @@ -9,6 +9,7 @@ import sys from collections.abc import Iterable, Mapping from dataclasses import dataclass +from types import MappingProxyType from typing import cast from cgis.core.index import ClassifyContext @@ -141,6 +142,9 @@ class PythonIndexFacts: dependency_roots: frozenset[str] #: `root.second` heads our own nodes have under those roots. dependency_heads: frozenset[str] + #: module_fqn -> {name: definer_fqn}: names a module forwards rather than defines, + #: from explicit re-exports and expanded star imports (#417). + reexports: Mapping[str, Mapping[str, str]] def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> PythonIndexFacts: @@ -149,6 +153,8 @@ def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> Pytho local_roots: set[str] = set() layout_roots: set[str] = set() file_imports: dict[str, dict[str, str]] = {} + reexports: dict[str, dict[str, str]] = {} + star_imports: dict[str, list[str]] = {} for node in nodes.values(): parts = node.id.split(".") local_roots.add(parts[0]) @@ -156,7 +162,12 @@ def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> Pytho layout_roots.add(parts[1]) if node.type == NodeType.FILE: file_imports[os.path.normpath(node.file_path)] = node.metadata.get("import_map") or {} + if node.metadata.get("reexports"): + reexports[node.id] = dict(node.metadata["reexports"]) + if node.metadata.get("star_imports"): + star_imports[node.id] = list(node.metadata["star_imports"]) external, first_party, corroborated = _split_import_roots(file_imports, node_ids) + _expand_star_imports(star_imports, reexports, node_ids) declared = set(cast("list[str]", state.get("python_dependencies", []))) # Before first-party prefixes join: the set the old builder intersected (#495). colliding = (local_roots | layout_roots) & declared @@ -167,6 +178,7 @@ def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> Pytho external_roots=frozenset(external), dependency_roots=frozenset(colliding), dependency_heads=_heads_under(colliding, node_ids), + reexports=MappingProxyType(reexports), ) @@ -185,3 +197,92 @@ def classify(fqn: str, ctx: ClassifyContext) -> NodeNamespace: if root in facts.external_roots: return NodeNamespace.EXTERNAL return NodeNamespace.UNKNOWN + + +def _public_names_under(module_fqn: str, node_ids: set[str]) -> dict[str, str]: + """The symbols `from import *` would bring in directly. + + Only names defined immediately under the module: a dotted remainder is a + symbol inside one of its children, and a leading underscore is private — + `import *` takes neither. + + The module is tried verbatim and then with leading segments stripped. A + star import written absolutely (`from app.tests.fixtures import *`) + records the path as the author typed it, while node ids are + `tests.fixtures` when the project was ingested at app/ — the same prefix + mismatch that made the first version of this feature resolve nothing, and + it reaches the star sources as well as the map keys. + """ + parts = module_fqn.split(".") + for i in range(len(parts)): + prefix = ".".join(parts[i:]) + "." + names: dict[str, str] = {} + for node_id in node_ids: + if not node_id.startswith(prefix): + continue + name = node_id[len(prefix) :] + if "." not in name and not name.startswith("_"): + names.setdefault(name, node_id) + if names: + return names + return {} + + +def _close_over_sources( + by_module: dict[str, dict[str, str]], + star_imports: dict[str, list[str]], + reexports: dict[str, dict[str, str]], +) -> None: + """Pull each source's own forwarded names in, repeatedly, until stable. + + A star import of a package that itself re-exports needs those names too: + `app/__init__` starring `app.models`, which stars `app.models.rating`, + must reach Rating. Repeating terminates because names are only ever + added, never removed. + """ + changed = True + while changed: + changed = False + for module_fqn, sources in star_imports.items(): + target = by_module[module_fqn] + for source in sources: + inherited = {**reexports.get(source, {}), **by_module.get(source, {})} + for name, definer in inherited.items(): + if name not in target: + target[name] = definer + changed = True + + +def _expand_star_imports( + star_imports: dict[str, list[str]], + reexports: dict[str, dict[str, str]], + node_ids: set[str], +) -> None: + """Turn `from X import *` into the names X actually exports. + + The extractor cannot do this: it sees one file, and the names belong to + the *exporting* module. Here every node is known. + + `__all__` is not read. Where a module has one, the difference is a module + deliberately hiding a public name — rare, and the extra entry is inert: + it only ever fires if some file actually imports that name. + + An explicit re-export already recorded wins over a starred one: a module + that both stars a submodule and names a symbol explicitly meant the + explicit one. + """ + by_module = { + module_fqn: { + name: definer + for source in sources + for name, definer in _public_names_under(source, node_ids).items() + } + for module_fqn, sources in star_imports.items() + } + _close_over_sources(by_module, star_imports, reexports) + for module_fqn, names in by_module.items(): + if not names: + continue + existing = reexports.setdefault(module_fqn, {}) + for name, definer in names.items(): + existing.setdefault(name, definer) diff --git a/src/cgis/languages/python/resolution.py b/src/cgis/languages/python/resolution.py index ac485764..84b3b5dc 100644 --- a/src/cgis/languages/python/resolution.py +++ b/src/cgis/languages/python/resolution.py @@ -8,4 +8,25 @@ def reconcile_import(fqn: str, _source_file: str | None, view: ResolutionView) - return view.strip_layout_prefixes(fqn) -RULES = ResolutionRules(reconcile_import=reconcile_import) +def annotated_alias(type_fqn: str, view: ResolutionView) -> str: + """`T` when `type_fqn` is an `Annotated[T, Depends(...)]` DI alias, else `type_fqn`. + + A parameter typed `service: ServiceDep` holds a `Service`, but its recorded + type is the alias, which has no methods — so `service.run()` stayed + unresolved and impact stopped short of every FastAPI route injected that way + (#550). Read for a parameter's calls and for `self.` alike. + Moved from `cgis.resolver.symbols` in language-packs step 4: the Python + pack's type normalizer, which the engine applies to every declared type. + """ + alias_fqn = view.map_to_node_fqn(type_fqn) or type_fqn + if not view.is_variable_node(alias_fqn): + return type_fqn + annotated: str | None = view.nodes[alias_fqn].metadata.get("annotated_type") + if not annotated: + return type_fqn + # Recorded through the alias module's import map, so it may carry a layout + # prefix (`app.domains.x.T` against a node `domains.x.T`). + return view.map_to_node_fqn(annotated) or annotated + + +RULES = ResolutionRules(type_normalizers=(annotated_alias,), reconcile_import=reconcile_import) diff --git a/src/cgis/languages/typescript/index.py b/src/cgis/languages/typescript/index.py index c9fe8080..52659f95 100644 --- a/src/cgis/languages/typescript/index.py +++ b/src/cgis/languages/typescript/index.py @@ -40,6 +40,8 @@ class TypeScriptIndexFacts: internal_roots: frozenset[str] layout_prefixes: frozenset[str] + #: TypeScript records no forwarded names (yet); always empty. + reexports: Mapping[str, Mapping[str, str]] #: npm packages the manifests declare, as written, less our workspace packages. npm_dependencies: frozenset[str] #: normalized file_path -> the bare specifiers that file imports, as written. @@ -70,6 +72,7 @@ def index_facts(nodes: Mapping[str, Node], state: Mapping[str, object]) -> TypeS return TypeScriptIndexFacts( internal_roots=frozenset(), layout_prefixes=frozenset(), + reexports=MappingProxyType({}), npm_dependencies=frozenset(cast("list[str]", state.get("npm_dependencies", []))), file_bare_imports=MappingProxyType(bare), file_shadowed_globals=MappingProxyType( diff --git a/src/cgis/resolver/indices.py b/src/cgis/resolver/indices.py index 2d54918c..d76bce95 100644 --- a/src/cgis/resolver/indices.py +++ b/src/cgis/resolver/indices.py @@ -277,6 +277,18 @@ def normalized_file_path(self, source_fqn: str, edge_file_path: str | None) -> s _ROOT_PACKAGE = "__init__" +def _merged_reexports(facts: Mapping[str, IndexFacts]) -> dict[str, dict[str, str]]: + """Every pack's forwarded names; a module two packs both forward is an error.""" + merged: dict[str, dict[str, str]] = {} + for pack_facts in facts.values(): + for module_fqn, names in pack_facts.reexports.items(): + if module_fqn in merged: + msg = f"two packs forward names from {module_fqn}" + raise ValueError(msg) + merged[module_fqn] = dict(names) + return merged + + class IndexBuilder: """Builds a SymbolIndex from extracted nodes (Phase 1: indexing). @@ -309,8 +321,6 @@ def build( variable_symbols: dict[str, list[str]] = {} file_variable_symbols: dict[tuple[str, str], list[str]] = {} file_imports: dict[str, dict[str, str]] = {} - reexports: dict[str, dict[str, str]] = {} - star_imports: dict[str, list[str]] = {} suffix_map: dict[str, list[str]] = {} internal_roots: set[str] = set() @@ -320,8 +330,6 @@ def build( node, file_imports, self_types, - reexports, - star_imports, ) # Index global functions/symbols @@ -351,7 +359,7 @@ def build( facts = pack_index_facts(MappingProxyType(nodes_by_id), state) roots = frozenset(internal_roots.union(*(f.internal_roots for f in facts.values()))) prefixes = frozenset[str]().union(*(f.layout_prefixes for f in facts.values())) - self._expand_star_imports(star_imports, reexports, set(nodes_by_id)) + reexports = _merged_reexports(facts) # Read-only views, not copies: MappingProxyType wraps in O(1), so this # buys write protection without touching ingest cost (#183). @@ -371,120 +379,23 @@ def build( facts=MappingProxyType(facts), ) - @staticmethod - def _public_names_under(module_fqn: str, node_ids: set[str]) -> dict[str, str]: - """The symbols `from import *` would bring in directly. - - Only names defined immediately under the module: a dotted remainder is a - symbol inside one of its children, and a leading underscore is private — - `import *` takes neither. - - The module is tried verbatim and then with leading segments stripped. A - star import written absolutely (`from app.tests.fixtures import *`) - records the path as the author typed it, while node ids are - `tests.fixtures` when the project was ingested at app/ — the same prefix - mismatch that made the first version of this feature resolve nothing, and - it reaches the star sources as well as the map keys. - """ - parts = module_fqn.split(".") - for i in range(len(parts)): - prefix = ".".join(parts[i:]) + "." - names: dict[str, str] = {} - for node_id in node_ids: - if not node_id.startswith(prefix): - continue - name = node_id[len(prefix) :] - if "." not in name and not name.startswith("_"): - names.setdefault(name, node_id) - if names: - return names - return {} - - @staticmethod - def _close_over_sources( - by_module: dict[str, dict[str, str]], - star_imports: dict[str, list[str]], - reexports: dict[str, dict[str, str]], - ) -> None: - """Pull each source's own forwarded names in, repeatedly, until stable. - - A star import of a package that itself re-exports needs those names too: - `app/__init__` starring `app.models`, which stars `app.models.rating`, - must reach Rating. Repeating terminates because names are only ever - added, never removed. - """ - changed = True - while changed: - changed = False - for module_fqn, sources in star_imports.items(): - target = by_module[module_fqn] - for source in sources: - inherited = {**reexports.get(source, {}), **by_module.get(source, {})} - for name, definer in inherited.items(): - if name not in target: - target[name] = definer - changed = True - - @classmethod - def _expand_star_imports( - cls, - star_imports: dict[str, list[str]], - reexports: dict[str, dict[str, str]], - node_ids: set[str], - ) -> None: - """Turn `from X import *` into the names X actually exports. - - The extractor cannot do this: it sees one file, and the names belong to - the *exporting* module. Here every node is known. - - `__all__` is not read. Where a module has one, the difference is a module - deliberately hiding a public name — rare, and the extra entry is inert: - it only ever fires if some file actually imports that name. - - An explicit re-export already recorded wins over a starred one: a module - that both stars a submodule and names a symbol explicitly meant the - explicit one. - """ - by_module = { - module_fqn: { - name: definer - for source in sources - for name, definer in cls._public_names_under(source, node_ids).items() - } - for module_fqn, sources in star_imports.items() - } - cls._close_over_sources(by_module, star_imports, reexports) - for module_fqn, names in by_module.items(): - if not names: - continue - existing = reexports.setdefault(module_fqn, {}) - for name, definer in names.items(): - existing.setdefault(name, definer) - @staticmethod def _index_metadata( node: Node, file_imports: dict[str, dict[str, str]], self_types: dict[str, dict[str, str]], - reexports: dict[str, dict[str, str]], - star_imports: dict[str, list[str]], ) -> None: """Read the maps the extractor attached to a FILE or CLASS node. - A FILE carries its import map, re-exports and star imports; a CLASS the - attribute types recorded from annotations (spec D1). A TypeScript FILE's - bare imports and bindings are the TypeScript pack's index facts (step 4). + A FILE carries its import map; a CLASS the attribute types recorded from + annotations (spec D1). Re-exports and star imports are the Python pack's, + and a TypeScript FILE's bare imports and bindings the TypeScript pack's, + index facts (step 4). All are extractor output the resolver consumes rather than derives. """ if node.type == NodeType.FILE: path = os.path.normpath(node.file_path) file_imports[path] = node.metadata.get("import_map") or {} - forwarded = node.metadata.get("reexports") - if forwarded: - reexports[node.id] = dict(forwarded) - wildcards = node.metadata.get("star_imports") - if wildcards: - star_imports[node.id] = list(wildcards) elif node.type == NodeType.CLASS: declared = node.metadata.get("self_types") if declared: diff --git a/src/cgis/resolver/symbols.py b/src/cgis/resolver/symbols.py index 55064ea1..64bbaad7 100644 --- a/src/cgis/resolver/symbols.py +++ b/src/cgis/resolver/symbols.py @@ -4,7 +4,7 @@ from cgis.core.models import RAW_CLASS_PREFIX, Edge, EdgeType, Node, NodeNamespace from cgis.core.resolution import AFTER_IMPORT_MAP -from cgis.languages import rules_for +from cgis.languages import rules_for, type_normalizers from cgis.resolver.indices import IndexBuilder, SymbolIndex @@ -35,6 +35,13 @@ def _positioned(index: SymbolIndex, position: str, name: str, file_path: str) -> return None +def _normalized(index: SymbolIndex, type_fqn: str) -> str: + """`type_fqn` through every pack's type normalizers, in manifest order (step 4, D2).""" + for normalize in type_normalizers(): + type_fqn = normalize(type_fqn, index) + return type_fqn + + def receiver_type_fqn(index: SymbolIndex, owner_fqn: str, attr: str) -> str | None: """Resolve `self.` on class `owner_fqn` to the type it was declared as. @@ -50,7 +57,7 @@ def receiver_type_fqn(index: SymbolIndex, owner_fqn: str, attr: str) -> str | No return None # `self.svc = svc` with `svc: ServiceDep` declares the alias, not the service # behind it — the same gap as a parameter's own calls (#550 review). - return _through_annotated_alias(index, index.map_to_node_fqn(declared) or declared) + return _normalized(index, index.map_to_node_fqn(declared) or declared) def _declaring_class( @@ -272,7 +279,7 @@ def _resolve_local_type_call(self, name: str, source_node: Node) -> str | None: class_fqn = local_types.get(var_name) if not class_fqn: return None - class_fqn = _through_annotated_alias(self.index, class_fqn) + class_fqn = _normalized(self.index, class_fqn) candidate = f"{class_fqn}.{method_name}" resolved = self.index.map_to_node_fqn(candidate) if resolved: @@ -299,27 +306,6 @@ def _resolve_via_global_symbols(self, name: str, file_path: str | None) -> str | return None -def _through_annotated_alias(index: SymbolIndex, type_fqn: str) -> str: - """`T` when `type_fqn` is an `Annotated[T, Depends(...)]` DI alias, else `type_fqn`. - - A parameter typed `service: ServiceDep` holds a `Service`, but its recorded - type is the alias, which has no methods — so `service.run()` stayed - unresolved and impact stopped short of every FastAPI route injected that way - (#550). Read for a parameter's calls and for `self.` alike. - Module-level, as `_package_call`: `SymbolResolver` is at the god-object - threshold. - """ - alias_fqn = index.map_to_node_fqn(type_fqn) or type_fqn - if not index.is_variable_node(alias_fqn): - return type_fqn - annotated: str | None = index.nodes[alias_fqn].metadata.get("annotated_type") - if not annotated: - return type_fqn - # Recorded through the alias module's import map, so it may carry a layout - # prefix (`app.domains.x.T` against a node `domains.x.T`). - return index.map_to_node_fqn(annotated) or annotated - - # --- DISPATCHES (#545) ------------------------------------------------------- # # A call resolves to the method its receiver's declared type has, so a call through diff --git a/tests/unit/test_python_index.py b/tests/unit/test_python_index.py index b4ccfcd6..25d9d706 100644 --- a/tests/unit/test_python_index.py +++ b/tests/unit/test_python_index.py @@ -89,3 +89,12 @@ def test_classification_chain() -> None: assert classify("len", ctx) is NodeNamespace.STDLIB assert classify("js_builtins.Math", ctx) is NodeNamespace.STDLIB assert classify("nothing.known", ctx) is NodeNamespace.UNKNOWN + + +def test_star_imports_expand_to_the_public_names_under_the_source() -> None: + """`from pkg.mod import *` forwards `A`, never the private `_b` (#417).""" + file_node = _node( + "pkg.app", NodeType.FILE, file_path="pkg/app.py", metadata={"star_imports": ["pkg.mod"]} + ) + facts = _facts([file_node, _node("pkg.mod.A"), _node("pkg.mod._b")]) + assert facts.reexports == {"pkg.app": {"A": "pkg.mod.A"}} diff --git a/tests/unit/test_resolution_rules.py b/tests/unit/test_resolution_rules.py index 1fa5d5f3..4b819f1d 100644 --- a/tests/unit/test_resolution_rules.py +++ b/tests/unit/test_resolution_rules.py @@ -81,3 +81,34 @@ def test_js_builtins_are_the_typescript_packs_fallback() -> None: assert py.fallback is None view = IndexBuilder().build([]) assert ts.fallback("Math.max", "app/x.ts", view) == ("js_builtins.Math.max", 0.8) + + +def test_the_annotated_alias_is_pythons_normalizer() -> None: + rules = rules_for("app/x.py") + assert rules is not None + assert len(rules.type_normalizers) == 1 + assert type_normalizers() == rules.type_normalizers + + +def test_a_normalizer_applies_without_a_source() -> None: + """receiver_type_fqn has no source file: ServiceDep still reads as Service (#550, D2).""" + alias = Node( + id="app.deps.ServiceDep", + type=NodeType.VARIABLE, + name="ServiceDep", + file_path="app/deps.py", + start_line=1, + end_line=1, + metadata={"annotated_type": "app.svc.Service"}, + ) + service = Node( + id="app.svc.Service", + type=NodeType.CLASS, + name="Service", + file_path="app/svc.py", + start_line=1, + end_line=1, + ) + index = IndexBuilder().build([alias, service]) + (normalize,) = type_normalizers() + assert normalize("app.deps.ServiceDep", index) == "app.svc.Service" From ed57fd305e8d20988f592a93dbb75ec119ec07dc Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 14:17:32 +0000 Subject: [PATCH 08/10] docs: step 4 implementation notes; inventory Measured on the branch (main f0367de alongside): - resolution self-drift 0.386 -> 0.180, unresolved 0.444 -> 0.440 (lock 0.40/0.45): the domain's imports are now the single chain engine -> symbols -> indices. The mid-step hygiene breach (0.459 after Task 3) closed by itself once the Python rules left; the re-baseline agreed for the end of the step was not needed. - owner-api ingest, median of three: main 21.26 s, branch 21.74 s (+2.2%) - review-path inventory gains core/resolution.py and the two packs' resolution modules and the TS builtins. - `git grep` of src/cgis/resolver for suffixes, pack names and js_builtins: none. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 5 ++++- docs/specs/2026-10-14-language-packs-design.md | 14 ++++++++++++++ tests/unit/review_path_inventory.txt | 4 ++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 5abf7d20..4ccb3d8c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,7 +76,10 @@ IngestionPipeline (pipeline.py) `target="raw_call:"`, set `Node.language = ""`). 2. In its `__init__.py`, define `PACK`, subclassing `cgis.core.extractor.LanguagePack` (mark methods `@override`) (`name`, `extensions`, `code_fence`, `is_test`, `scope_of`, `extractor`, `project_reader`, - `index_facts`, `classify`). `classify` answers for references from the pack's own files and + `index_facts`, `classify`, `resolution`). `resolution()` returns `ResolutionRules` + (`cgis.core.resolution`): type normalizers, steps at a position (`after:import_map`), a + fallback with its confidence, `reconcile_import` — all reading the index as `ResolutionView`. + Metadata other files resolve against goes in the manifest's `cross_file_metadata`. `classify` answers for references from the pack's own files and runs the whole chain; a source with no pack gets the generic tail in `SymbolIndex.classify_fqn`. 3. Add one `Manifest` to `MANIFESTS` in `src/cgis/languages/__init__.py`, with a `load` function that imports the pack statically inside it (lazy, yet visible to the review-fingerprint walk). Nothing diff --git a/docs/specs/2026-10-14-language-packs-design.md b/docs/specs/2026-10-14-language-packs-design.md index b7aea6cc..076aaa03 100644 --- a/docs/specs/2026-10-14-language-packs-design.md +++ b/docs/specs/2026-10-14-language-packs-design.md @@ -263,6 +263,20 @@ The generic chain, with today's positions of language logic: `reconcile_import` takes the language-split half of `resolve_import_target` (indices.py:141–146). Step 4 of the migration extracts these one at a time. +> **Implementation note (step 4):** the contract is `cgis.core.resolution` — +> `ResolutionRules` (normalizers, steps at validated positions, a fallback +> returning `(target, confidence)`, `reconcile_import`) and `ResolutionView`, the +> protocol a rule reads the index through, so packs never import the resolver +> (ruling D4). `layout_prefixes`, `reexports` and `suffix_map` stay generic on +> `SymbolIndex` — `map_to_node_fqn`/`follow_reexport` serve every language — but +> the packs compute them; only the TS-only facts left it (D1). Every pack's type +> normalizers apply, in manifest order, because `receiver_type_fqn` has no source +> (D2). The `engine-packs` allowlist is empty: `cgis.resolver` names no language. +> The owner-api ingest moved +2.2% (median of three). One measured lesson: typing +> an engine helper with `SymbolIndex` instead of `ResolutionView` added an +> engine→indices import that turned the domain's chain into a triangle and its +> self-drift from 0.38 to 0.59; through the protocol it is 0.18. + **Incremental ingest.** `cross_file_metadata` lists, per node type, the metadata keys other files resolve against. It is declared in the pack's registry manifest, not read from the pack module, so collecting it imports diff --git a/tests/unit/review_path_inventory.txt b/tests/unit/review_path_inventory.txt index 7edb5352..a88e1d14 100644 --- a/tests/unit/review_path_inventory.txt +++ b/tests/unit/review_path_inventory.txt @@ -7,6 +7,7 @@ src/cgis/core/js_globals.py src/cgis/core/models.py src/cgis/core/paths.py src/cgis/core/project.py +src/cgis/core/resolution.py src/cgis/guardian/__init__.py src/cgis/guardian/axes.py src/cgis/guardian/chunked.py @@ -40,13 +41,16 @@ src/cgis/languages/python/dependencies.py src/cgis/languages/python/extractor.py src/cgis/languages/python/index.py src/cgis/languages/python/project.py +src/cgis/languages/python/resolution.py src/cgis/languages/typescript/__init__.py src/cgis/languages/typescript/_bindings.py +src/cgis/languages/typescript/builtins.py src/cgis/languages/typescript/extractor.py src/cgis/languages/typescript/index.py src/cgis/languages/typescript/naming.py src/cgis/languages/typescript/npm.py src/cgis/languages/typescript/project.py +src/cgis/languages/typescript/resolution.py src/cgis/languages/typescript/tsconfig.py src/cgis/languages/typescript/workspaces.py src/cgis/query/analysis/__init__.py From e56814159b6dd2306b7c135c81005948edb47b68 Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 14:27:56 +0000 Subject: [PATCH 09/10] docs(spec): step 4 note says what moved, and what language knowledge remains The note claimed the packs compute suffix_map (it is still built generically) and that cgis.resolver "names no language" (true of its imports, not its text: the Python __init__ root rule, the JS-builtins root in the generic classify tail, and _is_dunder remain for step 5). Final review, minor 3. Co-Authored-By: Claude Opus 5.5 --- docs/specs/2026-10-14-language-packs-design.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/specs/2026-10-14-language-packs-design.md b/docs/specs/2026-10-14-language-packs-design.md index 076aaa03..760f8f0d 100644 --- a/docs/specs/2026-10-14-language-packs-design.md +++ b/docs/specs/2026-10-14-language-packs-design.md @@ -268,10 +268,15 @@ The generic chain, with today's positions of language logic: > returning `(target, confidence)`, `reconcile_import`) and `ResolutionView`, the > protocol a rule reads the index through, so packs never import the resolver > (ruling D4). `layout_prefixes`, `reexports` and `suffix_map` stay generic on -> `SymbolIndex` — `map_to_node_fqn`/`follow_reexport` serve every language — but -> the packs compute them; only the TS-only facts left it (D1). Every pack's type +> `SymbolIndex` — `map_to_node_fqn`/`follow_reexport` serve every language; the +> packs compute `layout_prefixes` and `reexports` (Python's star expansion), while +> `suffix_map` is still built generically. Only the TS-only facts left it (D1). Every pack's type > normalizers apply, in manifest order, because `receiver_type_fqn` has no source -> (D2). The `engine-packs` allowlist is empty: `cgis.resolver` names no language. +> (D2). The `engine-packs` allowlist is empty: `cgis.resolver` imports no language +> pack. Some language knowledge remains in its text, for step 5 to weigh: the +> Python `__init__` root rule inside `strip_layout_prefixes` (reached from the +> generic `resolve_layout`), the JS-builtins root in `classify_fqn`'s generic tail, +> and `_is_dunder` in DISPATCHES. > The owner-api ingest moved +2.2% (median of three). One measured lesson: typing > an engine helper with `SymbolIndex` instead of `ResolutionView` added an > engine→indices import that turned the domain's chain into a triangle and its From 15af3818098d35851818feb08b3013c0c0edda16 Mon Sep 17 00:00:00 2001 From: zaebee Date: Wed, 14 Oct 2026 14:38:52 +0000 Subject: [PATCH 10/10] perf(languages): type_normalizers() is collected once The engine asks for the normalizers on every declared type, and the packs' rules never change once loaded; the tuple was rebuilt on each call (~4.6 ms on owner-api). Gemini. Co-Authored-By: Claude Opus 5.5 --- src/cgis/languages/__init__.py | 7 ++++++- tests/unit/test_resolution_rules.py | 5 +++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/src/cgis/languages/__init__.py b/src/cgis/languages/__init__.py index 2be99d1e..44f05e57 100644 --- a/src/cgis/languages/__init__.py +++ b/src/cgis/languages/__init__.py @@ -168,8 +168,13 @@ def rules_for(source_file: str | None) -> ResolutionRules | None: return None if pack is None else _rules(pack.name) +@cache def type_normalizers() -> tuple[TypeNormalizer, ...]: - """Every pack's type normalizers, in manifest order (step 4, ruling D2).""" + """Every pack's type normalizers, in manifest order (step 4, ruling D2). + + Cached: the engine asks for them on every declared type, and the packs' + rules never change once loaded. + """ return tuple(n for m in MANIFESTS for n in _rules(m.name).type_normalizers) diff --git a/tests/unit/test_resolution_rules.py b/tests/unit/test_resolution_rules.py index 4b819f1d..aae94c26 100644 --- a/tests/unit/test_resolution_rules.py +++ b/tests/unit/test_resolution_rules.py @@ -112,3 +112,8 @@ def test_a_normalizer_applies_without_a_source() -> None: index = IndexBuilder().build([alias, service]) (normalize,) = type_normalizers() assert normalize("app.deps.ServiceDep", index) == "app.svc.Service" + + +def test_the_normalizers_are_collected_once() -> None: + """On the hot path (every declared type): the tuple is built once, not per call.""" + assert type_normalizers() is type_normalizers()