Skip to content

refactor(languages): step 4 — resolution rules per pack - #601

Merged
zaebee merged 10 commits into
mainfrom
feat/language-packs-step4
Oct 14, 2026
Merged

zaebee merged 10 commits into
mainfrom
feat/language-packs-step4

Conversation

@zaebee

@zaebee zaebee commented Oct 14, 2026

Copy link
Copy Markdown
Owner

Step 4 of the language-packs migration (spec docs/specs/2026-10-14-language-packs-design.md "Resolution" + "Incremental ingest", plan docs/plans/2026-10-14-language-packs-step4.md, part of #146). Every language branch of the resolver becomes a rule a pack contributes; cgis.resolver imports no language pack — the engine-packs allowlist is empty. No graph changes. One language branch per commit.

What moved

from to
pipeline._CROSS_FILE_METADATA Manifest.cross_file_metadata (Python declares today's table; the pipeline unions them; collecting it imports no pack)
— core/resolution.py → ResolutionView (protocol; SymbolIndex satisfies it, so packs never import the resolver), ResolutionRules (type normalizers, steps at validated positions, fallback → (target, confidence), reconcile_import); LanguagePack.resolution(); registry rules_for, type_normalizers
indices.resolve_import_target's TS half, PathAlias, _alias_candidates, _workspace_candidates, _resolve_typescript_import, _ordered_aliases languages/typescript/{index,resolution}.py — reconcile_import; facts gain workspace_packages, path_aliases
symbols._package_call TS package_call, the after:import_map step; facts gain file_bare_bindings
resolver/js_builtins.py languages/typescript/builtins.py; the TS fallback at 0.8; facts gain file_shadowed_globals
indices Python branch of resolve_import_target Python reconcile_import → strip_layout_prefixes (now public, part of the view)
symbols._through_annotated_alias Python annotated_alias type normalizer
IndexBuilder._expand_star_imports & co, explicit re-exports PythonIndexFacts.reexports; SymbolIndex.reexports = merge of the packs'

Identity and equivalence

  • graph_identity.py diff, main's cgis f0367de vs this branch: cgis src/ @715cd9a identical (2019 / 7858), owner-api @8de2245 identical (19115 / 96258), ui/ identical (286 / 497).
  • The final reviewer loaded f0367de's resolver into a scratch package and compared ResolverEngine(...).resolve() — every resolved edge (full dump incl. confidence), every virtual node, reexports, layout_prefixes, suffix_map, workspace/alias/binding/shadowed facts and the inheritance tree — on cgis src, ui/, owner-api and owner-web, plus a synthetic fixture with a tsconfig alias, a workspace package, a .d.ts, a bare import, a shadowed global and a Python Annotated DI alias used through self.: identical.

Measurements

  • owner-api ingest, median of three: main 21.26 s, branch 21.74 s (+2.2%, budget 5%); resolver alone +3.4% (≈0.1 s).
  • resolution self-drift 0.386 → 0.180, unresolved 0.444 → 0.440 (lock 0.40 / 0.45): the domain's imports are now the single chain engine → symbols → indices.

Two episodes, both in the commit messages:

  • After the TS reconciliation moved (Task 3) the resolver's unresolved ratio rose to 0.459 > 0.45 — a denominator effect (moved code carried more resolved calls) plus one dynamic rule call. I stopped as the plan required; the agreed answer was a single re-baseline at the end. Once the Python rules moved (Task 6) it fell to 0.440 by itself — nothing was re-baselined. Interim commits ran the gate with that one test deselected and a JSON check (itself tested against fabricated failures) that no other drift bound broke; squash-merging keeps them out of main.
  • Typing the engine's fallback helper with SymbolIndex imported indices into engine, turning the chain into a triangle: drift 0.594. Typed with core's ResolutionView (ruling D4) it is 0.180.

Rulings

  • D1 layout_prefixes, reexports, suffix_map stay generic on SymbolIndex (map_to_node_fqn/follow_reexport serve every language); packs compute layout_prefixes and reexports; only TS-only facts left the index.
  • D2 The engine applies every pack's type normalizers in manifest order (receiver_type_fqn has no source).
  • D3 Step positions are validated; an unknown one fails at pack load.
  • D4 Rules read a ResolutionView, never SymbolIndex.

Remaining language knowledge in cgis.resolver (text, not imports) — for step 5

The Python __init__ root rule inside strip_layout_prefixes (reached from generic resolve_layout), the JS-builtins root in classify_fqn's generic tail, _is_dunder in DISPATCHES. Recorded in the spec's step-4 note.

Deferred minors (final review)

  • Two stale comments/docstrings (symbols.py "beside _package_call", IndexBuilder.build on workspace ordering).
  • TS rules index view.facts["typescript"] directly — a hand-built SymbolIndex without it would KeyError (IndexBuilder always supplies it).
  • type_normalizers() rebuilds its tuple per call (~4.6 ms on owner-api).

Gate: format, lint, mypy strict, 3226 passed (drift tests included), docstring 99%.

🤖 Generated with Claude Code

zaebee and others added 9 commits October 14, 2026 13:37
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pipeline's incremental signature reads the union over the manifests; the
Python manifest carries today's table, TypeScript none, so the union is the old
table exactly — and collecting it imports no pack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ules)

cgis.core.resolution: the view a rule reads the index through (SymbolIndex
satisfies it; strip_layout_prefixes becomes public for it), and the rules a pack
contributes — type normalizers, steps at validated positions, a fallback with
its confidence, an import reconciler. Packs return empty rules until the next
commits move one language branch each. Registry: rules_for, type_normalizers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
resolve_import_target keeps the exact-id hit and hands the rest to the source
pack's reconcile_import: TypeScript's reads tsconfig aliases, workspace packages
and `.d` declarations from its own index facts; Python's strips corroborated
layout prefixes. SymbolIndex loses workspace_packages and path_aliases, and the
resolver its last suffix checks (_PYTHON_SUFFIXES, _TYPESCRIPT_SUFFIXES).

Self-drift: resolution hygiene unresolved_ratio 0.444 -> 0.459 (161 UNKNOWN of
351 calls, from 169 of 381): the moved TS helpers took more resolved calls than
unresolved ones out of the domain, and the rule call `rules.reconcile_import` is
one new dynamic dispatch. Re-baselined once at the end of step 4 (user decision).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ort_map step

resolve_global_call runs the source pack's steps at `after:import_map` through a
module-level helper (SymbolResolver is at the god-object threshold). The TS
step reads the file's bare bindings from the TS pack's index facts; SymbolIndex
loses file_bare_bindings. The resolver no longer imports a pack: the
engine-packs allowlist is empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
js_builtins.py moves into the TypeScript pack; the engine asks the source pack's
fallback for a call nothing resolved and takes its (target, confidence) — 0.8
for a JS runtime global, as before. A missing file still rewrites nothing. The
file's shadowed globals are TS index facts; SymbolIndex loses
file_shadowed_globals, and js_builtin_target its now-redundant language check.

The engine's fallback helper reads the index as core's ResolutionView, not as
SymbolIndex: importing cgis.resolver.indices into engine made an
engine→symbols→indices triangle that took resolution's self-drift from 0.384
to 0.594. Through the protocol the domain's imports are a single chain and the
drift is 0.177.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… are the Python pack's

_through_annotated_alias becomes the Python pack's type normalizer; the engine
applies every pack's normalizers in manifest order (receiver_type_fqn has no
source file; ruling D2). Explicit re-exports and star-import expansion move into
PythonIndexFacts.reexports; SymbolIndex.reexports is the merge of every pack's
(a module two packs both forward is an error) and stays generic, since
follow_reexport serves every language (D1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on the branch (main f0367de alongside):
- resolution self-drift 0.386 -> 0.180, unresolved 0.444 -> 0.440 (lock 0.40/0.45):
  the domain's imports are now the single chain engine -> symbols -> indices.
  The mid-step hygiene breach (0.459 after Task 3) closed by itself once the
  Python rules left; the re-baseline agreed for the end of the step was not needed.
- owner-api ingest, median of three: main 21.26 s, branch 21.74 s (+2.2%)
- review-path inventory gains core/resolution.py and the two packs' resolution
  modules and the TS builtins.
- `git grep` of src/cgis/resolver for suffixes, pack names and js_builtins: none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…remains

The note claimed the packs compute suffix_map (it is still built generically)
and that cgis.resolver "names no language" (true of its imports, not its text:
the Python __init__ root rule, the JS-builtins root in the generic classify
tail, and _is_dunder remain for step 5). Final review, minor 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements Step 4 of the Language Packs migration, successfully moving language-specific resolution rules (including type normalizers, positioned steps, fallbacks, and import reconcilers) out of the core resolver engine and into the TypeScript and Python language packs. This decoupling is achieved by introducing the cgis.core.resolution contract, allowing the engine-packs allowlist to be emptied. Feedback on these changes highlights a performance improvement opportunity to cache the type_normalizers() function on the hot path, as well as several reliability improvements to defensively use .get("typescript") on view.facts to prevent potential KeyErrors in the TypeScript resolution rules.

Comment on lines +171 to +173
def type_normalizers() -> tuple[TypeNormalizer, ...]:
"""Every pack's type normalizers, in manifest order (step 4, ruling D2)."""
return tuple(n for m in MANIFESTS for n in _rules(m.name).type_normalizers)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The type_normalizers() function is called on the hot path of resolution (e.g., inside _normalized in symbols.py). Currently, it rebuilds the tuple of normalizers on every single call, which can be a performance bottleneck (taking ~4.6 ms on large codebases like owner-api as noted in the PR description). Since the manifests and rules are static/cached, we can cache the result of type_normalizers() using @cache to avoid this overhead.

Suggested change
def type_normalizers() -> tuple[TypeNormalizer, ...]:
"""Every pack's type normalizers, in manifest order (step 4, ruling D2)."""
return tuple(n for m in MANIFESTS for n in _rules(m.name).type_normalizers)
@cache
def type_normalizers() -> tuple[TypeNormalizer, ...]:
"""Every pack's type normalizers, in manifest order (step 4, ruling D2)."""
return tuple(n for m in MANIFESTS for n in _rules(m.name).type_normalizers)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 15af381: type_normalizers() is @cached — the packs' rules never change once loaded. Pinned by test_the_normalizers_are_collected_once (type_normalizers() is type_normalizers()), which failed before the change.

Comment on lines +110 to +114
def reconcile_import(fqn: str, source_file: str | None, view: ResolutionView) -> str | None:
"""A TS import's node through tsconfig aliases, then workspace packages (#504, #507, #508)."""
facts = cast("TypeScriptIndexFacts", view.facts["typescript"])
aliased = _alias_candidates(fqn, source_file or "", facts.path_aliases)
return _resolve_typescript_import(fqn, view.nodes, facts.workspace_packages, aliased)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Accessing view.facts["typescript"] directly can raise a KeyError if the ResolutionView / SymbolIndex is hand-built or partially constructed (e.g., in unit tests or minimal environments) without the "typescript" key. It is safer and more defensive to use .get("typescript") and handle the None case gracefully.

Suggested change
def reconcile_import(fqn: str, source_file: str | None, view: ResolutionView) -> str | None:
"""A TS import's node through tsconfig aliases, then workspace packages (#504, #507, #508)."""
facts = cast("TypeScriptIndexFacts", view.facts["typescript"])
aliased = _alias_candidates(fqn, source_file or "", facts.path_aliases)
return _resolve_typescript_import(fqn, view.nodes, facts.workspace_packages, aliased)
def reconcile_import(fqn: str, source_file: str | None, view: ResolutionView) -> str | None:
"""A TS import's node through tsconfig aliases, then workspace packages (#504, #507, #508)."""
facts = view.facts.get("typescript")
if facts is None:
return None
facts = cast("TypeScriptIndexFacts", facts)
aliased = _alias_candidates(fqn, source_file or "", facts.path_aliases)
return _resolve_typescript_import(fqn, view.nodes, facts.workspace_packages, aliased)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining, deliberately — the same call as on #600. Every SymbolIndex the pipeline builds comes from IndexBuilder.build, which always supplies every registered pack's facts, and since #600 SymbolIndex.facts is a required field so an index without them cannot be constructed by accident. A .get(...) returning None here would turn a mis-built index into silently unresolved TypeScript imports — a wrong graph with no error — which is exactly what the identity and resolver tests exist to rule out. A KeyError names the missing pack at the point of use. The same reasoning applies to package_call and fallback (the two sibling threads).

Comment on lines +117 to +130
def package_call(name: str, file_path: str, view: ResolutionView) -> str | None:
"""A TypeScript call on a name imported from a package: `useState` -> `react.useState`.

Only when the binding lands in a package (#556). A binding into one of our
own modules through an alias or workspace name is left to the lookups that
already resolve those, so internal resolution is unchanged.

Bindings are per file, not per scope: a parameter that shadows an imported
name (`function f(map) { map() }` beside `import { map } from "lodash"`) is
read as the import. Accepted — shadowing an import is rare, and scope
tracking is a separate piece of work.
"""
facts = cast("TypeScriptIndexFacts", view.facts["typescript"])
bindings = facts.file_bare_bindings.get(file_path)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to reconcile_import, accessing view.facts["typescript"] directly via view.facts["typescript"] can raise a KeyError if the facts dictionary does not contain "typescript". Using .get("typescript") with a fallback or early return is safer.

Suggested change
def package_call(name: str, file_path: str, view: ResolutionView) -> str | None:
"""A TypeScript call on a name imported from a package: `useState` -> `react.useState`.
Only when the binding lands in a package (#556). A binding into one of our
own modules through an alias or workspace name is left to the lookups that
already resolve those, so internal resolution is unchanged.
Bindings are per file, not per scope: a parameter that shadows an imported
name (`function f(map) { map() }` beside `import { map } from "lodash"`) is
read as the import. Accepted — shadowing an import is rare, and scope
tracking is a separate piece of work.
"""
facts = cast("TypeScriptIndexFacts", view.facts["typescript"])
bindings = facts.file_bare_bindings.get(file_path)
def package_call(name: str, file_path: str, view: ResolutionView) -> str | None:
"""A TypeScript call on a name imported from a package: `useState` -> `react.useState`.
Only when the binding lands in a package (#556). A binding into one of our
own modules through an alias or workspace name is left to the lookups that
already resolve those, so internal resolution is unchanged.
Bindings are per file, not per scope: a parameter that shadows an imported
name (`function f(map) { map() }` beside `import { map } from "lodash"`) is
read as the import. Accepted — shadowing an import is rare, and scope
tracking is a separate piece of work.
"""
facts = view.facts.get("typescript")
if facts is None:
return None
facts = cast("TypeScriptIndexFacts", facts)
bindings = facts.file_bare_bindings.get(file_path)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining — see the reply on reconcile_import above: a missing "typescript" entry can only come from an index built outside IndexBuilder, and failing loudly there is preferable to a silently skipped package call.

Comment on lines +148 to +151
def fallback(name: str, file_path: str, view: ResolutionView) -> tuple[str, float] | None:
"""A call on a JS runtime global the file does not shadow: `js_builtins.<name>` (#111)."""
facts = cast("TypeScriptIndexFacts", view.facts["typescript"])
shadowed = facts.file_shadowed_globals.get(file_path, frozenset())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the other functions, accessing view.facts["typescript"] directly can raise a KeyError if "typescript" is missing from view.facts.

Suggested change
def fallback(name: str, file_path: str, view: ResolutionView) -> tuple[str, float] | None:
"""A call on a JS runtime global the file does not shadow: `js_builtins.<name>` (#111)."""
facts = cast("TypeScriptIndexFacts", view.facts["typescript"])
shadowed = facts.file_shadowed_globals.get(file_path, frozenset())
def fallback(name: str, file_path: str, view: ResolutionView) -> tuple[str, float] | None:
"""A call on a JS runtime global the file does not shadow: `js_builtins.<name>` (#111)."""
facts = view.facts.get("typescript")
if facts is None:
return None
facts = cast("TypeScriptIndexFacts", facts)
shadowed = facts.file_shadowed_globals.get(file_path, frozenset())

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining — see the reply on reconcile_import above: a missing "typescript" entry can only come from an index built outside IndexBuilder, and failing loudly there is preferable to a silently skipped package call.

The engine asks for the normalizers on every declared type, and the packs'
rules never change once loaded; the tuple was rebuilt on each call (~4.6 ms on
owner-api). Gemini.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@zaebee
zaebee merged commit 261cee3 into main Oct 14, 2026
4 checks passed
@zaebee
zaebee deleted the feat/language-packs-step4 branch October 14, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant