Skip to content

refactor(languages): step 2 — per-pack project readers - #599

Merged
zaebee merged 7 commits into
mainfrom
feat/language-packs-step2
Oct 14, 2026
Merged

zaebee merged 7 commits into
mainfrom
feat/language-packs-step2

Conversation

@zaebee

@zaebee zaebee commented Oct 14, 2026

Copy link
Copy Markdown
Owner

Step 2 of the language-packs migration (spec docs/specs/2026-10-14-language-packs-design.md "Project facts", plan docs/plans/2026-10-14-language-packs-step2.md, part of #146). Each pack reads its own project configs; import_names.py no longer knows any language. No graph and no ingest_state value changes.

What moved

from to
cgis/python_dependencies.py languages/python/dependencies.py
cgis/npm_dependencies.py, workspaces.py, tsconfig_paths.py languages/typescript/{npm,workspaces,tsconfig}.py
TS half of ImportNameCollector (_note_configs_above, note, three lines of collect) languages/typescript/project.py → TypeScriptProjectReader / TypeScriptFacts
— languages/python/project.py → PythonProjectReader / PythonFacts
— cgis/core/project.py → ProjectReader, ProjectFacts protocols; LanguagePack.project_reader(root, extractors); registry project_readers()
TS module naming languages/typescript/naming.py (tree-sitter-free)

import_names.py routes walk files and the directories above the root to every pack's reader and merges their facts with ImportNames.from_facts; ImportNames fields, record, differ_from and the store keys are unchanged, so the pipeline and resolver are untouched.

Identity report

graph_identity.py diff (nodes, edges and ingest_state), main's cgis 64e05c6 vs this branch over the same tree:

  • cgis src/ @ 715cd9a — identical (2019 / 7858)
  • owner-api @ 8de2245 — identical (19115 / 96258; python_dependencies non-empty)
  • this repo's ui/ — identical (286 / 497; npm_dependencies non-empty)
  • snapshot (incl. ingest_state) untouched; test_configs_above_root.py changed by one moved import line only.
  • The final reviewer reproduced all three independently and probed five scenario repos (TS-only, Python + root package.json, pnpm monorepo from a subdirectory, unbounded search, pre-facts stored graph) on base and branch, with the lazy mapping and with a .py-only mapping: identical facts throughout; a stored graph missing the facts rebuilds once, then stays stable.

Acceptance

  • Importing the packs loads no grammar; a TS-only ingest loads only tree_sitter_typescript (mutation-checked: a top-level PythonExtractor import in the Python pack fails it).
  • A Python repo with a tooling package.json at its root loads only tree_sitter_python (step-1 deferred minor; main loaded both).
  • Two packs claiming one ingest_state key, a key the collector cannot store, or a fact of the wrong JSON shape all fail loudly.
  • extraction domain: drift 0.315 → 0.250, unresolved 0.346 → 0.334 (lock 0.40 / 0.37).

Rulings (where the code departs from the spec's text)

  • S1 The spec's ProjectFacts.record(store)/differs_from(store) contradicts "a pack never touches storage" (the pack-isolation gate). Facts are state() data keyed by ingest_state key; the collector records and compares them. Spec carries an implementation note.
  • S2 Pack modules are imported up front (readers must see configs before the first file); grammars load inside extractor(). Supersedes step 1's R7. Side effect: the seven config-reader modules joined the review-fingerprint closure — editing one now changes the reviewer's identity; the inventory pins them.
  • S3 The Python reader keeps its own manifest search (from the root itself, up to .git); note_above is the TS reader's.
  • S4 Readers get the walk's extractor mapping, so a .py-only test pipeline keeps today's TS facts.

Final review (independent, execution tools)

  • Fixed (test-first): from_facts silently dropped unknown keys and cast hid wrong shapes (a tuple would rebuild on every incremental run); the review-path inventory was stale after the readers joined the closure.
  • Deferred minors: a Python-only repo with a root tsconfig.json still loads the TS grammar (TsconfigPaths.aliases asks for the namer before scoping — pre-existing, same class as the package.json fix); the "Reading tsconfig.json above the ingest root" log line now comes from cgis.languages.typescript.project.

Gate: format, lint, mypy strict, 3191 passed, docstring 99%.

🤖 Generated with Claude Code

zaebee and others added 7 commits October 14, 2026 10:55
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Readers (step 2) need every pack before the first file; the grammar can wait
for the first parse. TS module naming moves to a tree-sitter-free naming.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
python_dependencies -> languages/python/dependencies; npm_dependencies,
workspaces, tsconfig_paths -> languages/typescript/{npm,workspaces,tsconfig}.
Pure move plus imports; snapshot and ingest_state unchanged. Extraction domain
drift 0.305, unresolved 0.347 (lock 0.40/0.37).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…out knowing languages

ProjectReader/ProjectFacts (cgis.core.project) per pack: Python reads declared
dependencies, TypeScript npm names, workspace packages and tsconfig aliases.
Facts are data keyed by their ingest_state key; the collector records and
compares them through the store, since a pack never touches storage (ruling S1).
Store keys, ImportNames and the snapshot (ingest_state included) unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…grammar

WorkspacePackages.note asked for the TS extractor before learning the manifest
was the repository root's, which it skips anyway; a Python repo with a tooling
package.json (prettier, commitlint) loaded the grammar for nothing. Step-1
deferred minor; pinned by a fresh-interpreter CLI ingest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rs on the review path

from_facts read only the four keys it knows and cast their values: a new pack's
fact under another key vanished without a trace, and a tuple where the store
reads back a list rebuilt the graph on every incremental run. Unknown keys now
raise ValueError and wrong shapes TypeError (final review I1).

Each pack's __init__ imports its project reader, so the seven config-reader
modules are now in the review-fingerprint closure (a side effect of ruling S2);
the inventory is regenerated so dropping one fails again (final review I2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements Step 2 of the Language Packs design, refactoring project configuration reading to be decentralized and language-specific. It introduces the ProjectReader and ProjectFacts protocols in cgis.core.project, allowing each language pack (Python and TypeScript) to read its own project configurations (dependencies, workspaces, and tsconfig paths) without coupling the core collector to specific languages. Additionally, language pack modules are updated to import their tree-sitter grammars lazily inside extractor(), preventing unnecessary grammar loading (e.g., loading TypeScript grammar in Python-only repositories with a root package.json). The ImportNameCollector is refactored into a language-free collector that routes files to pack-specific readers and merges their facts. No review comments were provided for this pull request, so there is no feedback to evaluate.

@sonarqubecloud

Copy link
Copy Markdown

@zaebee
zaebee merged commit 804a398 into main Oct 14, 2026
4 checks passed
@zaebee
zaebee deleted the feat/language-packs-step2 branch October 14, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant