Repository navigation
refactor(languages): step 1 — lazy registry, python/typescript packs - #597
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the contract into core Pure move (git mv) plus import rewrites; the identity snapshot is unchanged. BaseExtractor/ModuleNamer -> cgis.core.extractor; _python_ast -> _syntax (an `_ast` module shadows the stdlib one). Self-graph gates re-pointed: Invariant 2 now has a guard that fails when it examines no node; extraction domain -> cgis.languages (drift 0.278 -> 0.351, unresolved 0.357 -> 0.355; lock 0.40/0.37 kept). The A/B task check now looks for gold files at the task's pinned sha (the tree the runner checks out) instead of HEAD; the review-path inventory is regenerated (the closure lost exactly the eight moved files, all present under new paths). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tractors/registry cgis.languages holds one manifest per pack and imports a pack only when a file of its language is met; extractors(roots) is a Mapping that builds each extractor on first lookup. CLI, MCP server, auto-refresh, guardian, mermaid and the scripts go through the registry; the surface-packs allowlist is empty. Each manifest loads its pack with a static import inside a function rather than importlib: the review-fingerprint walk cannot follow a dynamic import (test_no_dynamic_imports_in_the_review_path), and mypy now checks each PACK against the LanguagePack protocol. Extraction domain re-measured with the registry in it: drift 0.315, unresolved 0.346 (lock 0.40/0.37). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rammar The walk's test-file skip comes from the registry manifests (skips), the pipeline fetches only the extractor a file needs, and the workspace/tsconfig readers ask for the TS extractor only when a package.json or tsconfig exists. Pinned by a fresh-interpreter CLI ingest with a README in the repo (a file no pack reads is what made a value-walking lookup build every extractor), and a TS control. Each of the three laziness sites was mutation-checked against it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… Sonar flag Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 switched _node_slug to pack_for_node, which reads Node.language. A directory row from `cgis structure` and a TS node stored before #513 both carry the "python" default, so their ids lost the package prefix (cgis_query → query, Button → components_Button). Back to the path lookup main used; pinned by a package-row test and an old-database TSX test (final review I1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request implements Step 1 of the Language Packs design by moving the Python and TypeScript extractors into a lazy, manifest-driven registry under cgis.languages and moving the base extractor contract to cgis.core.extractor. All surfaces, scripts, and tests have been updated to interact solely with the registry, allowing language packs to be loaded lazily only when their respective file extensions are encountered. Additionally, the pipeline and workspace/tsconfig readers have been optimized to prevent eagerly loading the TypeScript grammar in Python-only repositories. No review comments were provided, so I have no feedback to provide on the review itself.
@OverRide Sonar S1172 read PythonPack.code_fence(path) and extractor(extension, ...) as unused parameters; they are the protocol's signature. Packs now subclass the protocol and mark each method @OverRide, which Sonar, ruff (ARG002 — the noqa goes) and mypy all read as an override. The protocol moves from the registry to cgis.core.extractor beside BaseExtractor: the registry imports each pack, so a pack importing the protocol from the registry was a cycle (test_cgis_has_no_circular_dependencies). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
This was referenced Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 1 of the language-packs migration (spec
docs/specs/2026-10-14-language-packs-design.md, plandocs/plans/2026-10-14-language-packs-step1.md, part of #146). Moves code; changes no graph.What moved
extractors/base.pycore/extractor.py(BaseExtractor,ModuleNamer)extractors/python_extractor.py,_python_{ast,annotations,classes,functions,imports,types}.pylanguages/python/extractor.py,_{syntax,annotations,classes,functions,imports,types}.pyextractors/typescript_extractor.py,_typescript_bindings.pylanguages/typescript/extractor.py,_bindings.pyextractors/registry.pylanguages/__init__.py— lazy registry (deleted, no shim)New:
languages/python/__init__.pyandlanguages/typescript/__init__.pyeach exposePACK(LanguagePack:name,extensions,code_fence,is_test,scope_of,extractor). The registry holds oneManifestper pack and imports a pack only when a file of its language is met;extractors(roots)is aMappingthat builds each extractor on first lookup. CLI, MCP server, auto-refresh, guardian, mermaid and the scripts go through the registry — thesurface-packsallowlist goes from 3 entries to 0.Identity report
graph_identity.py diff, main's cgis (f2eea44 = c2f4a63 + an fsspec bump, nosrc/change) vs this branch's cgis over the same tree:src/@ 715cd9a — identical (2019 nodes / 7858 edges)tests/fixtures/graph_identity/snapshot.jsonuntouched; the snapshot test passes under both hash seeds.ui/(TS/TSX with package.json and tsconfig): identical.Acceptance
cgis ingest(fresh interpreter, repo with a.pyand a README) importstree_sitter_pythononly; control: adding a.tsfile loadstree_sitter_typescript. Each of the three laziness sites (pipeline lookup, workspaces, tsconfig) was mutation-checked against it. On main, importing the CLI loaded both grammars.extractiondomain re-pointed tocgis.languages: drift 0.278 → 0.315, unresolved 0.357 → 0.346 (lock 0.40 / 0.37 unchanged).Rulings (where the code departs from the spec's text)
foo.test.ts) isManifest.never_ingested+skips(); a pack'sis_testis the node flag and delegates tocore.paths.is_test_path(theNodevalidator keeps calling it — core cannot import packs).extractor(extension, roots)andcode_fence(path).extractor_for(path, roots)is realised as the lazyextractors(roots)mapping;IngestionPipelinekeeps itsMappingparameter.cgis.core.extractor— the gate reads anycgis.languages.<x>as a pack.scope_of._python_ast→_syntax(_astshadows the stdlib module).importlib: the review-fingerprint walk cannot follow a dynamic import (test_no_dynamic_imports_in_the_review_path), and mypy now checks eachPACKagainst the protocol.pack_for_node: directory rows fromcgis structureand TS nodes stored before fix(resolver): resolve TypeScript imports through tsconfig paths aliases #513 carryNode.language's"python"default (final review finding, fixed test-first).test_agent_ab_tasknow checks gold files at the task's pinned sha (the tree the runner checks out), not HEAD;review_path_inventory.txtregenerated (lost exactly the moved files, all back under new paths).Deferred minors (from the final review)
package.json(prettier/commitlint) in a Python-only repo still loads the TS grammar —workspaces.note()evaluates the lazy extractor before its root check. Main loaded it eagerly anyway.PurePosixPath.suffix, the pipeline byendswith— differs only for a file named exactly.py(pre-existing).docs/architecture/SELF_PORTRAIT.mdstill namescgis.extractors.*(point-in-time reading).Gate: format, lint, mypy strict, 3171 passed, docstring 99%. Independent reviewer (execution tools) ran before this PR.
🤖 Generated with Claude Code