Skip to content

Explore repository-context providers for repository-wide topology #159

Description

@imjlk

Outcome

Add a sibling repository-context provider plane for evidence-backed workspace,
project, package, source-root, target, task, entrypoint and project-dependency
topology. Join it to code files under the same input-generation transaction and
expose it through a typed branch of the existing single MCP tool.

Do not overload language providers, parse every build DSL in core, or mix
generated summaries/embeddings with declared or tool-resolved build facts.

Verified current boundary

The language graph is good at symbol-scale semantics. Repository orientation
still repeatedly falls back to rg, directory walks and raw manifests for:

  • workspace/package membership;
  • applications versus libraries;
  • source/test/generated roots;
  • build targets/tasks/entrypoints; and
  • project-to-project dependencies.

Current IGraphProvider is intentionally a language-slice owner:

  • languages is non-empty;
  • provider selection rejects overlapping language ownership;
  • facts is the GraphEdgeKind code-semantic union; and
  • a bulk session replaces language slices atomically.

Generalizing it to ?anything that emits graph-shaped data? would weaken those
invariants. The current dump provenance/generation machinery is reusable, but
repository topology needs a sibling contract and fact vocabulary.

Decision: separate fact plane and provider registry

Add:

IRepositoryContextProvider
IRepositoryContextSession
REPOSITORY_CONTEXT_PROVIDERS
ISamchonRepositoryContextDump

Reuse provider lifecycle concepts?resolve, prepare, build inputs,
configuration, refusal, open/refresh/close, universe/manifest/content digests,
source digests, shard transaction and coverage?but do not require a language
slice.

The first ontology is deliberately small.

Node kinds

  • workspace
  • project
  • package
  • source-set
  • source-root
  • generated-root
  • build-target
  • task
  • entrypoint

Relation kinds

  • contains
  • depends-on
  • source-of
  • test-of
  • produces
  • invokes
  • entrypoint-of
  • joins-file

Repository nodes/edges stay in a separate dump plane. Do not encode a Gradle
project as a GraphNodeKind.class or reuse code-semantic calls.

Evidence and authority

Every topology fact has:

  • provider/tool/version;
  • authority: tool-resolved, declared, or later inferred;
  • source manifest/export artifact and source evidence where available;
  • build configuration/target universe;
  • input and content digests; and
  • complete, partial, or unsupported coverage per topology family.

Version one accepts only tool-resolved and versioned declared facts.
Directory naming conventions, guessed entrypoints and generated prose are out.
If inference is added later it must remain a lower authority and visibly
separate.

Authoritative adapter routes

Prefer the owning tool's machine-readable model:

Ecosystem First authoritative route
npm/pnpm/Yarn package-manager workspace/member output plus versioned package manifests/lockfile
Cargo cargo metadata --format-version 1, including workspace members, targets, features and resolved dependencies
Go go env GOWORK and go list -json/module/workspace metadata
Gradle Tooling API project hierarchy, project dependencies, source directories and tasks; wrapper-aware daemon reuse
Maven effective project/reactor model and dependency graph from Maven APIs, not raw parent/property guessing
Scala/BSP workspace build targets, sources, resources, dependencies and scalac options from BSP
.NET MSBuild/Roslyn solution projects, target frameworks, project references and generated documents
CMake versioned CMake File API codemodel
Bazel configured target graph from cquery --output=proto; action/output facts from aquery only when needed
Clang generic compilation database TUs plus include/source joins; not a full build-project ontology
Swift SourceKit-LSP build system/BSP or SwiftPM/Xcode build description
Dart/Flutter pub workspace/package configuration and analysis contexts
Composer Composer package/lock/installed/autoload model
Ruby Bundler resolved specs/groups plus explicit gem/package manifests
Python versioned pyproject.toml workspace/build-backend metadata plus the selected environment's installed distribution/import model

Parsing a manifest is acceptable when that manifest itself is the versioned
declarative contract. Do not write a general Groovy/Kotlin/Starlark/YAML parser
to emulate a build tool. Adapter failures or unavailable tool models produce
partial/unsupported coverage and a fallback to smaller declared facts.

Metadata commands must have a documented side-effect/security policy. Avoid
executing arbitrary tasks/scripts solely for orientation; if a build tool must
evaluate project code, report it before invocation and use the project wrapper/
configured environment.

Identity, joins and generation compatibility

Repository identities include provider/ecosystem, workspace root, project/
package/target coordinate and configuration. Same project under two Gradle
configurations, Cargo feature sets or Bazel configured targets is not one node.

Join topology to code at file level first:

  • source/generated roots contain normalized graph file identities;
  • build target/source set joins files;
  • entrypoint joins its source file and, when the language provider proves it,
    a code symbol; and
  • project dependencies remain repository edges even when no symbol-level edge
    is present.

File-level joins are sufficient for version one. Add symbol joins only for
explicit entrypoint/generated-symbol evidence, not name guessing.

Topology and language shards commit independently, but an MCP generation may
combine them only when their input manifests/configuration coordinates are
compatible. On mismatch:

  1. retain the last compatible topology generation if its inputs still match;
  2. otherwise publish the code graph with topology coverage marked unavailable;
  3. never silently join new code to stale topology.

A topology-only edit invalidates joins/affected targets, not unrelated semantic
code shards.

MCP surface decision

Keep one MCP tool, inspect_code_graph, but add a versioned topology request
branch and typed result rather than forcing project nodes into code operations.
The branch supports:

  • workspace/project/package orientation;
  • dependencies/dependents;
  • source sets/roots and generated roots;
  • build targets/tasks;
  • declared/tool-resolved entrypoints; and
  • file joins into existing graph handles.

tour, overview and entrypoints may consume topology internally and include
an optional topology section, but their code-semantic nodes/edges retain their
meaning.

This is an intentional coordinated application-schema revision. Update the
application parity fixture/reference contract in the same change or gate the
branch behind an explicit protocol capability. Do not smuggle structured
topology into the English audit string.

Every result returns topology provenance, coverage, generation and join status
beside #63's language coverage.

Incremental and performance behavior

Topology is shardable by provider/workspace/project/target:

  • no-op: validate tool/manifests and load prior shards;
  • manifest/lock edit: rerun only the owning metadata adapter and affected joins;
  • project/source-root create/delete/rename: replace affected topology shards;
  • build configuration/feature/target change: select a new universe;
  • provider failure/cancel/malformed output: retain the prior compatible context;
    and
  • language graph update: recompute only file joins whose identities moved.

Use resident owning tools where they exist?Gradle Tooling API daemon, BSP
server, SourceKit-LSP, Roslyn workspace. Simple declared manifests should parse
in-process. Target no-op below 250 ms after validated inputs; record tool startup,
model query, normalization/join and MCP readiness separately.

First implementation slice

Dogfood a mixed fixture with:

  1. this repository's pnpm workspace;
  2. a Cargo workspace using cargo metadata;
  3. a Gradle multi-project build using Tooling API; and
  4. a CMake project using File API codemodel.

This exercises declared and tool-resolved authority, multiple ecosystems,
configuration, generated/source roots and cross-plane file joins without
attempting every adapter at once.

Implementation order:

  1. define ontology, identity, dump/session/shard/coverage contracts;
  2. add deterministic fake-provider transaction and stale-join tests;
  3. implement pnpm/package manifest adapter and dogfood this repository;
  4. implement Cargo, Gradle and CMake adapters;
  5. add the versioned MCP topology branch and use topology in
    overview/tour/entrypoints;
  6. add remaining adapters behind the same conformance suite; and
  7. publish support documentation only for adapters with real fixtures.

Correctness and impact experiment

Use a pinned repository-orientation question, for example:

Explain the workspaces, applications and packages, their source/test/generated
roots and entrypoints, and the project dependency flow relevant to X.

Record:

  • topology MCP calls and returned evidence;
  • rg/directory walk/raw manifest reads;
  • correctness of projects, dependencies, roots and entrypoints;
  • unsupported/inferred claims;
  • input tokens/tool latency; and
  • whether file/code joins led to the correct semantic follow-up.

The goal is not zero shell reads. It is that stable, authoritative topology is
read from one indexed model and shell/source reads become explicit gaps.
Do not run a paid agent A/B benchmark without separate authorization.

Acceptance

  • Repository context has a sibling registry/fact plane; language ownership
    and GraphEdgeKind remain unchanged.
  • Ontology, identity, provenance, coverage and joins are versioned and
    validated.
  • pnpm, Cargo, Gradle and CMake mixed fixtures assert exact projects,
    targets, roots, dependencies, entrypoints and file joins.
  • Declared/tool-resolved/inferred authority cannot be conflated.
  • Incompatible code/topology generations never join.
  • No-op, manifest/lock/config edit, create/delete/rename, provider failure,
    cancel, stale result and retry are atomic.
  • The single MCP tool has a typed/versioned topology request/result and
    operation-scoped coverage.
  • Orientation experiments show correctness and tool/read changes without
    claiming shell search is eliminated.
  • Adapters use authoritative tool models where available and do not emulate
    build DSLs in core.
  • Focused tests, pnpm build, pnpm test, and pnpm coverage pass.

Non-goals

Primary references

Verified residual on 1bf9486

PR #164 delivered the sibling plane and first four adapters, but the next solo
campaign reproduced correctness gaps beyond the previously recorded missing
orientation experiment.

Initial static resident generations cannot join

createResidentGraphSource computes the initial static state's input generation
but publishes the raw buildStaticGraphResult dump without attaching it. A
fresh mode: "static" MCP server therefore withholds every joins-file edge on
its first topology call and reports the no-generation explanation intended for
unrevalidated graph files and legacy dumps. LSP/hybrid initial builds already
attach their committed generation; the defect is the explicit-static branch.

Repository path classification is inconsistent

Cargo, Gradle, and CMake each carry a private isInside implementation:

const relative = path.relative(root, file);
return relative !== ".." && !relative.startsWith(`..${path.sep}`);

On Windows, path.relative("D:\\a", "C:\\b") is the absolute C:\\b, so all
three report cross-drive targets/roots/artifacts as internal. The shared
isSubPath utility already handles that case and preserves Windows
case-insensitive comparison.

The pnpm adapter has a separate instance of the same class of error. Manifest
roots and entrypoints are always emitted with external: false. A reproduced
absolute main outside the workspace became file: ../outside-entry.js while
remaining non-external. A files: ["../outside-root"] entry escaped its package,
used a non-canonical pkg/../outside-root identity, and normalized its public
root to a different value.

pnpm publication files are labeled as source directories

The pnpm adapter maps every simple package files entry to source-root unless
its spelling resembles a generated directory. Dogfooding this repository
therefore labels README.md, LICENSE, lint.config.ts, and tsconfig.json as
source roots. npm's files field is a publication allow-list and may name
individual files; it does not establish that each path is a source directory.

The same boundary trusts the manifest cast without checking consumed fields. A
string-valued files field is iterated character by character as roots, and a
string-valued scripts field becomes numeric task names. Valid open-ended
exports condition keys should remain supported, but wrong field types must
fail closed with the manifest identity.

Nested workspace creation is not observed

The pnpm and Cargo sessions watch current member manifests and only one nearby
ancestor. In real temporary workspaces using groups/*/*, the initial
groups/a/one member watches groups/a but not groups. Creating
groups/b/two changes no watched identity, so the second refresh returns
changed: false and permanently omits the package until another input moves.
The directory ancestor chain from each member to the workspace root is the
owning discovery surface; hashing immediate entries at each level detects new
siblings without emulating either workspace DSL.

Cargo cannot execute on a standard Windows rustup install

The Cargo adapter hardcodes cargo.cmd on Windows. rustup installs
cargo.exe; on the campaign checkout the real provider failed with
cargo.cmd is not recognized while cargo.exe was present on PATH. Use the
platform-aware command resolver around the native cargo name and add a
hermetic executable-resolution test.

Residual consequence surface

These defects affect topology dump identity, resident no-op and package
create/rename/delete behavior, the external trust flag, file-join admission,
TypeScript API/MCP results, Windows Cargo/CMake/Gradle users, and the orientation
experiment. A fix must keep inferred facts out of protocol v1 and preserve
atomic last-good recovery.

Residual acceptance and verification

  • A fresh static resident topology call admits joins when code/topology
    generations are current; graph-file and legacy no-generation cases remain
    unavailable for their existing reasons.
  • Cargo, Gradle, CMake, and pnpm use one canonical workspace-containment rule,
    including Windows cross-drive and case behavior.
  • pnpm source/generated roots are canonical, package-confined directories;
    individual publication files are not relabeled as roots.
  • Consumed pnpm manifest/model fields are validated at the boundary and
    malformed shapes fail closed without partial facts.
  • pnpm and Cargo detect member creation at every nested workspace depth,
    plus deletion, rename, no-op, failed refresh, and recovery.
  • A hermetic Windows-path case proves the Cargo adapter resolves a native
    cargo.exe rather than inventing cargo.cmd.
  • The repository-orientation experiment reports authoritative projects,
    packages, roots, entrypoints, dependencies, joins, unsupported claims,
    tool/read changes, and latency without claiming shell reads are eliminated.
  • Focused topology tests, pnpm build, pnpm test, and pnpm coverage pass.

No paid agent A/B run or global toolchain installation is authorized by this
issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions