Repository navigation
feat(code-index): index authenticated Git submodules - #4801
ScriptedAlchemy wants to merge 46 commits into
Conversation
Keep parent repository source admission unchanged; use one shared walker for exclusion evidence instead of a second filesystem scan. Refs #4593
|
|
Refs #4593 — validation blocker at exact head The repository-gates job failed before Rust execution: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42433144833/job/128586224987 . The actual failures are:
No source fix, Rust test execution, independent review, branch update, merge or issue closure was completed in this follow-up. This environment has no executable repository checkout or Bazel; direct repository retrieval failed, and no usable remote execution/workflow-dispatch capability was exposed. This is a validation limitation, not evidence that the implementation is correct. #4801 remains diagnostic-only and draft. It does not yet satisfy authenticated submodule admission, pinned child-tree capture, child-aware freshness, or consistent retrieval/graph behavior. Retain Coordination: the dedicated #4593 owner remains responsible for implementation; PR maintenance owns CI assistance without racing its branch. Do not repair unrelated decision-model generated paths by importing forbidden work. #4776 and all Kev/decision-model work remain draft and unmergeable by instruction; #4763 remains read-only. No protections or tests were weakened. |
|
Refs #4593 — exact-head CI repair The PR is still partial: it adds skipped-nested-repository diagnostics and regression source, but NOT authenticated submodule indexing, freshness, or indexed retrieval. HOLD MERGE even if its current-head diagnostics tests pass: keep this same PR open to complete #4593. Being reviewable or passing preliminary tests does not mean the issue's acceptance is met. No complete-implementation, native macOS or independent-review proof exists. Keep |
Limit skipped-repository samples during traversal, honor directory policy before collecting diagnostics, and quote bounded path labels. Propagate source-walk failures instead of returning an empty successful audit. Add behavioral coverage for bounded sampling, policy scope, preserved ecosystem caveats, Unicode/control-character paths, and missing roots. Refs #4593
|
Refs #4593 — published source repair Fixed unbounded skipped-repository collection, diagnostics escaping configured directory exclusions, unquoted/unbounded path labels, and swallowed filesystem-walk failures. Added six behavioral regression cases (one Unix-only filename case); existing initialized-submodule and linked/nested-repository exclusions are preserved. Only Current-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42442123433 — queued when inspected. These Rust regressions have NOT executed; author diff review is NOT independent review. The prior successful draft run skipped Rust/Clippy/native jobs, so it is not acceptance evidence for these changes. The existing merge hold remains. This repairs the current audit implementation, not the still-unimplemented optional authenticated submodule indexing feature. Keep the PR draft/open and #4593 open. Continue the integrated admission/capture/freshness/retrieval implementation here rather than opening a duplicate or treating more diagnostic fixes as feature completion. |
|
Refs #4593 — published nonforce, expected-head fast-forward |
Use one source boundary for interactive walks, Git candidate classification, worktree capture (including ignored-file admissions), and freshness. Parent tracked files hidden by a new nested .git marker become tombstones, invalidate clean HEAD-tree reuse, and return when the marker disappears. Reject source paths passing through symlinked directories or lexical escapes. Add real Git classification and filesystem regressions for nested directory, gitdir-file, symlink, path traversal and removal behavior. Refs #4593
|
Refs #4593 — published source repair |
|
Refs #4593 — published exact-head CI repair |
Sample .git marker evidence for tracked and explicitly admitted source ancestors even when Git dirwalk never recurses into tracked-only folders. Test marker creation and removal against a real Git parent index. Refs #4593
Directory metadata changes when the marker is removed, so the original cached roster must not be treated as restored merely by marker absence. Refs #4593
A registered gitlink remains an untrusted nested source boundary even if its child .git marker is absent. Keep live grep and audit in default-deny parity with the parent index; include/glob cannot lift repository trust. Exercise a real registered Git submodule with its marker removed. Refs #4593
|
Refs #4593 — published narrow default-deny source fix Current-head run: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42466514013 (queued at check). NO Bazel behavioral Rust pass or independent review claim. Do not merge or mark ready. Feature acceptance remains integrated authenticated opt-in, pinned historical child capture, child status/freshness and cross-boundary graph/index/live retrieval, with nonzero Bazel/native validation. Preserve #4800 and protected Kev/release lanes. |
Apply the two exact rustfmt_check_2024 hunks from hosted run 42466514013. No source behavior changes; authenticated submodule indexing remains pending. Refs #4593
|
Refs #4593 — narrow hosted Bazel formatting repair published at Exact-head run 42466514013, repository-gates job 128687191524, failed only HOLD MERGE: this PR remains DRAFT and contains only exclusion/diagnostic/freshness hardening. The requested default-off authenticated submodule inventory, selected historical child-tree capture, dirty child freshness, and graph/indexed/live retrieval parity remain unimplemented. No independent review. #4593 stays open. Please do not merge or change the draft selection to conceal missing behavior, and preserve #4800/#4508, protected #4776 Kev and read-only #4763. |
|
Refs #4593 — new source-review boundary mismatch, regression needed; not a claimed exploit or passing test (reviewed at exact head The default-deny live walker now checks parent stage-0 gitlink roots through Test the concrete scenario with a real parent 160000 gitlink under Current-head hosted workflow 42468887836 has passed repository gates, but Rust native/OS tests and Clippy were skipped for this draft; the Dashboard job remained queued when checked. Do not push a speculative fix that cancels that run. Fix via the one canonical authenticated submodule boundary, not a third independent scanner, and keep DRAFT/HOLD MERGE, |
A parent stage-0 gitlink remains tracked repository source even when its child checkout loses the .git marker. Check every indexed ancestor before an ignored dependency is admitted or revalidated, preserving the existing proof for ordinary ignored sources. Add a real local Git submodule regression that removes the child marker and checks both the admission proof and production source-read refusal. Refs #4593
|
Refs #4593 — published The ignored-dependency gix proof now checks every parent index ancestor for mode-160000 gitlinks, not only the exact candidate path. This closes the omitted-marker admission path discussed in comment 6720823044 for verified ignored dependencies, including the production source read and existing ignored-roster revalidation. Added a real local Git submodule regression source: verified stage-0 gitlink and No Rust regression has executed through Bazel, and there is no independent review. Current-head hosted CI 42482445184 was queued. Previous-head run 42468887836 completed successfully after re-running a Dashboard job blocked by a transient download timeout; draft native Rust/Clippy remained skipped. I independently checked the disposable Git fixture commands locally, but that is not a repository Rust test or acceptance evidence. HOLD MERGE / keep DRAFT / #4593 OPEN. The default-off authenticated submodule inventory, selected child-commit historical capture, child-aware freshness, and indexed/graph/live retrieval parity remain undone. Protected Kev/release PRs, #4800/#4508, and CI protections were not changed. |
Refs #4593. Full authenticated child indexing still incomplete.
|
Refs #4593 — guarded source repair on existing DRAFT #4801, HOLD MERGE. Commit One file changed: The integrated canonical default-off authenticated submodule opt-in, shared inventory across child candidate capture/freshness, pinned historical submodule trees, and parent-relative graph/retrieval are not implemented. Do not mark ready, merge, or close #4593. Protected #4776 Kev, #4763 release and #4800/#4508 peers untouched. |
Refs #4593. Reuse the parent's index-plus-HEAD boundary for ignored sources and cover the real staged-removal checkout scenario. The authenticated opt-in indexing feature remains incomplete.
|
Refs #4593 — source repair published on existing DRAFT PR; HOLD MERGE. Commit Ignored-source proof now calls the same index-plus-HEAD parent gitlink boundary as the source walker. The real-Git fixture commits a parent with a Verification limit: read back the exact commit and diff; no nonzero Bazel Rust test executed locally. Current-head hosted run 42503394287 was queued and chooses |
|
Refs #4593 — NEW ACTIONABLE NATIVE WINDOWS FAILURE; DRAFT / HOLD MERGE (2026-11-09 UTC). The independently dispatched hosted run 42587465701, on exact tested commit
Required source follow-up: preserve the falsifiable test unchanged; on native Windows compare authenticated Issue #4593 stays OPEN. PR #4801 stays DRAFT / UNMERGED. Independent review and exact-head native acceptance remain outstanding; do not race other issue owners, #4800/#4508, Kev/#4776, or release #4763. |
Refs #4593 Normalize the existing child-root canonical identity before comparing it to the scheduler's plain path. Windows canonicalize returns a verbatim disk path even for the same checkout; the raw comparison silently dropped live child candidates after a child HEAD advance. Keep the strict existing-root/no-symlink boundary. The already-failing native Windows submodule graph regression exercises this path; run it on the new head with Bazel before accepting.
|
Refs #4593 — published narrow Windows live-capture repair Cause isolated from the failed native Windows test: Not yet validated on this commit: local Bazel/checkout unavailable, and new current-head CI 42696190816 is queued with draft-native jobs expected to skip. Require >0 executed Bazel runtime tests on Windows and Linux on |
Retain each authenticated child binding while its source iterator is alive. Reuse the graph reader's checkout/marker liveness proof before advancing the child walk, including the depth-zero root. Return a typed walk error and permanently stop that child if its binding changes. Add real-Git regressions for marker removal, retargeting, mid-iteration binding loss and a replaced root, with positive parent/child controls. Rust regression execution and native verification remain required. Refs #4593
|
Refs #4593 — review repair published in Finding and repair
The walker now retains each Regression sourceAdded four real-Git cases in Publication and verificationExactly three files changed: No Rust compilation, formatting or behavioral test pass is claimed for this commit. Local Bazel/checkout execution is unavailable here. Current-head PR CI https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42720663699 was queued when checked and is selected with Coordination: keep this same PR DRAFT and HOLD MERGE. No duplicate PR, base retarget, test weakening, CI cancellation, peer-branch edits, model/release changes, merge or issue closure was performed. The existing #4835 stack is preserved; final integration must still reach master before #4593 can close. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Refs #4593 Reject older generation parse identities without an include_submodules field rather than silently treating them as default-off. Seal both true and false explicitly, with a regression for round-trip and omitted-field refusal. Apply the two exact Bazel rustfmt corrections reported for the new authenticated submodule source-walk regressions. Native Bazel behavioral tests and independent review remain required.
|
Refs #4593 — verified source repair published, keep DRAFT / HOLD MERGE. Commit
An independent review also raised scoped-search HEAD-tree traversal cost in thread Verification status: new exact-head CI 42729122675 queued at check. Native Rust/Linux/Windows/macOS and Clippy remain unverified on |
Avoid scanning every parent index entry and HEAD-tree object before a literal-prefix source search. Probe index ancestors and the sorted prefix range, and resolve only HEAD ancestors and the matching subtree. Broad or basename scopes keep the complete fail-closed boundary inventory. Add a real-Git regression where an unrelated committed subtree is unreadable: its absence must not block an exact-file scope, while a full source walk must still refuse the incomplete HEAD authority. Refs #4593
|
Refs #4593 — published narrow scoped-source performance repair 71602073 by expected-head NONFORCE fast-forward from Verification hold: new Rust regression has NOT executed by Bazel, and the previously failing Windows child-HEAD regression remains unproven on this head. Current-head CI 42740034779 is queued with |
Apply the five precise formatting corrections from repository-gates run 42740034779 for source_walk.rs. No behavior or policy changes. Refs #4593
|
Refs #4593 — published exact-head Bazel formatting repair At prior head Created one commit on the original remote parent and moved only |
|
Refs #4593 — new source-review finding on current draft head The new lazy-walk guard in Concrete falsifiable regression: create a real registered, initialized parent+child and successfully construct HOLD MERGE / keep DRAFT / #4593 OPEN. Exact-head CI 42748171498 is running/queued with native Rust and Windows skipped; no speculative push was made that would cancel it. Earlier Windows regression and independent approval remain outstanding. This comment does not approve or resolve any review threads. |
Refs #4593 Recheck the parent's exact staged mode-160000 gitlink and pin, staged .gitmodules identity, live registration and owned module store on existing child source and graph reads. The child marker alone is not sufficient authority. Add real-Git regressions for working and staged registration removal, staged gitlink removal and pin replacement after reader construction. These new Rust tests still require exact-head nonzero Bazel validation.
|
Refs #4593 — prepared fix now PUSHED, not just an unattached commit object. Published Exactly two files changed: Not acceptance yet: the new Rust regressions have not executed through Bazel. The repeated exact parent-index and working-registration checks also need performance validation; this change does not claim atomic filesystem snapshotting. Exact-head all-event CI includes run 42761466450, queued with The PR was unexpectedly ready when read. Restored DRAFT under the user's standing instruction and verified draft/unmerged after the push. Keep HOLD MERGE and #4593 open until exact-head nonzero Bazel verification, independent approval, and eventual acceptance on master. Base/prerequisite stack and protected model/release/scheduler work were not changed. |
Refs #4593 Apply the single exact formatter hunk reported by the current-head hosted Bazel rustfmt_check_2024 failure (run 42761466450). Retain all four new behavioral regression assertions unchanged.
|
Refs #4593 — exact-head hosted Bazel formatting repair published on existing DRAFT #4801. At Published HOLD MERGE / keep DRAFT / issue #4593 OPEN. Exact-head nonzero Bazel source-walk/authentication, live graph/pinned-history and previously failing Windows child-commit regression, performance measurement of per-entry parent-authentication checks, independent approval and integration onto master remain outstanding. No peer, protected model/release work or CI policy was modified. |
Refs #4593 Reject existing child source walks and graph reads when the child Git administration changes core.worktree without changing the .git marker. Reuse the initial pinned-store topology proof on every bound read. Add a real-Git regression that changes only the child repository worktree configuration after source and graph readers were created. The native Git fixture demonstrates the mismatch; Rust acceptance still needs nonzero Bazel tests on this exact commit.
|
Refs #4593 — published narrow submodule authority repair on existing DRAFT #4801, HOLD MERGE. Commit Concrete issue: after a child source walker or graph reader is constructed, changing the parent-owned child Git repository's Repair: existing bound reads reuse the initial pinned-store topology verification, including child Git/common directory and current checkout identity. The added real-Git Rust regression first proves positive parent/child access, changes only the child repository worktree configuration, checks that fresh admission rejects it, and requires both an already-open graph reader and lazy source walk to refuse further child access with a typed error. Validation hold: these new Rust tests have not executed through Bazel. The prior-head CI passed applicable repository/dashboard gates but skipped native Rust; current-head native Windows (including the prior failed child-HEAD regression), Linux/macOS, performance of per-entry binding checks, and independent approval are still required. Do not mark ready, merge, claim completion or close #4593. No protected/peer refs, CI settings or release work were changed. |
DRAFT / HOLD MERGE
Refs #4593. Keep the issue open pending independent review.
A project can set
index.include_submodules.v1and restart its daemon to index a checked-out direct submodule under its parent-relative paths. The default is false. Grep, structural search, the unmounted-file audit and graph indexing share the boundary. The audit reports skipped nested repositories.Live capture binds the staged parent gitlink and exact pin, the staged and working .gitmodules registration, and the child checkout to the parent's Git administration. It tracks child commits and local additions, edits and removals. Linked worktrees, unrelated clones, nested gitlinks, aliases into child repositories and uninitialized children stay outside live indexing. Tracked parent source aliases retain their logical graph paths when every link and the terminal file belong to the same parent checkout. Historical capture uses the selected parent tree and the pinned object in the parent-owned module store. If that store is missing, the historical generation is unavailable.
This PR is stacked on draft #4835, which fixes the shared dashboard fixture used by the failing Linux readiness journey. At
a39c36da7dall 48 feature-file patches match their previous versions. On this stacked head, eight native submodule tests passed across the walker and runtime, and the dashboard opening-to-unavailable test that previously failed on Linux passed on macOS. The scheduler fixture fix from the preceding head had already landed on master inadc55f8eca. Earlier full Linux and targeted Windows verification.The dashboard fixture now uses the shared Linux launcher download fix. At
8b58d91c10all 48 feature-file patches still match the preceding head. The third frontend attempt on the preceding head also stopped at a release download with HTTP 504. Full Linux and targeted Windows verification on the new head.The authenticated release download prerequisite #4847 is now included through the dashboard fixture. On
77ce539b4f, all 50 feature-file patches match the reviewed alias correctionda69c12c2a. Full Linux and targeted Windows on the combined head.Verification
At
f36fbb3841, the hosted Linux fleet passed all 183 Bazel test targets, including 107 uncached executions. The Clippy job, dashboard and shipped Linux CLI build also passed. Native Windows is queued on the same head. The subsequent master merge did not change the submodule indexer or its tests.At
f36fbb3841, the two complete native Linux ARM64 suites passed as an ordinary user on ext4: 377 source-index cases and 695 runtime cases, with two existing opt-in memory tests ignored. The first full run exposed a stale reconcile assertion expecting a tracked same-parent alias to vanish. The restored test now requires both tracked paths, the exact committed HEAD revision, and no redundant background refresh. The runtime suite reran all 695 cases after the correction. Hosted Linux and Windows on this head.At
da69c12c2a, independent review found and repaired a default-policy regression that had removed previously supported parent source aliases even in checkouts with no submodules. Native macOS passed all 377 code-index unit tests, five authenticated submodule runtime tests, and the restored historical/live regression. Both policy settings retained the local parent alias and refused an alias into a registered child. The historical and live captures agreed on six committed paths, and an unstaged change to the regular target updated all three Rust aliases in live capture while historical digests remained intact. Alias repair · Hosted PR CI on this headThe preceding hosted Linux fleet passed all 183 Bazel targets at
8b58d91c10(95 executed). The dashboard API, authenticated code-index runtime, and MCP suites ran in 212.2, 116.9 and 296.4 seconds respectively. The hosted Dashboard, Clippy, shipped Linux CLI build and repository gates also passed. The Windows job on that head stopped before target execution when GitHub’s release endpoint returned HTTP 504 for rules_python 2.4.6. Windows download failure The draft and independent review hold remain in place.On the latest stacked commit
8b58d91c10, native macOS Bazel passed all eight authenticated submodule cases and all 89 dashboard API tests. The dashboard suite exercised the opening-to-unavailable journey that previously timed out on the Linux fleet.On the stacked commit
a39c36da7d, Linux ARM64 passed all eight targeted submodule tests as an ordinary user, together with the previously failing dashboard opening-to-unavailable case on a native tmpfs scratch directory. The profile admission refused Docker overlay storage before the dashboard started during the initial local run. The same fixture completed in 0.37 s on tmpfs. The hosted frontend job failed twice while GitHub returned HTTP 504 for the pinned Bazel launcher binaries, before frontend tests began. The independent full Linux and native Windows run is pending.At
478da430ede75cb22cb1f7ffc97a5f14364a0f46, the complete macOS Bazel code-index-runtime unit suite passed 694 tests in 175.75 seconds. Two existing opt-in memory-scale regressions were ignored by the ordinary suite. The crate’s Bazel rustfmt check also passed. With an incomplete staged.gitmodulesand valid working file, repeated graph-serving and retained-reconcile calls now abstain while the probe fails. Repairing the staged registration restores reuse of the same generation. The suite also exercises positive live and historical submodule graphs and the default-off path.On
b96724957d79e242838c22b14b48c9d7c17e64b1, macOS Bazel ran 52 focused tests across six suites.The MCP production library and repository-wide Bazel rustfmt check passed. The workspace dependency fetch and BUILD generator completed. A separate scheduler fixture correction keeps backend identities consistent with the hosted environment.
At
26d7171bf8, all 548 MCP integration tests and all 22 storage/runtime integration tests passed on macOS, including both regressions from the earlier Linux run. The grep journey uses an actual separate Git directory. Restarting with a retained dependency that moved behind a nested repository or symlink returns the specific refusal before reading that path. The changed Clippy and rustfmt targets passed. The code-index suite passed all 377 cases on this head. An unfiltered runtime repeat reached the 300-second Bazel timeout on the shared Mac. Two disjoint runs on this same head then covered the complete suite: all 11 matching committed-tree cases passed, and the complementary run passed 683 cases with the usual two opt-in cases ignored. Across both indexing crates and the two affected integration targets, 1,641 tests passed on this head.On Linux ARM64 at the same commit, eight authenticated submodule tests passed across the source walker (three) and index runtime (five). They exercise the real live and historical graph across restart, default-off behavior, and refusal to reuse a persisted or retained generation after a failed submodule probe.
Hosted Linux ran both complete indexing suites (377 code-index and 695 code-index-runtime, with two opt-in cases ignored), and all 675 MCP tests passed. Clippy passed. The Linux fleet passed 182 of 183 targets. The dashboard API suite had one failure in 89 cases: the loopback fixture timed out during initial readiness before the opening-to-unavailable assertion. The effect on this run of releasing the chosen port before binding remains unproven. The preceding Windows job targeted that earlier head. Independent review remains pending. Keep this PR draft and HOLD MERGE.