Skip to content

feat(input): support GitHub tree subdirectories - #561

Open
deepujain wants to merge 4 commits into
NVIDIA:mainfrom
deepujain:feat/552-git-url-subdir
Open

deepujain wants to merge 4 commits into
NVIDIA:mainfrom
deepujain:feat/552-git-url-subdir

Conversation

@deepujain

Copy link
Copy Markdown
Contributor

Summary

  • resolve GitHub /tree/<branch>/<subdirectory> URLs by cloning the canonical repository at the requested branch
  • validate that the requested subdirectory exists and is a real directory
  • add an input-handler regression for support subdir within git url #552

Fixes #552

Validation

  • ruff check
  • ruff format --check
  • py_compile
  • git diff --check

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Reviewed current head 0f830f11d976a8aa4024f50c504e609f2ecda6f6. The canonical GitHub clone target, shallow branch checkout, subdirectory existence check, and hosted validation are useful, but the URL-to-local-path boundary is not safe yet.

The decoded subdirectory components are validated before PurePosixPath splits embedded separators. A percent-encoded absolute or parent-bearing component can therefore reset or escape clone_dir when passed to joinpath, causing a GitHub URL to select an unrelated local directory. The inline finding gives concrete examples and the required containment check.

The parser also assumes the first segment after /tree/ is the complete ref, so ordinary GitHub URLs for refs containing / are cloned at the wrong branch and subdirectory. Resolve the longest valid ref (or explicitly reject/document ambiguous refs), and add tests for slash-containing refs, encoded separators, absolute paths, and parent traversal.

Comment thread src/skillspector/input_handler.py Outdated

@yashrajp22 yashrajp22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked head 17de0665e82827f806c513ad81e082551a5c04f0. The encoded-path containment regressions now pass. The slash-containing branch issue from the existing review still reproduces; I am not duplicating it inline. The new inline finding covers cleanup after an invalid subdirectory selection.

Validation: 947 tests passed and one was skipped across the 12 selected file/node jobs; the initial combined run hit its ten-minute limit. All 48 reports from the 12 full sample directories match across base/head and source/wheel modes, including existing partial-analysis flags. Focused real-Git checks reproduced the remaining failures in both modes, and the installed CLI successfully scanned a real public GitHub tag/subdirectory. Verification was static-only; no live LLM coverage is claimed.

Comment thread src/skillspector/input_handler.py Outdated
@deepujain
deepujain force-pushed the feat/552-git-url-subdir branch from 17de066 to 8e070a8 Compare September 17, 2026 22:40

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed current head 8e070a8697e68cc24ed6c84a15e5f57fa72050a2 against both current threads, the complete input-handler/test diff, surrounding clone lifecycle and URL parsing, and exact-head checks.

The encoded-separator containment issue is resolved by decoded-segment rejection plus a final resolved-path containment check. Two required issues remain:

  1. The parser still assumes the first segment after /tree/ is the complete ref. A normal GitHub URL for a branch such as feature/foo is cloned at feature and interprets foo as part of the subdirectory, potentially failing or selecting the wrong tree. Resolve the longest valid remote ref or explicitly support an unambiguous ref-selection mechanism, with slash-containing-ref regressions.
  2. After _clone_git() succeeds, a missing, file, or symlink subdirectory raises before cleanup ownership is returned. These failed requests leave the checkout behind. Clean the owned clone on every post-clone selection failure and cover each rejected target shape.

All six exact-head checks pass, but these correctness/resource-lifecycle defects, unresolved threads, active change requests, and GitHub BLOCKED state prevent merge.

Priority: P0 — remote-input selection is a trust boundary and failed requests leak full clones.

Signed-off-by: Deepak Jain <deepujain@gmail.com>
Signed-off-by: Deepak Jain <deepujain@gmail.com>
@deepujain
deepujain force-pushed the feat/552-git-url-subdir branch from 8e070a8 to 2e8887a Compare September 18, 2026 23:09
Signed-off-by: Deepak Jain <deepujain@gmail.com>
@deepujain

Copy link
Copy Markdown
Contributor Author

All review comments are addressed on the current head 032436d:

  • rng1995 P0 (decoded path escapes via %2F/%2E%2E): fixed in 2e8887a (fix(input): reject encoded Git tree path escapes); decoded separators, absolute paths, and parent-bearing components are rejected before selection, with clone-root containment in place and regressions for encoded absolute and parent-bearing components.
  • yashrajp22 (owned checkout left on disk after post-clone selection failure): fixed in 032436d (fix(input): clean failed GitHub tree selections); failures now clean the owned temporary checkout, with missing-directory and file-target regressions.

CI is green on this head: https://github.com/NVIDIA/SkillSpector/actions/runs/35485659552 (lint, test-unit, docker-smoke, DCO). Per your 2026-09-18 note, this PR is ready for re-review.

GitHub branch names may contain slashes, so the first /tree/ segment is
not necessarily the complete ref. Resolve the longest advertised
branch/tag name via git ls-remote before splitting ref from subdirectory,
and reject URLs that name no known ref. Adds slash-ref regressions.

Signed-off-by: Deepak Jain <deepujain@gmail.com>
@deepujain

Copy link
Copy Markdown
Contributor Author

Addressed the remaining slash-containing ref issue in 896b82e: a GitHub branch name may itself contain slashes, so the first /tree/ segment is not necessarily the complete ref. The parser now resolves the longest advertised branch/tag name via git ls-remote before splitting ref from subdirectory, so /tree/feature/foo/sub clones branch feature/foo instead of branch feature. URLs naming no known ref are rejected with a clear error. Verified live against a real slash branch on this repo; regressions added for slash refs, longest-match preference, unknown refs, and ref-only URLs.

CI is re-running on this head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

support subdir within git url

4 participants