Skip to content

[WEB-4211] feat: skip specific folders to not parse #120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MohamedAklamaash wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[WEB-4211] feat: skip specific folders to not parse #120

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
90b7617
feat: skip specific folders to not parse
MohamedAklamaash May 5, 2026
032217b
fix: added comma
MohamedAklamaash May 5, 2026
4216d5d
fix: fixed type error
MohamedAklamaash May 5, 2026
7cff01d
Merge remote-tracking branch 'origin/main' into HEAD
MohamedAklamaash Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion scripts/coding_discovery_tools/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@
MAX_SEARCH_DEPTH = 10 # Maximum directory depth to search recursively
SKIP_DIRS = frozenset[str]({

@greptile-apps greptile-apps Bot Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Python 3.8 import break

frozenset[str](...) still raises TypeError on Python 3.8 because built-in collection generics are not subscriptable there. This file is imported by the extraction helpers before traversal starts, so a Python 3.8 runtime can fail at module import instead of running discovery. The runtime type parameter is not needed here, and the other frozenset constants in this file already use the compatible form.

Suggested change
SKIP_DIRS = frozenset[str]({
SKIP_DIRS = frozenset({

'.git', 'node_modules', 'venv', '__pycache__', '.venv', 'vendor', '.idea', '.vscode', 'Library', '.Trash', '.cache',
'Photos', 'Music', 'Movies', 'Pictures', 'Videos'
'Photos', 'Music', 'Movies', 'Pictures', 'Public', 'Templates', 'Videos'
})
Comment thread
greptile-apps[bot] marked this conversation as resolved.
Comment on lines +38 to +41

@greptile-apps greptile-apps Bot May 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 frozenset[str](...) uses the generic-alias call syntax introduced in Python 3.9. On Python 3.8, frozenset[str] raises TypeError: 'type' object is not subscriptable at import time, breaking all platforms. The type annotation adds no runtime value here; the idiomatic form is just frozenset({...}).

Suggested change
SKIP_DIRS = frozenset[str]({
'.git', 'node_modules', 'venv', '__pycache__', '.venv', 'vendor', '.idea', '.vscode', 'Library', '.Trash', '.cache',
'Photos', 'Music', 'Movies', 'Pictures', 'Public', 'Templates', 'Videos'
})
SKIP_DIRS = frozenset({
'.git', 'node_modules', 'venv', '__pycache__', '.venv', 'vendor', '.idea', '.vscode', 'Library', '.Trash', '.cache',
'Photos', 'Music', 'Movies', 'Pictures', 'Public', 'Templates', 'Videos'
})


# System directories to skip when searching from root (macOS/Unix)
SKIP_SYSTEM_DIRS = {
'/System', '/Library', '/private', '/usr', '/bin', '/sbin', '/opt',
Expand Down
5 changes: 3 additions & 2 deletions scripts/coding_discovery_tools/macos_extraction_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,14 @@
import os
from datetime import datetime
from pathlib import Path
import re
from typing import List, Dict, Optional, Tuple

from .constants import MAX_CONFIG_FILE_SIZE, MAX_SEARCH_DEPTH, SKIP_DIRS, SKIP_SYSTEM_DIRS
from .mcp_extraction_helpers import is_home_dotdir_descendant

logger = logging.getLogger(__name__)

SKIP_PATTERN = re.compile("|".join(map(str, map(re.escape, SKIP_DIRS))), re.IGNORECASE)
Comment thread
greptile-apps[bot] marked this conversation as resolved.

def is_running_as_root() -> bool:
"""
Expand Down Expand Up @@ -88,7 +89,7 @@ def should_skip_path(path: Path) -> bool:
Returns:
True if path should be skipped, False otherwise
"""
return any(part in SKIP_DIRS for part in path.parts)
return any(SKIP_PATTERN.search(part) for part in path.parts)


def should_skip_system_path(path: Path) -> bool:
Expand Down
6 changes: 4 additions & 2 deletions scripts/coding_discovery_tools/windows/cursor_cli/settings_extractor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@
from pathlib import Path
from typing import Optional, List, Dict

@greptile-apps greptile-apps Bot May 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Same cross-platform coupling as windows_extraction_helpers.py

This file also imports SKIP_PATTERN directly from macos_extraction_helpers. Moving the compiled pattern (or its construction) to constants.py would remove this dependency for both Windows files at once.

@MohamedAklamaash MohamedAklamaash May 5, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skip dirs obj is being shared b/w windows and mac os as of now, we should move out of this idea and create a different obj for windows all together.

@greptile-apps greptile-apps Bot May 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense — platform-specific skip patterns give you more flexibility to diverge the behavior later (e.g., Windows-only paths like AppData, $Recycle.Bin that don't exist on macOS). In that case, defining a separate SKIP_DIRS / pattern in windows_extraction_helpers.py (or a windows_constants.py) and removing the import from macos_extraction_helpers entirely is the right call. The cross-platform coupling comment is no longer applicable here.

Tip: You can give me more context with .greptile/rules.md so I get this right next time.


from ...constants import MAX_SEARCH_DEPTH, SKIP_DIRS, WINDOWS_SKIP_USER_DIRS
from scripts.coding_discovery_tools.macos_extraction_helpers import SKIP_PATTERN as SKIPDIR_PATTERN

from ...constants import MAX_SEARCH_DEPTH, WINDOWS_SKIP_USER_DIRS
from ...windows_extraction_helpers import (
is_running_as_admin,
read_file_content,
Expand Down Expand Up @@ -182,7 +184,7 @@ def _walk_for_cursor_cli_settings(
if not entry.is_dir():
continue

if entry.name in SKIP_DIRS or entry.name in system_dirs:
if SKIPDIR_PATTERN.search(entry.name) is not None or entry.name in system_dirs:
continue

if entry.name == self.CURSOR_DIR_NAME:
Expand Down
6 changes: 4 additions & 2 deletions scripts/coding_discovery_tools/windows_extraction_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@
from pathlib import Path
from typing import List, Dict, Optional, Tuple, Callable

from .constants import MAX_CONFIG_FILE_SIZE, SKIP_DIRS
from scripts.coding_discovery_tools.macos_extraction_helpers import SKIP_PATTERN

@greptile-apps greptile-apps Bot May 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Windows helper importing macOS module

windows_extraction_helpers.py now imports SKIP_PATTERN from macos_extraction_helpers via an absolute path. This creates a cross-platform coupling: if macos_extraction_helpers ever gains a macOS-specific import (e.g. AppKit, subprocess calling dscl), this import will fail on Windows. SKIP_PATTERN is derived solely from SKIP_DIRS (a plain constant), so it belongs in constants.py alongside SKIP_DIRS and should be imported from there by both platforms.


from .constants import MAX_CONFIG_FILE_SIZE

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -155,7 +157,7 @@ def should_skip_path(path: Path, system_dirs: Optional[set] = None) -> bool:
True if path should be skipped, False otherwise
"""
# Skip common project directories (check all path parts for nested matches)
if any(part in SKIP_DIRS for part in path.parts):
if any(path for path in path.parts if SKIP_PATTERN.search(path) is not None):
return True

# Skip system directories if provided (Windows-specific)
Expand Down
Loading