DO NOT MERGE: control experiment for #105 - #106
Closed
Shengyu Fu (shengyfu) wants to merge 3 commits into
Closed
Conversation
`is_ignore_rules_file` decides whether a filesystem event should rebuild and republish the matcher in `ServerState::gitignore`. It recognized `.gitignore` at any depth and root-level `p4ignore.ini`, but not `.ignore`. `.ignore` is a first-class ignore source everywhere else: the walk collects it separately, the matcher applies it, and it even outranks `.gitignore`. Unlike `.gitignore` it is not git-gated, so it is the one source that works outside a repository. Because the event never matched, a `.ignore` written while the server was live never scheduled the refresh, and the write then fell through to the reindex path where `should_skip_watcher_path` drops any dot-prefixed segment. Nothing happened at all: the startup matcher stayed published and files under the newly excluded directory kept being indexed until the hourly reconcile or a restart. Match `.ignore` by file name alongside `.gitignore`, using the existing `tgrep_core::gitignore` filename constants. `p4ignore.ini` stays root-scoped, mirroring the walker. Fixes #104 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The watcher already discarded events for ignored paths, but only after the OS had delivered them. On Linux that is too late to matter: inotify has no recursive mode, so notify's `RecursiveMode::Recursive` walks the tree and spends one watch descriptor per directory. A repository whose `target/` or `node_modules/` holds most of its directories therefore burns most of the per-user `fs.inotify.max_user_watches` budget on events that are thrown away -- and because notify propagates the first registration failure, exhausting that budget makes `watch()` return an error and the server loses its watcher entirely. Subscribe per directory on inotify backends instead. `watchable_dirs` walks the tree once, pruning ignored, hidden and `--exclude`d subtrees before descending, and `WatchRegistry::sync` reconciles the live subscription set against it. The sync runs whenever the ignore matcher is published, so relaxing a rule subscribes to the tree it used to hide and tightening one drops it. A directory that cannot be subscribed is now reported and skipped rather than taking down the whole watcher. Non-recursive watches are not extended by notify, so a directory that appears at runtime is picked up in `watch_new_subtree`, which also indexes the files already inside it to close the create race. Windows (ReadDirectoryChangesW) and macOS (FSEvents) subscribe once for the whole subtree, so there is no per-directory registration to withhold; they keep the single recursive watch and delivery-time filtering. The behaviour the two paths must share -- new directories get indexed, new directories under an ignored path do not -- is tested everywhere. Along the way, `state.gitignore` had three publish sites and only one of them went through the helper. They are unified behind `publish_ignore_matcher` so the sync hook cannot be missed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bcc85b55-e821-447e-b9a9-144b78829ae4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Temporary. Flips PER_DIRECTORY_WATCHES to false to confirm the new Linux inotify watch-count test actually fails without the selective registration in #105. Will be closed immediately.