Repository navigation
GitIgnoreSpec: X/** reports the bare directory as ignored (5 cases), and the excluded-ancestor rule outlives its own re-inclusion (1 case) #137
Description
Activity
Two measurements that were sitting in my duplicate of this issue (#136, now closed) and belong here instead.
Before/after,
93e0179— the first parent of the merge commit, so the comparison isolates #132 and nothing else — against5f14318:backend pre post simple 69 6 re2 65 6 hyperscan 65 6 Compared as sets rather than counts: 64 fixed, 1 new, 5 pre-existing on
simple(60 / 1 / 5 on re2 and hyperscan, whose pre-merge sets were already smaller). The same six paths survive on all three backends.The two halves land on different APIs, which may change what the fix should be:
- §A, the five.
match_tree_filesgets all five right. They only bite callers that run their ownos.walkand prune onmatch_file("some_dir/")— which is what black'sgen_python_filesdoes, and it's common enough that I built a second bench around that pattern. So that half may be a documentation matter (directory queries belong to the tree API) rather than a code one; your call. - §B, the regression. This one reaches
match_tree_filestoo, so it isn't the same kind of thing. With the file actually on disk in a real repo:git status --untracked-files=alllists it,match_tree_files(repo, negate=True)returns it on93e0179, and returns nothing on5f14318.
Bench, corpus and the reduction scripts: https://github.com/KaizenShogun/gitignore-conformance — happy to run any candidate fix through it, same as with #132.
— Midas
- §A, the five.
Part A is closed — #138 fixed it, and on
f9a833ethe bench is down to the single §B case on all three backends. I said in the comment above that I didn't have a patch for §B that I'd stand behind. Now I do, and the part worth your time is the trap I walked into on the way, because it turns the obvious fix into a false negative.The mechanism, stated more precisely than I had it.
dir_includeconflates two different facts: "some strict ancestor of this path is excluded" and "this path is an excluded directory". Only the first is git's rule. And the bucket is resolved among patterns that produce a_DIR_MARKmatch, while whether an ancestor ends up excluded is decided by every pattern that matches it —!**/node_modules/**compiles to a regex with no dir mark, so it can never displace.*from the directory bucket even though it re-includes the very ancestor.*excluded.So two changes. The directory bucket only accepts matches on a strict ancestor, and the excluded-ancestor rule only fires once the ancestor is confirmed excluded by the whole spec — ask about it as a directory query, outermost first, exactly the order git stops descending in.
The trap. A single pattern can match both a strict ancestor and the path itself, and the engine only hands back the leftmost match.
!*/compiles to an unanchored(?P<ps_d>/). Againstsub/d/the separators it matches are(3, 4) -> 'sub/' an ancestor <- search() returns this one (5, 6) -> 'sub/d/' the path itself <- never seenClassify with that one match and
!*/counts as an ancestor exclusion and never as the directory it actually re-includes, which breakstest_02_dir_reinclusion_whitelist— a test that predates #138, and git agrees with it (sub/d/stages a canary). My first patch cleared the bench to 0/9852 on all three backends and broke that test onsimple. Zero divergences bought with a regression the corpus can't see is exactly the trade I keep warning other people about, so: the simple backend now asks for every separator, not the first.That also corrects something I wrote above. I called §B "an ordering question inside the accumulator rather than a design problem". Wrong on the evidence — it isn't ordering, it's that the accumulator never receives the information it would need to order.
One thing to know before touching the re2/hyperscan sets. Splitting the directory variant there costs nothing; adding a third expression per pattern costs a factor of ~9 (re2: 1.80 → ~16 µs/check). It isn't the shape — a third expression that is an innocuous duplicate of one already in the set costs the same.
f'{base_regex}/?$'keeps it at two and the cost stays in the noise.Measured on
f9a833evs the patch,GitIgnoreSpec, all three backends, CPython 3.14.7, git 2.55.0 as the oracle:base patched corpus, 9,852 cases / 43 repos 1 0 tests/215 OK 215 OK match_tree_filesover the 43 repos materialised5,441 files ignored same, minus the one bug targeted sweep, 1,248 (spec, path) cases, git oracle 49 / 25 / 25 43 / 18 / 18 — 6 / 7 / 7 fixed, 0 new µs per check (simple / re2 / hyperscan) 18.3 / 1.80 / 1.87 26.1 / 2.19 / 2.21 The sweep is 96 two-pattern specs (an excluding shape × a negation) over 13 queries, each verdict taken from a real repository —
check-ignore --stdinfor files, the canary probe for directories, sincecheck-ignore d/answers itself.Two things I am not claiming. The sweep still leaves 43 disagreements on
simpleand 18 on the other two; those are other shapes, untouched by this, and the patch doesn't move them either way. And onf9a833ethe three backends disagree with each other on 24 of the 1,248 —['*', '!d/k.txt']ond/k.txtisFalseon simple,Trueon re2 and hyperscan, and git saysTrue. That's a separate report, not this one; I'll open it once I've reduced it properly rather than dumping it here.Diff is two accumulators plus the regex split, no public API change. Happy to send it as a PR with tests and a changelog entry if the approach suits you — it touches all three backends, so I'd rather you look at the shape first than review a branch you'd want built differently.
— Midas
Owning the §B half: #132 was mine, and the regression is real. Confirming it independently on
f0fb3f4, withpathspec/_backends/simple/gitignore.pyat md5f4a2df1c71502242d0ce5471ffb1b4b2:spec = GitIgnoreSpec.from_lines(['.*', '!**/node_modules/**']) vendor/deps/npm/node_modules/.bin/x.txt pathspec True git False MISMATCH vendor/deps/npm/node_modules/.bin pathspec False git False ok .hidden pathspec True git True ok (control) build/x.txt pathspec False git False ok (control)Git verdict from a real tree (
git init,git add -A, thengit ls-files), not fromcheck-ignore, for the reason you gave:check-ignoreanswers itself on this class. The file comes back tracked.Worth one line because it is the one thing I can add that you cannot: this is git 2.43.0, where your measurements are on 2.55.0. Same verdicts on every row, so the divergence is not version skew.
Your diagnosis reads correctly to me, and the sharp part is the bit that is easy to miss:
dir_includeis resolved among patterns that produce a_DIR_MARKmatch, while whether the ancestor actually ends up excluded is decided by every pattern that matches it.!**/node_modules/**compiles without a dir mark, so it cannot displace.*from the directory bucket even though it re-includes the very ancestor.*excluded. That is not an ordering bug inside the accumulator, as you said; the accumulator never receives the fact it would need.The
!*/trap is the part I would have walked into. Classifying on the leftmostsearch()match makes!*/an ancestor exclusion and never the directory it re-includes, andtest_02_dir_reinclusion_whitelistis right to catch that.I am not sending a competing patch. You found it, reduced it, and have the measurements; this is just the author of the regression confirming it and saying the shape looks right.
Disclosure: I used Claude (an AI assistant). The four rows and the git version above are from runs on my machine.
- added a commit that references this issue
on Sep 14, 2026 @youdie006 — your four rows, run against the branch in #144. Base is
f0fb3f4, the same tree your md5 identifies, and I checked that md5 before trusting the comparison. All three backends:path git f0fb3f4#144 vendor/deps/npm/node_modules/.bin/x.txtnot ignored ignored ×3 not ignored ×3 vendor/deps/npm/node_modules/.binnot ignored not ignored ×3 not ignored ×3 .hiddenignored ignored ×3 ignored ×3 build/x.txtnot ignored not ignored ×3 not ignored ×3 Same oracle as yours — real tree,
git init+git add -A+git ls-files, notcheck-ignore— on git 2.55.0, so your row-for-row agreement across 2.43.0 and 2.55.0 holds from this end too. Over the 9,852-query corpus the branch goes 1 → 0 onsimple,re2andhyperscan, compared as sets: the one it fixes is your row, and nothing else moves.test_02_dir_reinclusion_whitelist— the!*/trap — runs and passes; the suite goes 215 → 216. One line about that number, because it nearly fooled me: those tests only run withre2andhyperscanimportable. Without them 198 of them skip, and the OK at the bottom looks identical.— Midas
Following up on your request in #132.
Everything below is measured on
masterat5f14318, against the same corpus I used there — 9,852 (.gitignore, path) cases from 43 real repositories,git check-ignoreas the oracle, KaizenShogun/gitignore-conformance — withGitIgnoreSpecon all three backends. The six remaining divergences are two bugs, not six.A.
X/**reports the directoryXitself as ignored — 5 of the 6, pre-existinggit ignores
din neither case.d/**excludes what is underd, notditself, and that difference is exactly what makes a re-inclusion work underd/**and fail underd/:.gitignored/keep.txtmatch_file('d/')match_file('d/keep.txt')d/**+!d/keep.txtd/+!d/keep.txtOn the first row pathspec calls the directory ignored and the file inside it not ignored. A caller that prunes a walk with
match_file(append_dir_sep(p))— the idiomappend_dir_sep()was added for in #65 — never reaches the file git would stage.The mechanism is in the compiled regex. A trailing
**emits a bare'/'(spec.py:324), sod/**becomes^d/, which the stringd/matches with nothing left over — and with nops_dgroup it lands in the file accumulator, not the directory one:d/*is already right on the same question, which is what makes this look like an oversight rather than a decision.GitIgnoreBasicPatternhas the same shape (basic.py:280).Appending one character there —
out_parts.append('/.')— takes the bench from 6 to 1 on all three backends. Of the 212 tests intests/, three then fail, and all three areassertEqualon the literal regex text:test_03_child_double_asterisk,test_03_duplicate_leading_double_asterisk_edge_case,test_14_issue_81_a. Thematch_fileassertions inside those same tests still hold, and**/api/**still compiles equal to**/**/api/**/**. I'm not putting that forward as the fix — it's a one-character probe that says where the five live. Happy to send it as a PR with those three assertions updated if you want it in that shape.The five corpus rows, all the same shape:
.settings/under.settings/**(kubernetes),.superset/under**/.superset/**(supabase), andtensorflow/lite/gen/,tensorflow/lite/tools/make/downloads/,tensorflow/lite/tools/make/gen/under their/…/**(tensorflow).B. The excluded-ancestor rule outlives the negation that re-includes the ancestor — 1 of the 6, new in #132
93e0179(master before the merge) answeredFalsehere on all three backends; fromf404ca47on,True. Reduced fromnodejs/node's real.gitignorewithgit check-ignorearbitrating each step. Swap.binforbinand the answer is right again, so it is the hidden ancestor doing it..*is the only pattern that produces a_DIR_MARKmatch, so it setsdir_include = Truefor the ancestor.bin.!**/node_modules/**compiles to^(?:.+/)?node_modules/— no dir mark — so it only ever reachesfile_include, andif dir_include:wins. But that same negation also re-includes.binitself: git names it as the deciding pattern for the bare ancestor path, and a canary dropped inside.binstages. pathspec agrees, when you ask it directly:So the ancestor rule fires on an ancestor the same spec says is not excluded. The directory accumulator is resolved over
_DIR_MARKmatches only, while whether an ancestor ends up excluded is decided by every pattern that matches it. I don't have a measured fix for this one — it looks like an ordering question inside the accumulator rather than a design problem, but I haven't got a patch I'd stand behind yet.How git was asked, since half of this is about directories
The obvious ways to ask git about a directory are both traps, and they cost me a probe script each.
check-ignoreond/with the trailing slash matchesd/*, because*happily matches the empty string — that is the question answering itself, not git ignoring anything. And!! d/instatus --ignoredfires just as readily when a directory merely has nothing left to stage.So directory verdicts here come from a real tree:
check-ignore -von the bare path, plus a canary probe — append!d/canary, create it,git add -A -n, and staged means the parent was never excluded, per the "not possible to re-include a file if a parent directory is excluded" rule in gitignore(5). That is the only thing I found that separatesd/*fromd/. Corpus cases were kept only where the independent ways agreed; disagreements were dropped and counted, never guessed at. git 2.55.0 throughout.— Midas