fix(ci): the invisible-character gate never matched anything - #163
fix(ci): the invisible-character gate never matched anything#163hyperpolymath wants to merge 1 commit into
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (1)
|
| Layer / File(s) | Summary |
|---|---|
Update invisible-character matching .github/workflows/dogfood-gate.yml |
The pattern uses Unicode code-point escapes and includes additional control, bidi, and word-joiner characters. grep uses -a so binary files are scanned as text. |
Estimated code review effort: 2 (Simple) | ~5 minutes
Merge Risk: 🔵 Low · up to fc62b
The CI workflow now detects additional invisible characters, but its BOM pattern may not work with GNU grep 3.8, leaving one case undetected or causing the check to fail on affected runners. The PR is otherwise mergeable with explicit owner follow-up.
Poem
A rabbit checks each hidden mark,
In daylight code and files dark.
New code points join the careful sweep,
While binary files no longer sleep.
The gate now catches what should not creep.
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Linked Issues check | The change fixes the pattern in one dogfood-gate.yml, but issue [#70] also requires a separate leading-BOM check, matching C0 detection in stdlib/ByteDetector.affine and config.ncl, automated co… |
Implement all remaining coding requirements from issue [#70]: add the separate leading-BOM check, update stdlib/ByteDetector.affine and config.ncl, add or update automated tests for detection and exclusions, and apply the corrected patt… |
✅ Passed checks (4 passed)
| Check name | Status | Explanation |
|---|---|---|
| Title check | ✅ Passed | The title clearly identifies the main change: fixing the CI invisible-character gate. |
| Description check | ✅ Passed | The description directly explains the detection failure, root cause, implemented fixes, and verification. |
| Out of Scope Changes check | ✅ Passed | The reported change is limited to the invisible-character detection pattern and grep invocation in dogfood-gate.yml. It is related to the linked issue and contains no apparent out-of-scope changes… |
| Docstring Coverage | ✅ Passed | No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0… |
Full details: Linked Issues check
Explanation
The change fixes the pattern in one dogfood-gate.yml, but issue [#70] also requires a separate leading-BOM check, matching C0 detection in stdlib/ByteDetector.affine and config.ncl, automated coverage for required cases, and updates to the remaining estate-wide copies.
Resolution
Implement all remaining coding requirements from issue [#70]: add the separate leading-BOM check, update stdlib/ByteDetector.affine and config.ncl, add or update automated tests for detection and exclusions, and apply the corrected pattern to all remaining dogfood-gate.yml copies.
Full details: Out of Scope Changes check
Explanation
The reported change is limited to the invisible-character detection pattern and grep invocation in dogfood-gate.yml. It is related to the linked issue and contains no apparent out-of-scope changes.
Full details: Docstring Coverage
Explanation
No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
- Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
- Create stacked PR
- Commit on current branch
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
Up to standards ✅🟢 Issues
|
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
While this PR correctly identifies the need to switch from byte sequences to Unicode escapes and adds the necessary -a flag for grep, it contains a critical logic flaw in the regex definition. The PCRE engine used by grep -P cannot process Unicode codepoints above 255 without being explicitly told to operate in UTF-8 mode.
As currently implemented, the PATTERNS variable will cause grep to error out. Due to the redirection of stderr to /dev/null in the execution step, this error will be hidden, and the gate will report zero findings, effectively remaining broken. Additionally, there are no automated tests or 'dirty' sample files included to verify that this gate actually catches the intended characters.
About this PR
- The PR relies on manual verification but does not add automated test cases or sample 'dirty' files to the repository. Without these, it is difficult to ensure the linter remains functional and does not regress in the future.
Test suggestions
- Missing: Verify detection of a Non-Breaking Space (U+00A0) in a source file
- Missing: Verify detection of a Zero-Width Space (U+200B) in a source file
- Missing: Verify detection of a C0 control character, such as Backspace (\x08)
- Missing: Verify that files containing NUL bytes (\x00) are scanned rather than skipped as binary
- Missing: Verify that valid whitespace (Tab, LF, CR) does not trigger the gate
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Missing: Verify detection of a Non-Breaking Space (U+00A0) in a source file
2. Missing: Verify detection of a Zero-Width Space (U+200B) in a source file
3. Missing: Verify detection of a C0 control character, such as Backspace (\x08)
4. Missing: Verify that files containing NUL bytes (\x00) are scanned rather than skipped as binary
5. Missing: Verify that valid whitespace (Tab, LF, CR) does not trigger the gate
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| # non-breaking spaces, null bytes, and other invisible Unicode in source files. | ||
| set +e | ||
| PATTERNS='\xc2\xa0|\xe2\x80\x8b|\xe2\x80\x8c|\xe2\x80\x8d|\xef\xbb\xbf|\xc2\xad|\xe2\x80\x8e|\xe2\x80\x8f|\xe2\x80\xaa|\xe2\x80\xab|\xe2\x80\xac|\xe2\x80\xad|\xe2\x80\xae|\x00' | ||
| PATTERNS='\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' |
There was a problem hiding this comment.
🔴 HIGH RISK
To ensure that Unicode code points are correctly matched in UTF-8 source files, the PCRE engine must be told to operate in UTF-8 mode. Without the (*UTF) or (*UTF8) prefix, the regex will fail to compile for codepoints greater than 255 (e.g., \x{200b}), and since stderr is suppressed on line 136, the check will silently fail and report no issues.
| PATTERNS='\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' | |
| PATTERNS='(*UTF)\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' |
| -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ | ||
| -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ | ||
| -exec grep -Prl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | ||
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
There was a problem hiding this comment.
⚪ LOW RISK
Nitpick: The -r flag is redundant when grep is executed by find on individual file paths.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
1 participant
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.