fix(ci): the invisible-character gate never matched anything - #66
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (9)
🔇 Additional comments (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe workflow’s invisible-character scan now uses Unicode code-point escapes, detects more C0 and directional formatting characters, and searches binary files as text. ChangesInvisible-character gate
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This change corrects invisible-character detection in the CI gate and adds coverage for missed control characters and binary files. No actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Linked Issues checkExplanation The PR implements codepoint escapes, C0 control detection, and grep -a in the CI gate [ Resolution Add the byte-wise leading-BOM check. Update stdlib/ByteDetector.affine and config.ncl with the same C0 control handling. Verify the compiled linter and CI gate against all required positive and negative test cases, including a leading BOM and the real backspace-corrupted workflow. Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Up to standards ✅🟢 Issues
|
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
The PR addresses the failure of the invisible-character CI gate by transitioning from byte-sequence patterns to PCRE codepoint escapes and utilizing the -a flag to ensure files containing NUL bytes are scanned. While Codacy results are up to standards, there is a critical gap in automated testing. All recommended test scenarios for validating the regex changes (NBSP detection, C0 control characters, and binary-file handling) are currently missing from the implementation. This increases the risk of regression in the future.
About this PR
- The PR lacks automated regression tests within the codebase to verify the updated regex patterns against the test cases mentioned in the description. It is recommended to add a test suite or a mock file set that triggers these patterns to ensure the CI gate remains functional.
Test suggestions
- Detect Non-Breaking Space (U+00A0) in a supported source file
- Detect C0 control characters such as Backspace (\x08) in a source file
- Verify that files containing NUL bytes (\x00) are scanned and flagged rather than skipped as binary
- Ensure valid whitespace (TAB, LF, CR) does not trigger the invisible character gate
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Detect Non-Breaking Space (U+00A0) in a supported source file
2. Detect C0 control characters such as Backspace (\x08) in a source file
3. Verify that files containing NUL bytes (\x00) are scanned and flagged rather than skipped as binary
4. Ensure valid whitespace (TAB, LF, CR) does not trigger the invisible character gate
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ | ||
| -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ | ||
| -exec grep -Prl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | ||
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
There was a problem hiding this comment.
🟡 MEDIUM RISK
Suggestion: The use of -r (recursive) is redundant since find already provides individual files. More importantly, 2>/dev/null silences all error output, which can hide critical issues like PCRE compilation errors or invalid UTF-8 sequences. For better performance, use + to process multiple files per process, and add -- to handle filenames starting with dashes.
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | |
| -exec grep -aPl "$PATTERNS" -- {} + > /tmp/empty-lint-results.txt |
1 participant
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.