fix(ci): the invisible-character gate never matched anything - #86
fix(ci): the invisible-character gate never matched anything#86hyperpolymath wants to merge 1 commit into
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (2)
|
| Layer / File(s) | Summary |
|---|---|
Scanner pattern and command .github/workflows/dogfood-gate.yml |
The pattern uses Unicode code-point escapes and includes additional control and invisible formatting characters. The scan command uses grep -aPrl so binary files are processed as text. |
Estimated code review effort: 2 (Simple) | ~5 minutes
Merge Risk: ⚪ Minimal · up to 517fc
This localized workflow fix corrects invisible-character detection and adds handling for previously missed control characters; no actionable merge-blocking risk remains after normal checks.
Poem
A rabbit checks each hidden sign
Unicode marks now show in line
Binary files join the scan
Control codes face the gate’s plan
Clean text hops safely through
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Linked Issues check | The PR implements the codepoint escapes, C0 control coverage, and grep -a changes for dogfood-gate.yml. It does not show the separate leading-BOM check or the required updates to stdlib/ByteDetector.a… | Implement the separate byte-wise leading-BOM check and update stdlib/ByteDetector.affine and config.ncl so the compiled linter and CI gate have matching C0 control coverage. Confirm whether the estate-wide copies are in scope, and update th… |
✅ Passed checks (4 passed)
| Check name | Status | Explanation |
|---|---|---|
| Title check | ✅ Passed | The title clearly identifies the CI invisible-character gate defect and describes the main change. |
| Description check | ✅ Passed | The description explains the root cause, the implemented changes, and the verification performed. It does not use the repository headings or include the RSR checklist, but it provides the key required… |
| Out of Scope Changes check | ✅ Passed | The changes are limited to the CI invisible-character gate and align with the linked issue objectives. No unrelated changes are present. |
| Docstring Coverage | ✅ Passed | No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0… |
Full details: Description check
Explanation
The description explains the root cause, the implemented changes, and the verification performed. It does not use the repository headings or include the RSR checklist, but it provides the key required information.
Full details: Linked Issues check
Explanation
The PR implements the codepoint escapes, C0 control coverage, and grep -a changes for dogfood-gate.yml. It does not show the separate leading-BOM check or the required updates to stdlib/ByteDetector.affine and config.ncl described in issue #70.
Resolution
Implement the separate byte-wise leading-BOM check and update stdlib/ByteDetector.affine and config.ncl so the compiled linter and CI gate have matching C0 control coverage. Confirm whether the estate-wide copies are in scope, and update them or link a follow-up issue with clear scope separation.
Full details: Docstring Coverage
Explanation
No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
- Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
- Create stacked PR
- Commit on current branch
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
Up to standards ✅🟢 Issues
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
While this PR correctly identifies the limitations of the previous byte-sequence approach, the current implementation fails to function as a blocking gate and contains a regex error that will likely cause the scanning tool to fail silently.
Specifically, GNU grep -P cannot process Unicode escapes larger than \xFF without the (*UTF8) prefix. Because stderr is suppressed in the script, this results in the CI job reporting zero findings even when forbidden characters are present. Furthermore, the workflow lacks logic to exit with a non-zero status, meaning it does not actually 'gate' the PR as intended. Although Codacy reports the PR is up to standards, these logic and workflow gaps represent significant implementation risks.
About this PR
- The PR lacks automated regression tests. No sample files containing the targeted invisible characters (e.g., NBSP, ZWSP, or C0 controls) were added to the repository to verify the CI gate's functionality in a repeatable way.
1 comment outside of the diff
.github/workflows/dogfood-gate.yml
line 151🔴 HIGH RISK
Suggestion: The PR title refers to this as a 'gate', but the workflow currently does not exit with a non-zero status when invisible characters are found, which means it won't actually block a PR. To make this a functional gate, you should explicitly check the $FINDINGS count and exit with an error if it is greater than zero. Ensure the exit occurs after the while loop so that all file annotations are still processed before the step stops.
Test suggestions
- Verify detection of Non-Breaking Space (U+00A0)
- Verify detection of Zero-Width Space (U+200B)
- Verify detection of C0 Control character Backspace (\x08)
- Verify detection of Byte Order Mark (U+FEFF)
- Verify that files containing null bytes are scanned rather than skipped
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify detection of Non-Breaking Space (U+00A0)
2. Verify detection of Zero-Width Space (U+200B)
3. Verify detection of C0 Control character Backspace (\x08)
4. Verify detection of Byte Order Mark (U+FEFF)
5. Verify that files containing null bytes are scanned rather than skipped
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| # non-breaking spaces, null bytes, and other invisible Unicode in source files. | ||
| set +e | ||
| PATTERNS='\xc2\xa0|\xe2\x80\x8b|\xe2\x80\x8c|\xe2\x80\x8d|\xef\xbb\xbf|\xc2\xad|\xe2\x80\x8e|\xe2\x80\x8f|\xe2\x80\xaa|\xe2\x80\xab|\xe2\x80\xac|\xe2\x80\xad|\xe2\x80\xae|\x00' | ||
| PATTERNS='\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' |
There was a problem hiding this comment.
🔴 HIGH RISK
GNU grep -P requires the (*UTF8) prefix to support Unicode codepoint escapes greater than \xFF. Without it, grep will fail with a 'hexadecimal value is greater than 255' error. Since stderr is redirected to /dev/null on line 145 and the exit code is not checked for fatal errors, this will cause the linter to silently skip all files.
| PATTERNS='\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' | |
| PATTERNS='(*UTF8)\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' |
1 participant
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.