Detect linter, formatter and type checker config files

CHANGELOG.md

@@ -4,7 +4,24 @@ The format of this changelog is based on [Keep a Changelog](https://keepachangel
 
 ## [Unreleased]
 
+### Added
+ - Detection of linter, formatter and type checker configuration files and their ignore counterparts: 30 tools across JavaScript, TypeScript, CSS, Python, Go, Ruby, Java, Groovy, C#/.NET and SQL, covering 96 exact filenames. Every convention was verified against the tool's current official documentation; see [`docs/linter-detection-rules.md`](https://panopticas.io/linter-detection-rules) for the inventory, the sources, and the candidates that were investigated and rejected.
+ - `LINTER_RULES` in `constants.py` is the single table describing those tools, expanded into the existing `METADATA_RULES` tables at import. `LINTER_ROLES` constrains the legal roles.
+ - Two new role tags, `formatter` and `typechecker`, joining the existing `linter`. A tool may hold more than one — Ruff, Biome, RuboCop, Standard and SQLFluff all lint and format.
+ - `.mjs` and `.cjs` now report as JavaScript, `.mts` and `.cts` as TypeScript. These were missing from the extension table entirely, so every file using them reported `Unknown` regardless of linters.
+ - Filetypes for extensionless config dotfiles that previously reported `Unknown`: `.editorconfig`, `.flake8`, `.globalconfig`, `.pylintrc`, `pylintrc`, `.isort.cfg` and `.style.yapf` resolve to `INI`, `.jshintrc` to `JSON`, `staticcheck.conf` to `TOML`, `.rufo` and `Steepfile` to `Ruby`, and `*.ruleset` to `XML`. Files accepting several formats get their own name: `ESLintRC`, `Prettierrc`, `StylelintRC`, plus `ESLintIgnore`, `PrettierIgnore`, `StylelintIgnore`, `JSHintIgnore` and `YapfIgnore`. `JSONC` and `JSON5` were also added.
+
+### Changed
+ - The tag vocabulary grew from 85 to 118 tags and the filetype vocabulary from 76 to 86. Both are still derived from the rules, so neither can drift.
+ - `eslint.config.js` tagged `eslint`; it now tags `ESLint`, matching the brand casing every other tool uses, and gains a `TypeScript` tag. kospex matches tags with a case-insensitive `LIKE`, so stored queries are unaffected.
+ - `.sqlfluff` and `.sqlfluffignore` moved into `LINTER_RULES`. Purely additive: `.sqlfluff` keeps `SQLFluff`, `SQL` and `linter` and gains `formatter` and `config`.
+
+ All tag changes are additions or case-only, so kospex is compatible either way; a bump is needed only to pick up the new tags, which requires a re-sync.
+
 ### Fixed
+ - `check_shebang()` caught only `FileNotFoundError` and `UnicodeDecodeError`, so any other way `open()` can fail propagated out of `get_language()`. A single unreadable file aborted a whole `panopticas assess` scan with `PermissionError`, and passing a directory raised `IsADirectoryError`. It now returns `None` for every path it cannot read, letting `get_language()` fall back to the extension tables. Passing a filename that does not exist on disk was already safe and still is.
+ - `panopticas urls` aborted the whole scan on one unreadable file. The loop caught `UnicodeDecodeError` — added in 0.0.19 so a binary file could not kill the run — but not `OSError`. Both are now treated as a file with no URLs. `extract_urls_from_file()` still raises by design; the handling belongs in the caller that is scanning a tree.
+ - `panopticas file` reported no file type for any file typed by basename rather than by extension, because it read `get_extension_filetype()`, which consults only `EXT_FILETYPES`. `panopticas file go.mod` showed a blank file type while `panopticas assess` showed `go.mod`; `setup.cfg` was blank the same way. The row now reads `get_language(skip_shebang=True)`, so both commands agree. The shebang rows are unaffected — a shebang-only script still reports no file type rather than borrowing its interpreter.
  - `panopticas file` labelled its first table row `File extenion`. Cosmetic, and the only change to shipped code since 0.0.19 — no library function, tag or JSON field changes, so kospex is unaffected and its pin does not need bumping for this
 
 ## 0.0.19 - 2026-08-15

CLAUDE.md

@@ -45,6 +45,41 @@ Rules to follow when adding a product:
 - `exact_filename` and `path_contains` keys are lowercase; path fragments end with `/`.
 - **Verify the convention against the product's current official docs before adding it.** A wrong rule mislabels a repository and the label flows into kospex. A missing rule is better than a wrong one. Watch for user-level paths (`~/.config/<tool>/`) — those are not repository artifacts and must not be added.
 
+### Linter, Formatter and Type Checker Detection
+
+> **Before adding or changing a rule, read [`docs/linter-detection-rules.md`](docs/linter-detection-rules.md)** — the full inventory, the source confirming each convention, and the candidates already investigated and **rejected**. Eleven were rejected as wrong; re-adding one would mislabel repositories.
+
+`LINTER_RULES` in `constants.py` describes each tool once — its languages, its
+roles, and the files that identify it:
+
+```python
+"ESLint": {
+    "languages": ["JavaScript", "TypeScript"],
+    "roles": ["linter"],
+    "config": ["eslint.config.js", ..., ".eslintrc.json"],
+    "ignore": [".eslintignore"],
+},
+```
+
+`_expand_linter_rules()` writes those into the existing `METADATA_RULES`
+tables at import — `config`/`ignore` into `exact_filename_rules`,
+`path_contains` into `path_contains_rules`, `extensions` into
+`extension_rules`. Tags are `languages + roles + [tool] + ["config"|"ignore"]`.
+`get_tags()` reads the expanded tables, so a new tool joins the vocabulary with
+no other change.
+
+Rules to follow when adding a tool:
+
+- Detection is **path-based only** — never open a file.
+- `roles` must come from `LINTER_ROLES` (`linter`, `formatter`, `typechecker`). A tool may hold several; Ruff, Biome, RuboCop, Standard and SQLFluff all lint and format.
+- Tool names use the vendor's own brand casing (`ESLint`, `RuboCop`, `golangci-lint`, `mypy`). This is the one thing the pre-existing rules got inconsistent.
+- `languages` names what the tool actually targets, not the ecosystem it ships in — Stylelint is `CSS`, not `JavaScript`. Leave it empty for genuinely neutral tools (`.editorconfig`).
+- `config` and `ignore` keys are lowercase; path fragments end with `/`.
+- Generated exclusion files are `ignore`, not `config` — `.rubocop_todo.yml` excludes offences.
+- **Do not add shared config files** (`pyproject.toml`, `setup.cfg`, `tox.ini`, `package.json`). They really do hold linter config, but which tool is inside is not knowable from the path, so tagging them mislabels nearly every repository that has one.
+- **Verify the convention against the tool's current official docs before adding it.** Watch for user-level paths — revive's only auto-discovered config is `$HOME/revive.toml` and was rejected for exactly this reason. A missing rule is better than a wrong one.
+- If a tool's config file has no extension, give it a filetype too: map to the documented content format where the tool documents one, otherwise give it its own name. Add every new filetype name to `NON_LANGUAGE_FILETYPES` — a test asserts the classification is complete.
+
 ## Development Commands
 
 ### Installation (Development)

changes/2026-10-linter-detection.md

@@ -0,0 +1,197 @@
+# Linter, formatter and type checker detection
+
+**Version:** 0.0.20
+**Date:** 2026-10-05
+
+## What changed
+
+Panopticas now detects the configuration and exclusion files of code-quality
+tooling — 30 tools, 96 exact filenames — across JavaScript, TypeScript, CSS,
+Python, Go, Ruby, Java, Groovy, C#/.NET and SQL.
+
+Before this change the only linters detected were ESLint's flat config and
+SQLFluff, and the two rules disagreed on tag shape: `eslint.config.js` tagged
+`["JavaScript", "linter", "eslint", "config"]` while `.sqlfluff` tagged
+`["SQLFluff", "SQL", "linter"]` — lowercase versus brand-cased tool name, with
+and without a `config` role.
+
+## Design
+
+### One table, not ninety-six literal rules
+
+`LINTER_RULES` in `constants.py` describes each tool once:
+
+```python
+"ESLint": {
+    "languages": ["JavaScript", "TypeScript"],
+    "roles": ["linter"],
+    "config": ["eslint.config.js", ..., ".eslintrc.json"],
+    "ignore": [".eslintignore"],
+},
+```
+
+`_expand_linter_rules()` writes those into the existing `METADATA_RULES`
+tables at import: `config` and `ignore` into `exact_filename_rules`,
+`path_contains` into `path_contains_rules`, `extensions` into
+`extension_rules`. Tags are `languages + roles + [tool] + ["config"|"ignore"]`.
+
+This was chosen over literal dict entries for three reasons. Prettier alone
+accepts 18 config filenames that all carry identical tags, so the literal form
+is 18 near-duplicate lines. The tag shape becomes structurally guaranteed
+rather than a convention each entry has to remember — which is precisely what
+went wrong between the `eslint` and `SQLFluff` rules. And the table reads as
+the tool inventory, mirroring how `AI_RULES` already works.
+
+Nothing downstream changed: `get_tags()` traverses `METADATA_RULES` after
+expansion, so the new tags joined the vocabulary with no change to `core.py`
+or `cli.py`. The existing vocabulary drift test in `tests/test_vocabulary.py`
+picked up all 96 rules for free.
+
+### Language tags name what a tool targets
+
+ESLint gets `JavaScript` and `TypeScript`; Stylelint gets `CSS` and
+deliberately not `JavaScript`; `.editorconfig` is language-neutral and carries
+no language tag at all. Prettier is capped at its three main targets rather
+than the seven parsers it ships, to keep one config file from carrying seven
+language tags.
+
+### Three roles
+
+`linter`, `formatter` and `typechecker`, constrained by `LINTER_ROLES`. A tool
+may hold several, and Ruff, Biome, RuboCop, Standard and SQLFluff all lint and
+format. `.rubocop_todo.yml` and `.standard_todo.yml` are classified `ignore`
+rather than `config` — both are generated to exclude existing offences, which
+makes them the exclusion file rather than a second config.
+
+## Research
+
+Every convention was verified against the tool's current official
+documentation before being added. The full evidence base, the sources, and the
+rejected candidates are in `docs/linter-detection-rules.md`.
+
+Eleven candidates were investigated and **rejected**, each of which would have
+been a wrong rule:
+
+- **revive** auto-discovers only `$HOME/revive.toml` — a user-level path, not
+  a repository artifact.
+- **SpotBugs**, **PMD** and **CodeNarc** take arbitrarily-named filter and
+  ruleset files passed explicitly on the command line.
+- **Black**, **StandardJS** and **Spotless** have no dedicated config file.
+- **gofmt**, **gofumpt** and **go vet** have no configuration at all.
+- **pycodestyle**'s dedicated file is user-level.
+- **dprint** is plausible but its filename list could not be confirmed.
+- Security scanners (Sonar, Bandit, Brakeman) are out of scope.
+
+`pyproject.toml`, `setup.cfg`, `tox.ini` and `package.json` genuinely hold
+linter configuration, but which tool is inside is not knowable from the path,
+and detection here is path-based only. Tagging them would mislabel nearly
+every repository that has one, so they keep only their existing build and
+dependency tags.
+
+## Side-effect: four missing extensions
+
+The work surfaced a gap unrelated to linters. `.mjs`, `.cjs`, `.mts` and
+`.cts` were absent from `EXT_FILETYPES` entirely, so every file using them
+reported `Unknown` and `get_languages()` missed them. They are now JavaScript
+and TypeScript respectively, matching GitHub Linguist, which types all four as
+`programming`. They surfaced here because config authors use them to escape
+the `"type"` field in `package.json`, which is why `eslint.config.mjs` and
+`.prettierrc.cjs` are common.
+
+## Filetype resolution
+
+Extensionless config dotfiles previously reported `Unknown`. They now resolve
+on one principle, which reconciles the two precedents already in the codebase
+(`.sqlfluff` got its own name; `setup.cfg` was mapped to `INI` "because it is
+read by configparser"): **where a tool documents a single content format, map
+to that format; where several are accepted, the file gets its own name.**
+
+`.cfg` and `.conf` are mapped by basename, not extension, because both are
+used for arbitrary formats elsewhere.
+
+## kospex impact
+
+All tag changes are additions or case-only:
+
+- `eslint` became `ESLint`. kospex matches with `tech_type LIKE '%|tag|%'`,
+  and SQLite `LIKE` is case-insensitive for ASCII, so stored queries still
+  match.
+- `.sqlfluff` kept every tag it had and gained `formatter` and `config`.
+
+No tag was renamed in a way that changes matching, and none was removed. A
+kospex pin bump is needed only to pick up the new tags, which requires a
+re-sync — `last_panopticas_version` handles that automatically.
+
+## Also fixed: `panopticas file` ignored basename mappings
+
+`panopticas file` built its "File type" row from
+`get_extension_filetype(extension)`, which consults only `EXT_FILETYPES` and
+never `LANGUAGE_BY_BASENAME`. Files typed by basename therefore reported
+nothing there while `panopticas assess` reported their type correctly — the two
+commands disagreed about the same file.
+
+This predates the linter work: `panopticas file go.mod` has shown a blank file
+type since basename mapping was introduced, as has `setup.cfg`. Mapping
+`.isort.cfg` and `staticcheck.conf` by basename (correct, because `.cfg` and
+`.conf` hold arbitrary formats elsewhere) would have added two more.
+
+The row now reads `core.get_language(file, skip_shebang=True)`, which checks
+`LANGUAGE_BY_BASENAME` before the extension table. Two details matter:
+
+- `skip_shebang=True` keeps the row distinct from the Shebang Language row.
+  Without it a shebang-only script would report `bash` as its *file* type,
+  conflating two rows that exist precisely to be compared.
+- `get_language()` returns the string `"Unknown"` for unrecognised files,
+  which is normalised to `None` so the JSON contract still emits null.
+
+The row remains a pure table lookup, so the trust comment above it still
+holds: the value is always a `constants.py` value, never raw input from the
+path.
+
+Across the 96 linter config filenames the row went from 55/96 resolved to
+96/96, and `go.mod` and `setup.cfg` are fixed as a side-effect.
+
+## Also fixed: unreadable paths aborted whole scans
+
+`check_shebang()` caught `FileNotFoundError` and `UnicodeDecodeError`, which
+covers a filename that does not exist on disk — the common case, and the
+reason `get_language("no-such-file.py")` has always correctly returned
+`Python`. But `open()` fails in other ways that are not `FileNotFoundError`:
+
+| Path | Raised | Caught before? |
+|---|---|---|
+| missing file | `FileNotFoundError` | yes |
+| a directory | `IsADirectoryError` | **no** |
+| unreadable file | `PermissionError` | **no** |
+| name too long | `OSError` | **no** |
+| NUL byte in path | `ValueError` | **no** |
+
+The handler is now `except (OSError, ValueError)`. `OSError` is the base class
+of every filesystem refusal; `ValueError` covers a NUL byte in the path and
+keeps undecodable content handled, since `UnicodeDecodeError` is a `ValueError`
+subclass.
+
+This mattered well beyond the type returned for one file. `identify_files()`
+walks whole trees, so a single unreadable file aborted an entire
+`panopticas assess` run with a `PermissionError` traceback. `count_lines()` in
+the same module was already robust — it ends with a catch-all — which is what
+made `check_shebang()` the last unguarded `open()` on the scanning path.
+
+`panopticas urls` had the identical bug one layer up: its loop caught
+`UnicodeDecodeError`, added in 0.0.19 so a binary file could not kill the run,
+but not `OSError`. Fixed in the CLI loop rather than in
+`extract_urls_from_file()`, whose raising behaviour is documented and
+deliberate — the decision to keep scanning belongs to the caller walking the
+tree.
+
+`panopticas file` was never exposed: Click rejects an unreadable path at the
+CLI boundary before the library sees it.
+
+## Testing
+
+`tests/test_linter_detection.py` — 141 tests covering table integrity,
+expansion into each `METADATA_RULES` table, representative tags per language,
+the config/ignore split, filetype resolution for every config filename, and
+negative guards asserting each rejected convention stays undetected.
+
+Fixture files for the novel filetype resolutions are in `src/tests/`.

docs/_layouts/default.html

@@ -30,6 +30,7 @@
                         <li><a href="/cli">CLI</a></li>
                         <li><a href="/api">API</a></li>
                         <li><a href="/ai-detection-rules">AI Detection</a></li>
+                        <li><a href="/linter-detection-rules">Linters</a></li>
                         <li><a href="https://github.com/kospex/panopticas">GitHub</a></li>
                         <li><a href="https://pypi.org/project/panopticas/">PyPI</a></li>
                         <div class="social-links">
@@ -69,6 +70,7 @@ <h4>Quick Links</h4>
                             <li><a href="/cli">CLI Reference</a></li>
                             <li><a href="/api">API Reference</a></li>
                             <li><a href="/ai-detection-rules">AI Detection Rules</a></li>
+                            <li><a href="/linter-detection-rules">Linter Detection Rules</a></li>
                             <li><a href="https://github.com/kospex/panopticas">GitHub</a></li>
                             <li><a href="https://pypi.org/project/panopticas/">PyPI</a></li>
                         </ul>