diff --git a/CHANGELOG.md b/CHANGELOG.md index 34d5c49..bbf5536 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,24 @@ The format of this changelog is based on [Keep a Changelog](https://keepachangel ## [Unreleased] +### Added + - Detection of linter, formatter and type checker configuration files and their ignore counterparts: 30 tools across JavaScript, TypeScript, CSS, Python, Go, Ruby, Java, Groovy, C#/.NET and SQL, covering 96 exact filenames. Every convention was verified against the tool's current official documentation; see [`docs/linter-detection-rules.md`](https://panopticas.io/linter-detection-rules) for the inventory, the sources, and the candidates that were investigated and rejected. + - `LINTER_RULES` in `constants.py` is the single table describing those tools, expanded into the existing `METADATA_RULES` tables at import. `LINTER_ROLES` constrains the legal roles. + - Two new role tags, `formatter` and `typechecker`, joining the existing `linter`. A tool may hold more than one — Ruff, Biome, RuboCop, Standard and SQLFluff all lint and format. + - `.mjs` and `.cjs` now report as JavaScript, `.mts` and `.cts` as TypeScript. These were missing from the extension table entirely, so every file using them reported `Unknown` regardless of linters. + - Filetypes for extensionless config dotfiles that previously reported `Unknown`: `.editorconfig`, `.flake8`, `.globalconfig`, `.pylintrc`, `pylintrc`, `.isort.cfg` and `.style.yapf` resolve to `INI`, `.jshintrc` to `JSON`, `staticcheck.conf` to `TOML`, `.rufo` and `Steepfile` to `Ruby`, and `*.ruleset` to `XML`. Files accepting several formats get their own name: `ESLintRC`, `Prettierrc`, `StylelintRC`, plus `ESLintIgnore`, `PrettierIgnore`, `StylelintIgnore`, `JSHintIgnore` and `YapfIgnore`. `JSONC` and `JSON5` were also added. + +### Changed + - The tag vocabulary grew from 85 to 118 tags and the filetype vocabulary from 76 to 86. Both are still derived from the rules, so neither can drift. + - `eslint.config.js` tagged `eslint`; it now tags `ESLint`, matching the brand casing every other tool uses, and gains a `TypeScript` tag. kospex matches tags with a case-insensitive `LIKE`, so stored queries are unaffected. + - `.sqlfluff` and `.sqlfluffignore` moved into `LINTER_RULES`. Purely additive: `.sqlfluff` keeps `SQLFluff`, `SQL` and `linter` and gains `formatter` and `config`. + + All tag changes are additions or case-only, so kospex is compatible either way; a bump is needed only to pick up the new tags, which requires a re-sync. + ### Fixed + - `check_shebang()` caught only `FileNotFoundError` and `UnicodeDecodeError`, so any other way `open()` can fail propagated out of `get_language()`. A single unreadable file aborted a whole `panopticas assess` scan with `PermissionError`, and passing a directory raised `IsADirectoryError`. It now returns `None` for every path it cannot read, letting `get_language()` fall back to the extension tables. Passing a filename that does not exist on disk was already safe and still is. + - `panopticas urls` aborted the whole scan on one unreadable file. The loop caught `UnicodeDecodeError` — added in 0.0.19 so a binary file could not kill the run — but not `OSError`. Both are now treated as a file with no URLs. `extract_urls_from_file()` still raises by design; the handling belongs in the caller that is scanning a tree. + - `panopticas file` reported no file type for any file typed by basename rather than by extension, because it read `get_extension_filetype()`, which consults only `EXT_FILETYPES`. `panopticas file go.mod` showed a blank file type while `panopticas assess` showed `go.mod`; `setup.cfg` was blank the same way. The row now reads `get_language(skip_shebang=True)`, so both commands agree. The shebang rows are unaffected — a shebang-only script still reports no file type rather than borrowing its interpreter. - `panopticas file` labelled its first table row `File extenion`. Cosmetic, and the only change to shipped code since 0.0.19 — no library function, tag or JSON field changes, so kospex is unaffected and its pin does not need bumping for this ## 0.0.19 - 2026-08-15 diff --git a/CLAUDE.md b/CLAUDE.md index 51475d8..d471c6d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -45,6 +45,41 @@ Rules to follow when adding a product: - `exact_filename` and `path_contains` keys are lowercase; path fragments end with `/`. - **Verify the convention against the product's current official docs before adding it.** A wrong rule mislabels a repository and the label flows into kospex. A missing rule is better than a wrong one. Watch for user-level paths (`~/.config//`) — those are not repository artifacts and must not be added. +### Linter, Formatter and Type Checker Detection + +> **Before adding or changing a rule, read [`docs/linter-detection-rules.md`](docs/linter-detection-rules.md)** — the full inventory, the source confirming each convention, and the candidates already investigated and **rejected**. Eleven were rejected as wrong; re-adding one would mislabel repositories. + +`LINTER_RULES` in `constants.py` describes each tool once — its languages, its +roles, and the files that identify it: + +```python +"ESLint": { + "languages": ["JavaScript", "TypeScript"], + "roles": ["linter"], + "config": ["eslint.config.js", ..., ".eslintrc.json"], + "ignore": [".eslintignore"], +}, +``` + +`_expand_linter_rules()` writes those into the existing `METADATA_RULES` +tables at import — `config`/`ignore` into `exact_filename_rules`, +`path_contains` into `path_contains_rules`, `extensions` into +`extension_rules`. Tags are `languages + roles + [tool] + ["config"|"ignore"]`. +`get_tags()` reads the expanded tables, so a new tool joins the vocabulary with +no other change. + +Rules to follow when adding a tool: + +- Detection is **path-based only** — never open a file. +- `roles` must come from `LINTER_ROLES` (`linter`, `formatter`, `typechecker`). A tool may hold several; Ruff, Biome, RuboCop, Standard and SQLFluff all lint and format. +- Tool names use the vendor's own brand casing (`ESLint`, `RuboCop`, `golangci-lint`, `mypy`). This is the one thing the pre-existing rules got inconsistent. +- `languages` names what the tool actually targets, not the ecosystem it ships in — Stylelint is `CSS`, not `JavaScript`. Leave it empty for genuinely neutral tools (`.editorconfig`). +- `config` and `ignore` keys are lowercase; path fragments end with `/`. +- Generated exclusion files are `ignore`, not `config` — `.rubocop_todo.yml` excludes offences. +- **Do not add shared config files** (`pyproject.toml`, `setup.cfg`, `tox.ini`, `package.json`). They really do hold linter config, but which tool is inside is not knowable from the path, so tagging them mislabels nearly every repository that has one. +- **Verify the convention against the tool's current official docs before adding it.** Watch for user-level paths — revive's only auto-discovered config is `$HOME/revive.toml` and was rejected for exactly this reason. A missing rule is better than a wrong one. +- If a tool's config file has no extension, give it a filetype too: map to the documented content format where the tool documents one, otherwise give it its own name. Add every new filetype name to `NON_LANGUAGE_FILETYPES` — a test asserts the classification is complete. + ## Development Commands ### Installation (Development) diff --git a/changes/2026-10-linter-detection.md b/changes/2026-10-linter-detection.md new file mode 100644 index 0000000..f16cc66 --- /dev/null +++ b/changes/2026-10-linter-detection.md @@ -0,0 +1,197 @@ +# Linter, formatter and type checker detection + +**Version:** 0.0.20 +**Date:** 2026-10-05 + +## What changed + +Panopticas now detects the configuration and exclusion files of code-quality +tooling — 30 tools, 96 exact filenames — across JavaScript, TypeScript, CSS, +Python, Go, Ruby, Java, Groovy, C#/.NET and SQL. + +Before this change the only linters detected were ESLint's flat config and +SQLFluff, and the two rules disagreed on tag shape: `eslint.config.js` tagged +`["JavaScript", "linter", "eslint", "config"]` while `.sqlfluff` tagged +`["SQLFluff", "SQL", "linter"]` — lowercase versus brand-cased tool name, with +and without a `config` role. + +## Design + +### One table, not ninety-six literal rules + +`LINTER_RULES` in `constants.py` describes each tool once: + +```python +"ESLint": { + "languages": ["JavaScript", "TypeScript"], + "roles": ["linter"], + "config": ["eslint.config.js", ..., ".eslintrc.json"], + "ignore": [".eslintignore"], +}, +``` + +`_expand_linter_rules()` writes those into the existing `METADATA_RULES` +tables at import: `config` and `ignore` into `exact_filename_rules`, +`path_contains` into `path_contains_rules`, `extensions` into +`extension_rules`. Tags are `languages + roles + [tool] + ["config"|"ignore"]`. + +This was chosen over literal dict entries for three reasons. Prettier alone +accepts 18 config filenames that all carry identical tags, so the literal form +is 18 near-duplicate lines. The tag shape becomes structurally guaranteed +rather than a convention each entry has to remember — which is precisely what +went wrong between the `eslint` and `SQLFluff` rules. And the table reads as +the tool inventory, mirroring how `AI_RULES` already works. + +Nothing downstream changed: `get_tags()` traverses `METADATA_RULES` after +expansion, so the new tags joined the vocabulary with no change to `core.py` +or `cli.py`. The existing vocabulary drift test in `tests/test_vocabulary.py` +picked up all 96 rules for free. + +### Language tags name what a tool targets + +ESLint gets `JavaScript` and `TypeScript`; Stylelint gets `CSS` and +deliberately not `JavaScript`; `.editorconfig` is language-neutral and carries +no language tag at all. Prettier is capped at its three main targets rather +than the seven parsers it ships, to keep one config file from carrying seven +language tags. + +### Three roles + +`linter`, `formatter` and `typechecker`, constrained by `LINTER_ROLES`. A tool +may hold several, and Ruff, Biome, RuboCop, Standard and SQLFluff all lint and +format. `.rubocop_todo.yml` and `.standard_todo.yml` are classified `ignore` +rather than `config` — both are generated to exclude existing offences, which +makes them the exclusion file rather than a second config. + +## Research + +Every convention was verified against the tool's current official +documentation before being added. The full evidence base, the sources, and the +rejected candidates are in `docs/linter-detection-rules.md`. + +Eleven candidates were investigated and **rejected**, each of which would have +been a wrong rule: + +- **revive** auto-discovers only `$HOME/revive.toml` — a user-level path, not + a repository artifact. +- **SpotBugs**, **PMD** and **CodeNarc** take arbitrarily-named filter and + ruleset files passed explicitly on the command line. +- **Black**, **StandardJS** and **Spotless** have no dedicated config file. +- **gofmt**, **gofumpt** and **go vet** have no configuration at all. +- **pycodestyle**'s dedicated file is user-level. +- **dprint** is plausible but its filename list could not be confirmed. +- Security scanners (Sonar, Bandit, Brakeman) are out of scope. + +`pyproject.toml`, `setup.cfg`, `tox.ini` and `package.json` genuinely hold +linter configuration, but which tool is inside is not knowable from the path, +and detection here is path-based only. Tagging them would mislabel nearly +every repository that has one, so they keep only their existing build and +dependency tags. + +## Side-effect: four missing extensions + +The work surfaced a gap unrelated to linters. `.mjs`, `.cjs`, `.mts` and +`.cts` were absent from `EXT_FILETYPES` entirely, so every file using them +reported `Unknown` and `get_languages()` missed them. They are now JavaScript +and TypeScript respectively, matching GitHub Linguist, which types all four as +`programming`. They surfaced here because config authors use them to escape +the `"type"` field in `package.json`, which is why `eslint.config.mjs` and +`.prettierrc.cjs` are common. + +## Filetype resolution + +Extensionless config dotfiles previously reported `Unknown`. They now resolve +on one principle, which reconciles the two precedents already in the codebase +(`.sqlfluff` got its own name; `setup.cfg` was mapped to `INI` "because it is +read by configparser"): **where a tool documents a single content format, map +to that format; where several are accepted, the file gets its own name.** + +`.cfg` and `.conf` are mapped by basename, not extension, because both are +used for arbitrary formats elsewhere. + +## kospex impact + +All tag changes are additions or case-only: + +- `eslint` became `ESLint`. kospex matches with `tech_type LIKE '%|tag|%'`, + and SQLite `LIKE` is case-insensitive for ASCII, so stored queries still + match. +- `.sqlfluff` kept every tag it had and gained `formatter` and `config`. + +No tag was renamed in a way that changes matching, and none was removed. A +kospex pin bump is needed only to pick up the new tags, which requires a +re-sync — `last_panopticas_version` handles that automatically. + +## Also fixed: `panopticas file` ignored basename mappings + +`panopticas file` built its "File type" row from +`get_extension_filetype(extension)`, which consults only `EXT_FILETYPES` and +never `LANGUAGE_BY_BASENAME`. Files typed by basename therefore reported +nothing there while `panopticas assess` reported their type correctly — the two +commands disagreed about the same file. + +This predates the linter work: `panopticas file go.mod` has shown a blank file +type since basename mapping was introduced, as has `setup.cfg`. Mapping +`.isort.cfg` and `staticcheck.conf` by basename (correct, because `.cfg` and +`.conf` hold arbitrary formats elsewhere) would have added two more. + +The row now reads `core.get_language(file, skip_shebang=True)`, which checks +`LANGUAGE_BY_BASENAME` before the extension table. Two details matter: + +- `skip_shebang=True` keeps the row distinct from the Shebang Language row. + Without it a shebang-only script would report `bash` as its *file* type, + conflating two rows that exist precisely to be compared. +- `get_language()` returns the string `"Unknown"` for unrecognised files, + which is normalised to `None` so the JSON contract still emits null. + +The row remains a pure table lookup, so the trust comment above it still +holds: the value is always a `constants.py` value, never raw input from the +path. + +Across the 96 linter config filenames the row went from 55/96 resolved to +96/96, and `go.mod` and `setup.cfg` are fixed as a side-effect. + +## Also fixed: unreadable paths aborted whole scans + +`check_shebang()` caught `FileNotFoundError` and `UnicodeDecodeError`, which +covers a filename that does not exist on disk — the common case, and the +reason `get_language("no-such-file.py")` has always correctly returned +`Python`. But `open()` fails in other ways that are not `FileNotFoundError`: + +| Path | Raised | Caught before? | +|---|---|---| +| missing file | `FileNotFoundError` | yes | +| a directory | `IsADirectoryError` | **no** | +| unreadable file | `PermissionError` | **no** | +| name too long | `OSError` | **no** | +| NUL byte in path | `ValueError` | **no** | + +The handler is now `except (OSError, ValueError)`. `OSError` is the base class +of every filesystem refusal; `ValueError` covers a NUL byte in the path and +keeps undecodable content handled, since `UnicodeDecodeError` is a `ValueError` +subclass. + +This mattered well beyond the type returned for one file. `identify_files()` +walks whole trees, so a single unreadable file aborted an entire +`panopticas assess` run with a `PermissionError` traceback. `count_lines()` in +the same module was already robust — it ends with a catch-all — which is what +made `check_shebang()` the last unguarded `open()` on the scanning path. + +`panopticas urls` had the identical bug one layer up: its loop caught +`UnicodeDecodeError`, added in 0.0.19 so a binary file could not kill the run, +but not `OSError`. Fixed in the CLI loop rather than in +`extract_urls_from_file()`, whose raising behaviour is documented and +deliberate — the decision to keep scanning belongs to the caller walking the +tree. + +`panopticas file` was never exposed: Click rejects an unreadable path at the +CLI boundary before the library sees it. + +## Testing + +`tests/test_linter_detection.py` — 141 tests covering table integrity, +expansion into each `METADATA_RULES` table, representative tags per language, +the config/ignore split, filetype resolution for every config filename, and +negative guards asserting each rejected convention stays undetected. + +Fixture files for the novel filetype resolutions are in `src/tests/`. diff --git a/docs/_layouts/default.html b/docs/_layouts/default.html index b2bfcb0..55bfc04 100644 --- a/docs/_layouts/default.html +++ b/docs/_layouts/default.html @@ -30,6 +30,7 @@
  • CLI
  • API
  • AI Detection
    • +
  • Linters
  • GitHub
  • PyPI
    • @@ -69,6 +70,7 @@

    Quick Links

  • CLI Reference
  • API Reference
  • AI Detection Rules
    • +
  • Linter Detection Rules
  • GitHub
  • PyPI
    • diff --git a/docs/linter-detection-rules.md b/docs/linter-detection-rules.md new file mode 100644 index 0000000..2794efa --- /dev/null +++ b/docs/linter-detection-rules.md @@ -0,0 +1,256 @@ +--- +layout: default +title: Linter Detection Rules +description: Every linter, formatter and type checker config file panopticas detects, the source confirming each convention, and the candidates that were investigated and rejected. +--- + +# Linter, Formatter and Type Checker Detection — Rules Reference + +**Rules verified:** 2026-10-05 (panopticas 0.0.20) +**Inventory below generated from:** `LINTER_RULES` in `src/panopticas/constants.py` + +Panopticas detects the configuration and exclusion files of code-quality +tooling and tags each with the languages the tool targets, its role, the tool +name, and whether the file configures or excludes. + +``` +.eslintrc.json -> ['JavaScript', 'TypeScript', 'linter', 'ESLint', 'config'] +.eslintignore -> ['JavaScript', 'TypeScript', 'linter', 'ESLint', 'ignore'] +.rubocop.yml -> ['Ruby', 'linter', 'formatter', 'RuboCop', 'config'] +mypy.ini -> ['Python', 'typechecker', 'mypy', 'config'] +.editorconfig -> ['formatter', 'EditorConfig', 'config'] +``` + +This document is the **reference and evidence base**: what is detected, the +source that confirms each convention, and — equally important — what was +rejected and why. + +## How matching works + +`LINTER_RULES` maps a tool to its languages, its roles, and the files that +identify it. A single expansion step writes those entries into the same +`METADATA_RULES` tables every other rule uses, so the tags are produced by one +code path and the shape cannot drift between tools. + +| Key | Expands into | Matched against | +|---|---|---| +| `config` | `exact_filename_rules` | the lowercased basename | +| `ignore` | `exact_filename_rules` | the lowercased basename | +| `path_contains` | `path_contains_rules` | a substring of the path | +| `extensions` | `extension_rules` | the file extension | + +Tags are `languages + roles + [tool] + ["config"` or `"ignore"]`. The three +roles are `linter`, `formatter` and `typechecker`; a tool may hold more than +one, and several do — Ruff, Biome, RuboCop, Standard and SQLFluff all lint and +format. + +Unlike AI artifact detection, these rules **add** tags rather than resolving to +a single winner, so `eslint.config.js` is both JavaScript and an ESLint config. + +## Detection is path-based only + +No rule opens a file. This is the same constraint AI detection works under, and +it is what makes the per-language shared config files undetectable: + +| File | Really can hold | Why it is not tagged | +|---|---|---| +| `pyproject.toml` | Ruff, Black, mypy, isort, yapf, Pylint, Pyright | which tool is inside is not knowable from the path | +| `setup.cfg` | Flake8, isort, yapf, mypy | same | +| `tox.ini` | Flake8, Pylint | same | +| `package.json` | Prettier, ESLint, Stylelint, StandardJS | same | + +Tagging any of these as a linter config would mislabel every repository that +has one, which is most of them. They keep their existing build and dependency +tags only. + +## The rejected list + +Each of these was investigated against the tool's current documentation and +**deliberately not added**. Re-adding one would mislabel repositories. + +| Candidate | Why rejected | +|---|---| +| **revive** (`revive.toml`) | Auto-discovers only `$HOME/revive.toml`. A user-level path, not a repository artifact — the same trap AI detection warns about. A project-level config exists but must be passed with `-config`, under any name. | +| **SpotBugs** (`spotbugs-exclude.xml`) | Filter files are arbitrarily named and passed with `-exclude`/`-include`. The documentation's `myExcludeFilter.xml` is illustrative, not a convention. | +| **PMD** (`ruleset.xml`) | Ruleset files are arbitrarily named and passed explicitly. No discovered filename. | +| **CodeNarc** (`codenarc.xml`) | Rulesets are arbitrary `.xml` or `.groovy` files. Detected indirectly through npm-groovy-lint, which does have a fixed config name. | +| **Black** | Configured only in `pyproject.toml`. No dedicated file exists to match. | +| **StandardJS** | Configured only in `package.json`. | +| **Spotless** | Configured only in `build.gradle` / `pom.xml`. | +| **gofmt**, **gofumpt**, **go vet** | No configuration file at all. | +| **pycodestyle** | Project-level config lives in `setup.cfg` or `tox.ini`; the dedicated file is the user-level `~/.config/pycodestyle`. | +| **dprint** | Plausible (`dprint.json`), but the filename list could not be confirmed from the official docs. Left out pending verification — a missing rule is better than a wrong one. | +| **Sonar**, **Bandit**, **Brakeman**, **GitLeaks** | Security scanners, out of scope. GitLeaks is already detected separately via `.gitleaksignore`. | + +### Two judgement calls worth knowing + +**`checkstyle.xml`** is not a documented Checkstyle convention — Checkstyle +discovers nothing and takes its config as an argument. It is included because +the name is unambiguous in practice, and alongside it sit two names that *are* +documented: `google_checks.xml` and `sun_checks.xml` ship with the +distribution, and `config/checkstyle/` is the Gradle Checkstyle plugin's +documented default directory. + +**`tsconfig.json`** is a *compiler* configuration, and the TypeScript handbook +is explicit that it does not handle linting. It is tagged `typechecker` because +it is nonetheless where a project's type checking is actually configured +(`strict`, `noImplicitAny`), which is the thing worth finding. `jsconfig.json` +is the same file applied to a JavaScript project. + +## Filetypes, not just tags + +Extensionless config dotfiles used to report `Unknown`. They now resolve on one +principle: **where a tool documents a single content format, map to that +format; where several are accepted, the file gets its own name.** + +| Resolves to its documented format | Gets its own name | +|---|---| +| `.editorconfig`, `.flake8`, `.globalconfig`, `.pylintrc`, `pylintrc`, `.isort.cfg`, `.style.yapf` → `INI` | `.eslintrc` → `ESLintRC` (JSON, YAML or JS) | +| `.jshintrc` → `JSON` | `.prettierrc` → `Prettierrc` (JSON or YAML) | +| `staticcheck.conf` → `TOML` | `.stylelintrc` → `StylelintRC` (JSON or YAML) | +| `.rufo`, `Steepfile` → `Ruby` (both are evaluated as Ruby) | `.eslintignore`, `.prettierignore`, `.stylelintignore`, `.jshintignore`, `.yapfignore` — gitignore-style pattern files | +| `*.ruleset` → `XML` | | + +`.cfg` and `.conf` are mapped by basename rather than by extension, because +both are used for arbitrary formats elsewhere — the same reasoning that already +applied to `setup.cfg`. + +### Module extensions + +Closing this gap also meant adding four extensions that were missing from +`EXT_FILETYPES` altogether, and which are not linter-specific at all: + +| Extension | Language | Meaning | +|---|---|---| +| `.mjs` | JavaScript | always an ES module | +| `.cjs` | JavaScript | always CommonJS | +| `.mts` | TypeScript | TypeScript emitting an ES module | +| `.cts` | TypeScript | TypeScript emitting CommonJS | + +These exist because Node needs to know a file's module system before parsing +it, and `.js` alone does not say. Config authors reach for them to escape the +`"type"` field in `package.json`, which is why `eslint.config.mjs` and +`.prettierrc.cjs` are so common. Both pairs match GitHub Linguist, which types +all four as `programming`. + +## Sources + +Every convention above was checked against the tool's own current +documentation: + +| Tool | Source | +|---|---| +| ESLint | [Configuration files](https://eslint.org/docs/latest/use/configure/configuration-files) (flat), [v8 configuration files](https://eslint.org/docs/v8.x/use/configure/configuration-files) (eslintrc) | +| Prettier | [Configuration file](https://prettier.io/docs/configuration) | +| Stylelint | [Configure](https://stylelint.io/user-guide/configure/) | +| Biome | [Configure Biome](https://biomejs.dev/guides/configure-biome/) | +| oxlint | [Linter config](https://oxc.rs/docs/guide/usage/linter/config.html) | +| TypeScript | [tsconfig.json](https://www.typescriptlang.org/docs/handbook/tsconfig-json.html) | +| EditorConfig | [editorconfig.org](https://editorconfig.org/) | +| Ruff | [Configuration](https://docs.astral.sh/ruff/configuration/) | +| Flake8 | [Configuring flake8](https://flake8.pycqa.org/en/latest/user/configuration.html) | +| Pylint | [Running pylint](https://pylint.readthedocs.io/en/stable/user_guide/usage/run.html) | +| mypy | [The mypy configuration file](https://mypy.readthedocs.io/en/stable/config_file.html) | +| Pyright | [Configuration](https://github.com/microsoft/pyright/blob/main/docs/configuration.md) | +| isort | [Config files](https://pycqa.github.io/isort/docs/configuration/config_files.html) | +| yapf | [README](https://github.com/google/yapf) | +| Prospector | [Profiles](https://prospector.landscape.io/en/master/profiles.html) | +| golangci-lint | [Configuration file](https://golangci-lint.run/docs/configuration/file/) | +| staticcheck | [Configuration](https://staticcheck.dev/docs/configuration/) | +| RuboCop | [Configuration](https://docs.rubocop.org/rubocop/configuration.html) | +| Standard | [README](https://github.com/standardrb/standard) | +| Reek | [README](https://github.com/troessner/reek) | +| Rufo | [README](https://github.com/ruby-formatter/rufo) | +| erb_lint | [README](https://github.com/Shopify/erb_lint) | +| Steep | [README](https://github.com/soutaro/steep) | +| Sorbet | [CLI](https://sorbet.org/docs/cli) | +| Checkstyle | [Gradle Checkstyle plugin](https://docs.gradle.org/current/userguide/checkstyle_plugin.html) | +| npm-groovy-lint | [Documentation](https://nvuillam.github.io/npm-groovy-lint/) | +| Roslyn | [Configuration files for code analysis rules](https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/configuration-files) | +| StyleCop | [Configuration](https://github.com/DotNetAnalyzers/StyleCopAnalyzers/blob/master/documentation/Configuration.md) | + +## Re-verifying + +Config filenames change. ESLint replaced `.eslintrc` with flat config and +dropped `.eslintignore`; Stylelint's docs warn its legacy locations "may be +removed in the future". Legacy names are kept deliberately — a repository that +still has `.eslintrc.json` is still a repository using ESLint. + +When re-verifying, check each source above for new accepted extensions, and +re-read the rejected list before adding anything in it. + +## Inventory + +`*` marks a path fragment rather than a filename. + +### JavaScript, TypeScript and CSS + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **ESLint** | linter | `JavaScript`, `TypeScript` | `eslint.config.js`, `eslint.config.mjs`, `eslint.config.cjs`, `eslint.config.ts`, `eslint.config.mts`, `eslint.config.cts`, `.eslintrc`, `.eslintrc.js`, `.eslintrc.cjs`, `.eslintrc.yaml`, `.eslintrc.yml`, `.eslintrc.json` | `.eslintignore` | +| **Prettier** | formatter | `JavaScript`, `TypeScript`, `CSS` | `.prettierrc`, `.prettierrc.json`, `.prettierrc.yml`, `.prettierrc.yaml`, `.prettierrc.json5`, `.prettierrc.js`, `.prettierrc.mjs`, `.prettierrc.cjs`, `.prettierrc.ts`, `.prettierrc.mts`, `.prettierrc.cts`, `.prettierrc.toml`, `prettier.config.js`, `prettier.config.mjs`, `prettier.config.cjs`, `prettier.config.ts`, `prettier.config.mts`, `prettier.config.cts` | `.prettierignore` | +| **Stylelint** | linter | `CSS` | `stylelint.config.js`, `stylelint.config.mjs`, `stylelint.config.cjs`, `stylelint.config.ts`, `.stylelintrc`, `.stylelintrc.js`, `.stylelintrc.mjs`, `.stylelintrc.cjs`, `.stylelintrc.yml`, `.stylelintrc.yaml`, `.stylelintrc.json` | `.stylelintignore` | +| **Biome** | linter, formatter | `JavaScript`, `TypeScript`, `CSS` | `biome.json`, `biome.jsonc`, `.biome.json`, `.biome.jsonc` | — | +| **oxlint** | linter | `JavaScript`, `TypeScript` | `.oxlintrc.json`, `.oxlintrc.jsonc`, `oxlint.config.ts`, `oxlint.config.mts` | — | +| **JSHint** | linter | `JavaScript` | `.jshintrc` | `.jshintignore` | +| **TypeScript** | typechecker | `TypeScript`, `JavaScript` | `tsconfig.json`, `jsconfig.json` | — | +| **EditorConfig** | formatter | *(none)* | `.editorconfig` | — | + +### Python + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **Ruff** | linter, formatter | `Python` | `ruff.toml`, `.ruff.toml` | — | +| **Flake8** | linter | `Python` | `.flake8` | — | +| **Pylint** | linter | `Python` | `pylintrc`, `.pylintrc`, `pylintrc.toml`, `.pylintrc.toml` | — | +| **mypy** | typechecker | `Python` | `mypy.ini`, `.mypy.ini` | — | +| **Pyright** | typechecker | `Python` | `pyrightconfig.json` | — | +| **isort** | formatter | `Python` | `.isort.cfg` | — | +| **yapf** | formatter | `Python` | `.style.yapf` | `.yapfignore` | +| **Prospector** | linter | `Python` | `.prospector.yaml`, `.prospector.yml` | — | + +### Go + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **golangci-lint** | linter | `Go` | `.golangci.yml`, `.golangci.yaml`, `.golangci.toml`, `.golangci.json` | — | +| **staticcheck** | linter | `Go` | `staticcheck.conf` | — | + +### Ruby + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **RuboCop** | linter, formatter | `Ruby` | `.rubocop.yml` | `.rubocop_todo.yml` | +| **Standard** | linter, formatter | `Ruby` | `.standard.yml` | `.standard_todo.yml` | +| **Reek** | linter | `Ruby` | `.reek.yml` | — | +| **Rufo** | formatter | `Ruby` | `.rufo` | — | +| **erb_lint** | linter | `Ruby` | `.erb_lint.yml` | — | +| **Steep** | typechecker | `Ruby` | `steepfile` | — | +| **Sorbet** | typechecker | `Ruby` | `sorbet/`* | — | + +### Java + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **Checkstyle** | linter | `Java` | `checkstyle.xml`, `google_checks.xml`, `sun_checks.xml`, `config/checkstyle/`* | — | + +### Groovy + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **npm-groovy-lint** | linter | `Groovy` | `.groovylintrc.json`, `.groovylintrc.js`, `.groovylintrc.yml` | — | + +### C# and .NET + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **Roslyn** | linter | `C#`, `.NET` | `.globalconfig`, `*.ruleset` | — | +| **StyleCop** | linter | `C#`, `.NET` | `stylecop.json`, `.stylecop.json` | — | + +### SQL + +| Tool | Role | Language tags | Config | Ignore | +|---|---|---|---|---| +| **SQLFluff** | linter, formatter | `SQL` | `.sqlfluff` | `.sqlfluffignore` | + + diff --git a/pyproject.toml b/pyproject.toml index 097d4a0..e7d41c4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "panopticas" -version = "0.0.19" +version = "0.0.20" description = "The panopticas file type and CLI tool." authors = [{ name = "Peter Freiberg", email = "peter.freiberg@gmail.com" }] dependencies = [ diff --git a/src/panopticas/cli.py b/src/panopticas/cli.py index 80e7a00..c688c62 100644 --- a/src/panopticas/cli.py +++ b/src/panopticas/cli.py @@ -294,10 +294,20 @@ def identify(file, as_json): """Assess a filetype.""" extension = core.get_fileext(file) shebang = core.check_shebang(file) + + # get_language() consults LANGUAGE_BY_BASENAME before the extension table, + # so files typed by basename (go.mod, setup.cfg, staticcheck.conf) report + # the same type here as they do in `assess`. Reading the extension table + # alone left those null. skip_shebang keeps this row distinct from the + # shebang rows below, and the UNKNOWN sentinel becomes null. + filetype = core.get_language(file, skip_shebang=True) + if filetype == core.UNKNOWN: + filetype = None + payload = { "file": file, "extension": extension, - "filetype": core.get_extension_filetype(extension), + "filetype": filetype, "shebang": shebang, "shebang_language": ( core.extract_shebang_language(shebang) if shebang else None), @@ -344,15 +354,17 @@ def find_urls(directory, all_files, as_json): # process's cwd — join with `directory` to read the file, but keep the # relative path in the record so JSON/table output matches prior output. # - # This command scans a whole directory, so one undecodable file (a - # binary such as a .png) must not abort the run for every other file. - # extract_urls_from_file() raises UnicodeDecodeError for those; treat - # them the same as a file with no URLs rather than propagating. + # This command scans a whole directory, so one file that cannot be read + # must not abort the run for every other file. extract_urls_from_file() + # raises by design — UnicodeDecodeError for a binary such as a .png, and + # OSError for anything the filesystem refuses (unreadable, a dangling + # symlink, a file deleted mid-scan). Treat all of them as a file with no + # URLs rather than propagating. records = [] for f in files: try: urls = core.extract_urls_from_file(os.path.join(directory, f)) - except UnicodeDecodeError: + except (OSError, UnicodeDecodeError): urls = [] records.append({"path": f, "urls": urls}) diff --git a/src/panopticas/constants.py b/src/panopticas/constants.py index 7de4705..3f96eb7 100644 --- a/src/panopticas/constants.py +++ b/src/panopticas/constants.py @@ -23,6 +23,8 @@ ".class": "Java Class", ".cpp": "C++", ".cs": "C#", + ".cjs": "JavaScript", # CommonJS module, explicit + ".cts": "TypeScript", # TypeScript emitting CommonJS ".csproj": "C# Project", ".css": "CSS", ".csv": "CSV", @@ -53,12 +55,16 @@ ".jpeg": "JPEG", ".js": "JavaScript", ".json": "JSON", + ".json5": "JSON5", + ".jsonc": "JSONC", # JSON with comments ".jsx": "JSX", ".kt": "Kotlin", ".lock": "Lock", ".m": "Objective-C", ".mailmap": "Mailmap", ".md": "Markdown", + ".mjs": "JavaScript", # ES module, explicit + ".mts": "TypeScript", # TypeScript emitting an ES module ".nvmrc": "nvmrc", ".pdf": "PDF", ".php": "PHP", @@ -96,11 +102,32 @@ ".yaml": "YAML", ".yml": "YAML", ".zip": "ZIP", + # Linter, formatter and type checker config files. Where the tool + # documents a single content format the extension maps to that format; + # where several are accepted the file gets its own name. See LINTER_RULES. + ".editorconfig": "INI", + ".eslintignore": "ESLintIgnore", + ".eslintrc": "ESLintRC", # JSON, YAML or JS + ".flake8": "INI", + ".globalconfig": "INI", + ".jshintignore": "JSHintIgnore", + ".jshintrc": "JSON", + ".prettierignore": "PrettierIgnore", + ".prettierrc": "Prettierrc", # JSON or YAML + ".pylintrc": "INI", + ".ruleset": "XML", + ".rufo": "Ruby", # evaluated as Ruby by the formatter + ".stylelintignore": "StylelintIgnore", + ".stylelintrc": "StylelintRC", # JSON or YAML + ".yapf": "INI", # .style.yapf + ".yapfignore": "YapfIgnore", # Special cases for files without extensions or .format files "codeowners": "CODEOWNERS", "dockerfile": "Dockerfile", "license": "Text", "makefile": "Makefile", + "pylintrc": "INI", + "steepfile": "Ruby", "cname": "CNAME", # Often GitHub et al will use a CNAME file for a URL to host from } @@ -111,6 +138,10 @@ # than adding ".cfg" to EXT_FILETYPES — that extension is used for arbitrary # formats elsewhere, so a blanket ".cfg" -> INI rule would over-claim. "setup.cfg": "INI", + # ".cfg" and ".conf" are used for arbitrary formats elsewhere, so these + # are mapped by basename for the same reason setup.cfg is. + ".isort.cfg": "INI", + "staticcheck.conf": "TOML", } # Classification of every value in EXT_FILETYPES and LANGUAGE_BY_BASENAME as @@ -183,6 +214,8 @@ "CSV", "INI", "JSON", + "JSON5", + "JSONC", "Properties", "TOML", "TSV", @@ -199,6 +232,14 @@ "GitAttributes", "Gitignore", "GitLeaksIgnore", + "ESLintIgnore", + "ESLintRC", + "JSHintIgnore", + "PrettierIgnore", + "Prettierrc", + "StylelintIgnore", + "StylelintRC", + "YapfIgnore", "Lock", "Mailmap", "nvmrc", @@ -250,7 +291,6 @@ "web.config": [".NET", "ASP.NET", "config"], "app.config": [".NET", "config"], "codeowners": ["Git"], - "eslint.config.js": ["JavaScript", "linter", "eslint", "config"], "pyproject.toml": ["build", "dependencies", "Python"], # setup.py / setup.cfg are setuptools-specific, so they carry the # backend tag. pyproject.toml does not: its build-backend is declared @@ -268,7 +308,6 @@ "jenkinsfile.groovy": ["pipeline", "Jenkins"], ".mailmap": ["Git"], ".python-version": ["Python", "dependencies"], - ".sqlfluff": ["SQLFluff", "SQL", "linter"], ".nvmrc": ["Node", "dependencies"], ".gitignore": ["Git", "ignore"], "dockerfile": ["IaC", "Docker", "dependencies"], @@ -276,7 +315,6 @@ "makefile": ["build"], "go.mod": ["Go", "module", "dependencies"], "go.sum": ["Go", "dependencies", "checksum"], - ".sqlfluffignore": ["SQLFluff", "ignore"], "codefresh.yml": ["pipeline", "Codefresh"], ".travis.yml": ["pipeline", "TravisCI"], "package.json": ["npm", "dependencies"], @@ -301,6 +339,241 @@ ], } +# The roles a code-quality tool can play. A LINTER_RULES entry may not use a +# role outside this set. +LINTER_ROLES = frozenset({ + "linter", # reports violations + "formatter", # rewrites code to a canonical form + "typechecker", # checks static types +}) + +LINTER_RULES = { + # --- JavaScript / TypeScript --------------------------------------- + "ESLint": { + "languages": ["JavaScript", "TypeScript"], + "roles": ["linter"], + "config": [ + # Flat config, the current format. + "eslint.config.js", "eslint.config.mjs", "eslint.config.cjs", + "eslint.config.ts", "eslint.config.mts", "eslint.config.cts", + # eslintrc, removed in v9 but still present in many repositories. + ".eslintrc", ".eslintrc.js", ".eslintrc.cjs", ".eslintrc.yaml", + ".eslintrc.yml", ".eslintrc.json", + ], + "ignore": [".eslintignore"], + }, + "Prettier": { + # Prettier formats far more than these three, but tagging every + # parser it ships would put seven languages on one config file. + "languages": ["JavaScript", "TypeScript", "CSS"], + "roles": ["formatter"], + "config": [ + ".prettierrc", ".prettierrc.json", ".prettierrc.yml", + ".prettierrc.yaml", ".prettierrc.json5", ".prettierrc.js", + ".prettierrc.mjs", ".prettierrc.cjs", ".prettierrc.ts", + ".prettierrc.mts", ".prettierrc.cts", ".prettierrc.toml", + "prettier.config.js", "prettier.config.mjs", + "prettier.config.cjs", "prettier.config.ts", + "prettier.config.mts", "prettier.config.cts", + ], + "ignore": [".prettierignore"], + }, + "Stylelint": { + "languages": ["CSS"], + "roles": ["linter"], + "config": [ + "stylelint.config.js", "stylelint.config.mjs", + "stylelint.config.cjs", "stylelint.config.ts", + ".stylelintrc", ".stylelintrc.js", ".stylelintrc.mjs", + ".stylelintrc.cjs", ".stylelintrc.yml", ".stylelintrc.yaml", + ".stylelintrc.json", + ], + "ignore": [".stylelintignore"], + }, + "Biome": { + "languages": ["JavaScript", "TypeScript", "CSS"], + "roles": ["linter", "formatter"], + "config": ["biome.json", "biome.jsonc", ".biome.json", ".biome.jsonc"], + }, + "oxlint": { + "languages": ["JavaScript", "TypeScript"], + "roles": ["linter"], + "config": [".oxlintrc.json", ".oxlintrc.jsonc", "oxlint.config.ts", + "oxlint.config.mts"], + }, + "JSHint": { + "languages": ["JavaScript"], + "roles": ["linter"], + "config": [".jshintrc"], + "ignore": [".jshintignore"], + }, + # tsc is a compiler, but tsconfig.json is where a project's type checking + # is configured (strict, noImplicitAny). jsconfig.json is the same file + # applied to a JavaScript project. + "TypeScript": { + "languages": ["TypeScript", "JavaScript"], + "roles": ["typechecker"], + "config": ["tsconfig.json", "jsconfig.json"], + }, + # Language-neutral by design, so it carries no language tag. + "EditorConfig": { + "languages": [], + "roles": ["formatter"], + "config": [".editorconfig"], + }, + # --- Python --------------------------------------------------------- + "Ruff": { + "languages": ["Python"], + "roles": ["linter", "formatter"], + "config": ["ruff.toml", ".ruff.toml"], + }, + "Flake8": { + "languages": ["Python"], + "roles": ["linter"], + "config": [".flake8"], + }, + "Pylint": { + "languages": ["Python"], + "roles": ["linter"], + "config": ["pylintrc", ".pylintrc", "pylintrc.toml", ".pylintrc.toml"], + }, + "mypy": { + "languages": ["Python"], + "roles": ["typechecker"], + "config": ["mypy.ini", ".mypy.ini"], + }, + "Pyright": { + "languages": ["Python"], + "roles": ["typechecker"], + "config": ["pyrightconfig.json"], + }, + "isort": { + "languages": ["Python"], + "roles": ["formatter"], + "config": [".isort.cfg"], + }, + "yapf": { + "languages": ["Python"], + "roles": ["formatter"], + "config": [".style.yapf"], + "ignore": [".yapfignore"], + }, + "Prospector": { + "languages": ["Python"], + "roles": ["linter"], + "config": [".prospector.yaml", ".prospector.yml"], + }, + # --- Go ------------------------------------------------------------- + "golangci-lint": { + "languages": ["Go"], + "roles": ["linter"], + "config": [".golangci.yml", ".golangci.yaml", ".golangci.toml", + ".golangci.json"], + }, + "staticcheck": { + "languages": ["Go"], + "roles": ["linter"], + "config": ["staticcheck.conf"], + }, + # --- Ruby ----------------------------------------------------------- + "RuboCop": { + "languages": ["Ruby"], + "roles": ["linter", "formatter"], + "config": [".rubocop.yml"], + # Written by --auto-gen-config to exclude existing offences, so it is + # the exclusion file rather than a second config. + "ignore": [".rubocop_todo.yml"], + }, + "Standard": { + "languages": ["Ruby"], + "roles": ["linter", "formatter"], + "config": [".standard.yml"], + "ignore": [".standard_todo.yml"], + }, + "Reek": { + "languages": ["Ruby"], + "roles": ["linter"], + "config": [".reek.yml"], + }, + "Rufo": { + "languages": ["Ruby"], + "roles": ["formatter"], + "config": [".rufo"], + }, + "erb_lint": { + "languages": ["Ruby"], + "roles": ["linter"], + "config": [".erb_lint.yml"], + }, + "Steep": { + "languages": ["Ruby"], + "roles": ["typechecker"], + "config": ["steepfile"], + }, + "Sorbet": { + "languages": ["Ruby"], + "roles": ["typechecker"], + # sorbet/config plus the generated sorbet/rbi tree. + "path_contains": ["sorbet/"], + }, + # --- Java ----------------------------------------------------------- + # Checkstyle itself documents no fixed filename; google_checks.xml and + # sun_checks.xml ship with the distribution and config/checkstyle/ is the + # Gradle plugin's documented default. PMD and SpotBugs are deliberately + # absent — their ruleset and filter files are arbitrarily named. + "Checkstyle": { + "languages": ["Java"], + "roles": ["linter"], + "config": ["checkstyle.xml", "google_checks.xml", "sun_checks.xml"], + "path_contains": ["config/checkstyle/"], + }, + # --- Groovy --------------------------------------------------------- + "npm-groovy-lint": { + "languages": ["Groovy"], + "roles": ["linter"], + "config": [".groovylintrc.json", ".groovylintrc.js", + ".groovylintrc.yml"], + }, + # --- C# / .NET ------------------------------------------------------ + "Roslyn": { + "languages": ["C#", ".NET"], + "roles": ["linter"], + "config": [".globalconfig"], + # Legacy Code Analysis rule sets, deprecated for .globalconfig but + # still common in older solutions. + "extensions": [".ruleset"], + }, + "StyleCop": { + "languages": ["C#", ".NET"], + "roles": ["linter"], + "config": ["stylecop.json", ".stylecop.json"], + }, + # --- SQL ------------------------------------------------------------ + "SQLFluff": { + "languages": ["SQL"], + "roles": ["linter", "formatter"], + "config": [".sqlfluff"], + "ignore": [".sqlfluffignore"], + }, +} + + +def _expand_linter_rules(): + """Expand LINTER_RULES into the METADATA_RULES tables.""" + for tool, spec in LINTER_RULES.items(): + base = list(spec.get("languages", [])) + list(spec["roles"]) + [tool] + for filename in spec.get("config", ()): + METADATA_RULES["exact_filename_rules"][filename] = base + ["config"] + for filename in spec.get("ignore", ()): + METADATA_RULES["exact_filename_rules"][filename] = base + ["ignore"] + for fragment in spec.get("path_contains", ()): + METADATA_RULES["path_contains_rules"][fragment] = base + ["config"] + for ext in spec.get("extensions", ()): + METADATA_RULES["extension_rules"][ext] = base + ["config"] + + +_expand_linter_rules() + # The complete set of legal `kind` values for an AI artifact. # A rule may not use a kind outside this set. AI_ARTIFACT_KINDS = { diff --git a/src/panopticas/core.py b/src/panopticas/core.py index 44083f8..d183101 100644 --- a/src/panopticas/core.py +++ b/src/panopticas/core.py @@ -177,7 +177,16 @@ def get_languages(): return sorted(LANGUAGE_FILETYPES, key=str.lower) def check_shebang(file_path): - """ Check if a file has a shebang """ + """ + Return a file's shebang line, or None if it has none or cannot be read. + + Callers may pass a path that is not a readable file — a name being + classified without touching disk, a directory, something unreadable — so + this never propagates an exception. A shebang is one signal among several + in get_language(), which falls back to the extension tables, and + identify_files() walks whole trees where one bad file must not abort the + scan. + """ try: with open(file_path) as file: @@ -187,12 +196,13 @@ def check_shebang(file_path): else: return None - except FileNotFoundError: - # TODO - better logging instead of print - #print(f"File {file_path} not found") - return None - except UnicodeDecodeError: - # TODO - log this exception + # OSError covers every filesystem refusal: missing (FileNotFoundError), + # a directory (IsADirectoryError), unreadable (PermissionError), a name + # the OS rejects as too long. ValueError covers undecodable content + # (UnicodeDecodeError is a subclass) and a NUL byte in the path, which + # open() rejects before it reaches the filesystem. + # TODO - log the exception rather than discarding it silently + except (OSError, ValueError): return None diff --git a/src/tests/.editorconfig b/src/tests/.editorconfig new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.eslintignore b/src/tests/.eslintignore new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.eslintrc b/src/tests/.eslintrc new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.flake8 b/src/tests/.flake8 new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.globalconfig b/src/tests/.globalconfig new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.golangci.yml b/src/tests/.golangci.yml new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.groovylintrc.json b/src/tests/.groovylintrc.json new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.isort.cfg b/src/tests/.isort.cfg new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.jshintignore b/src/tests/.jshintignore new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.prettierignore b/src/tests/.prettierignore new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.prettierrc b/src/tests/.prettierrc new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.prettierrc.json5 b/src/tests/.prettierrc.json5 new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.pylintrc b/src/tests/.pylintrc new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.rubocop.yml b/src/tests/.rubocop.yml new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.rufo b/src/tests/.rufo new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.style.yapf b/src/tests/.style.yapf new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.stylelintignore b/src/tests/.stylelintignore new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.stylelintrc b/src/tests/.stylelintrc new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/.yapfignore b/src/tests/.yapfignore new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/CodeAnalysis.ruleset b/src/tests/CodeAnalysis.ruleset new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/Steepfile b/src/tests/Steepfile new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/biome.jsonc b/src/tests/biome.jsonc new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/module.cjs b/src/tests/module.cjs new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/module.cts b/src/tests/module.cts new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/module.mjs b/src/tests/module.mjs new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/module.mts b/src/tests/module.mts new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/pylintrc b/src/tests/pylintrc new file mode 100644 index 0000000..e69de29 diff --git a/src/tests/staticcheck.conf b/src/tests/staticcheck.conf new file mode 100644 index 0000000..e69de29 diff --git a/tests/test_cli_json.py b/tests/test_cli_json.py index d881511..e21eb74 100644 --- a/tests/test_cli_json.py +++ b/tests/test_cli_json.py @@ -3,6 +3,8 @@ import json import os +import pytest + from click.testing import CliRunner from panopticas.cli import cli @@ -142,6 +144,58 @@ def test_documents_every_detection_method(self, tmp_path): assert payload["meta"] == [] assert payload["urls"] == ["https://example.com"] + def test_filetype_honours_basename_mappings(self, tmp_path): + # The File type row used to read get_extension_filetype(), which only + # consults EXT_FILETYPES. Files typed by basename — go.mod, setup.cfg, + # .isort.cfg, staticcheck.conf — reported null here while `assess` + # reported their type, because assess calls get_language(). + for name, expected in ( + ("go.mod", "go.mod"), + ("setup.cfg", "INI"), + (".isort.cfg", "INI"), + ("staticcheck.conf", "TOML"), + ): + target = tmp_path / name + target.write_text("") + payload = json.loads( + CliRunner().invoke( + cli, ["file", str(target), "--json"]).stdout) + assert payload["filetype"] == expected, name + + def test_filetype_agrees_with_assess(self, tmp_path): + for name in ("go.mod", ".isort.cfg", "staticcheck.conf", "module.mjs"): + (tmp_path / name).write_text("") + + one = json.loads( + CliRunner().invoke( + cli, ["assess", str(tmp_path), "--json"]).stdout) + by_path = {r["path"]: r["language"] for r in one["files"]} + + for name in by_path: + payload = json.loads( + CliRunner().invoke( + cli, ["file", str(tmp_path / name), "--json"]).stdout) + assert payload["filetype"] == by_path[name], name + + def test_unrecognised_filetype_is_null_not_the_sentinel(self, tmp_path): + # get_language() returns the string "Unknown" for unrecognised files. + # The JSON contract says null. + target = tmp_path / "mystery.xyz123" + target.write_text("nothing recognisable\n") + payload = json.loads( + CliRunner().invoke(cli, ["file", str(target), "--json"]).stdout) + assert payload["filetype"] is None + + def test_filetype_does_not_absorb_the_shebang_language(self, tmp_path): + # File type and Shebang Language are separate rows. A shebang-only + # script must leave File type null rather than reporting Shell. + target = tmp_path / "runme" + target.write_text("#!/bin/bash\necho hi\n") + payload = json.loads( + CliRunner().invoke(cli, ["file", str(target), "--json"]).stdout) + assert payload["filetype"] is None + assert payload["shebang_language"] == "bash" + def test_absent_shebang_is_null(self, tmp_path): target = tmp_path / "notes.md" target.write_text("hello\n") @@ -168,6 +222,28 @@ def test_documents_urls_per_file(self, tmp_path): assert by_path["README.md"] == ["https://example.com"] assert by_path["empty.md"] == [] + @pytest.mark.skipif( + hasattr(os, "geteuid") and os.geteuid() == 0, + reason="root can read a mode-000 file") + def test_unreadable_file_is_skipped_not_fatal(self, tmp_path): + # Same reasoning as the undecodable case below: scanning a whole + # directory must survive one file it cannot open. The handler caught + # UnicodeDecodeError but not PermissionError, so a single unreadable + # file aborted the command for every other file. + (tmp_path / "README.md").write_text("see https://example.com\n") + locked = tmp_path / "locked.md" + locked.write_text("https://hidden.example\n") + locked.chmod(0o000) + try: + result = CliRunner().invoke(cli, ["urls", str(tmp_path), "--json"]) + assert result.exit_code == 0, result.output + payload = json.loads(result.stdout) + by_path = {r["path"]: r["urls"] for r in payload["files"]} + assert by_path["README.md"] == ["https://example.com"] + assert by_path["locked.md"] == [] + finally: + locked.chmod(0o644) + def test_undecodable_file_is_skipped_not_fatal(self, tmp_path): # extract_urls_from_file() raises UnicodeDecodeError on binary # content. `urls` scans a whole directory, so one bad file (e.g. a diff --git a/tests/test_linter_detection.py b/tests/test_linter_detection.py new file mode 100644 index 0000000..b065aee --- /dev/null +++ b/tests/test_linter_detection.py @@ -0,0 +1,432 @@ +""" +Tests for linter, formatter and type checker config detection. + +Covers: LINTER_RULES table integrity, its expansion into METADATA_RULES, the +tags produced for each language's tooling, the filetype resolution of +extensionless config dotfiles, and negative guards for the conventions that +were deliberately rejected. +""" + +import pytest + +from panopticas import get_filename_metatypes, get_language, get_tags +from panopticas.constants import ( + LINTER_ROLES, LINTER_RULES, METADATA_RULES) + + +def tags_for(path): + """Tags as a set — rule order is not part of the contract.""" + return set(get_filename_metatypes(path)) + + +class TestLinterRulesTableIntegrity: + """The table itself must be well formed.""" + + def test_every_role_is_a_known_role(self): + for tool, spec in LINTER_RULES.items(): + for role in spec["roles"]: + assert role in LINTER_ROLES, ( + f"{tool} uses role {role!r}, which is not in LINTER_ROLES") + + def test_every_tool_declares_at_least_one_role(self): + for tool, spec in LINTER_RULES.items(): + assert spec["roles"], f"{tool} declares no role" + + def test_every_tool_declares_at_least_one_indicator(self): + for tool, spec in LINTER_RULES.items(): + indicators = (spec.get("config", []) + spec.get("ignore", []) + + spec.get("path_contains", []) + + spec.get("extensions", [])) + assert indicators, f"{tool} declares no files to match" + + def test_filenames_are_lowercase(self): + # get_filename_metatypes() lowercases the basename before matching, so + # an uppercase key could never match. + for tool, spec in LINTER_RULES.items(): + for filename in spec.get("config", []) + spec.get("ignore", []): + assert filename == filename.lower(), ( + f"{tool} key {filename!r} must be lowercase") + + def test_path_fragments_end_with_a_slash(self): + for tool, spec in LINTER_RULES.items(): + for fragment in spec.get("path_contains", []): + assert fragment.endswith("/"), ( + f"{tool} fragment {fragment!r} must end with /") + + def test_no_filename_is_claimed_by_two_tools(self): + seen = {} + for tool, spec in LINTER_RULES.items(): + for filename in spec.get("config", []) + spec.get("ignore", []): + assert filename not in seen, ( + f"{filename!r} claimed by both {seen.get(filename)} " + f"and {tool}") + seen[filename] = tool + + def test_no_tool_is_both_config_and_ignore_for_one_file(self): + for tool, spec in LINTER_RULES.items(): + overlap = set(spec.get("config", [])) & set(spec.get("ignore", [])) + assert not overlap, f"{tool}: {sorted(overlap)} is both" + + +class TestExpansionIntoMetadataRules: + """LINTER_RULES is expanded into the tables get_filename_metatypes reads.""" + + def test_config_filenames_reach_exact_filename_rules(self): + exact = METADATA_RULES["exact_filename_rules"] + for tool, spec in LINTER_RULES.items(): + for filename in spec.get("config", []): + assert filename in exact, f"{tool}: {filename} not expanded" + + def test_path_fragments_reach_path_contains_rules(self): + contains = METADATA_RULES["path_contains_rules"] + for tool, spec in LINTER_RULES.items(): + for fragment in spec.get("path_contains", []): + assert fragment in contains, f"{tool}: {fragment} not expanded" + + def test_extensions_reach_extension_rules(self): + ext_rules = METADATA_RULES["extension_rules"] + for tool, spec in LINTER_RULES.items(): + for ext in spec.get("extensions", []): + assert ext in ext_rules, f"{tool}: {ext} not expanded" + + def test_every_expanded_tag_is_in_the_vocabulary(self): + vocabulary = set(get_tags()) + for tool, spec in LINTER_RULES.items(): + for filename in spec.get("config", []): + produced = tags_for(filename) + assert produced <= vocabulary, ( + f"{filename} produced {sorted(produced - vocabulary)}") + + +class TestRoleTags: + """Every rule carries its role, its tool and config-or-ignore.""" + + def test_config_file_is_tagged_config(self): + assert "config" in tags_for(".eslintrc.json") + + def test_ignore_file_is_tagged_ignore(self): + assert "ignore" in tags_for(".eslintignore") + + def test_ignore_file_is_not_tagged_config(self): + assert "config" not in tags_for(".eslintignore") + + def test_formatter_is_distinct_from_linter(self): + prettier = tags_for(".prettierrc") + assert "formatter" in prettier + assert "linter" not in prettier + + def test_typechecker_role(self): + assert "typechecker" in tags_for("mypy.ini") + + def test_a_tool_can_be_both_linter_and_formatter(self): + ruff = tags_for("ruff.toml") + assert {"linter", "formatter"} <= ruff + + +class TestJavaScriptAndTypeScript: + + def test_eslint_flat_config(self): + assert {"JavaScript", "TypeScript", "linter", "ESLint", "config"} \ + <= tags_for("eslint.config.js") + + @pytest.mark.parametrize("filename", [ + "eslint.config.mjs", "eslint.config.cjs", "eslint.config.ts", + "eslint.config.mts", "eslint.config.cts", + ]) + def test_eslint_flat_config_extensions(self, filename): + assert "ESLint" in tags_for(filename) + + @pytest.mark.parametrize("filename", [ + ".eslintrc", ".eslintrc.js", ".eslintrc.cjs", ".eslintrc.yaml", + ".eslintrc.yml", ".eslintrc.json", + ]) + def test_eslint_legacy_config(self, filename): + assert "ESLint" in tags_for(filename) + + def test_eslint_is_recased_from_the_old_lowercase_tag(self): + # The pre-existing rule tagged `eslint`. Re-cased for consistency with + # every other tool; kospex matches tags with a case-insensitive LIKE, + # so this does not break stored queries. + assert "eslint" not in get_tags() + assert "ESLint" in get_tags() + + @pytest.mark.parametrize("filename", [ + ".prettierrc", ".prettierrc.json", ".prettierrc.yml", + ".prettierrc.yaml", ".prettierrc.json5", ".prettierrc.js", + ".prettierrc.mjs", ".prettierrc.cjs", ".prettierrc.ts", + ".prettierrc.mts", ".prettierrc.cts", ".prettierrc.toml", + "prettier.config.js", "prettier.config.mjs", "prettier.config.cjs", + "prettier.config.ts", "prettier.config.mts", "prettier.config.cts", + ]) + def test_prettier_config(self, filename): + assert {"formatter", "Prettier", "config"} <= tags_for(filename) + + def test_prettier_ignore(self): + assert {"formatter", "Prettier", "ignore"} <= tags_for(".prettierignore") + + def test_stylelint_is_tagged_css_not_javascript(self): + tags = tags_for(".stylelintrc.json") + assert {"CSS", "linter", "Stylelint"} <= tags + assert "JavaScript" not in tags + + def test_stylelint_ignore(self): + assert {"Stylelint", "ignore"} <= tags_for(".stylelintignore") + + @pytest.mark.parametrize("filename", [ + "biome.json", "biome.jsonc", ".biome.json", ".biome.jsonc", + ]) + def test_biome_is_linter_and_formatter(self, filename): + assert {"linter", "formatter", "Biome"} <= tags_for(filename) + + @pytest.mark.parametrize("filename", [ + ".oxlintrc.json", ".oxlintrc.jsonc", "oxlint.config.ts", + "oxlint.config.mts", + ]) + def test_oxlint_config(self, filename): + assert {"linter", "oxlint"} <= tags_for(filename) + + def test_jshint_config_and_ignore(self): + assert {"JavaScript", "linter", "JSHint", "config"} \ + <= tags_for(".jshintrc") + assert {"JSHint", "ignore"} <= tags_for(".jshintignore") + + def test_tsconfig_is_a_typechecker_config(self): + assert {"TypeScript", "typechecker", "config"} <= tags_for("tsconfig.json") + + def test_jsconfig_is_the_javascript_equivalent(self): + assert {"typechecker", "config"} <= tags_for("jsconfig.json") + + def test_editorconfig_is_language_neutral(self): + tags = tags_for(".editorconfig") + assert {"formatter", "EditorConfig", "config"} <= tags + for language in ("JavaScript", "Python", "C#", "Java"): + assert language not in tags + + +class TestPython: + + @pytest.mark.parametrize("filename,tool", [ + ("ruff.toml", "Ruff"), + (".ruff.toml", "Ruff"), + (".flake8", "Flake8"), + ("pylintrc", "Pylint"), + (".pylintrc", "Pylint"), + ("pylintrc.toml", "Pylint"), + (".pylintrc.toml", "Pylint"), + ("mypy.ini", "mypy"), + (".mypy.ini", "mypy"), + ("pyrightconfig.json", "Pyright"), + (".isort.cfg", "isort"), + (".style.yapf", "yapf"), + (".prospector.yaml", "Prospector"), + (".prospector.yml", "Prospector"), + ]) + def test_python_tool_config(self, filename, tool): + assert {"Python", tool, "config"} <= tags_for(filename) + + def test_yapf_ignore(self): + assert {"yapf", "ignore"} <= tags_for(".yapfignore") + + +class TestGo: + + @pytest.mark.parametrize("filename", [ + ".golangci.yml", ".golangci.yaml", ".golangci.toml", ".golangci.json", + ]) + def test_golangci_lint_config(self, filename): + assert {"Go", "linter", "golangci-lint", "config"} <= tags_for(filename) + + def test_staticcheck_config(self): + assert {"Go", "linter", "staticcheck", "config"} \ + <= tags_for("staticcheck.conf") + + +class TestRuby: + + def test_rubocop_is_linter_and_formatter(self): + assert {"Ruby", "linter", "formatter", "RuboCop", "config"} \ + <= tags_for(".rubocop.yml") + + def test_rubocop_todo_is_an_exclusion_file(self): + # --auto-gen-config writes it to exclude existing offences, so it is + # RuboCop's ignore counterpart rather than another config file. + assert {"RuboCop", "ignore"} <= tags_for(".rubocop_todo.yml") + + def test_standard_config_and_todo(self): + assert {"Ruby", "Standard", "config"} <= tags_for(".standard.yml") + assert {"Standard", "ignore"} <= tags_for(".standard_todo.yml") + + def test_reek_config(self): + assert {"Ruby", "linter", "Reek", "config"} <= tags_for(".reek.yml") + + def test_rufo_is_a_formatter(self): + assert {"Ruby", "formatter", "Rufo", "config"} <= tags_for(".rufo") + + def test_erb_lint_config(self): + assert {"Ruby", "linter", "erb_lint", "config"} \ + <= tags_for(".erb_lint.yml") + + def test_steepfile_is_a_typechecker_config(self): + assert {"Ruby", "typechecker", "Steep", "config"} <= tags_for("Steepfile") + + def test_sorbet_directory_is_matched_by_path(self): + assert {"Ruby", "typechecker", "Sorbet"} <= tags_for("sorbet/config") + + +class TestJava: + + @pytest.mark.parametrize("filename", [ + "checkstyle.xml", "google_checks.xml", "sun_checks.xml", + ]) + def test_checkstyle_config(self, filename): + assert {"Java", "linter", "Checkstyle", "config"} <= tags_for(filename) + + def test_checkstyle_gradle_convention_path(self): + # The Gradle Checkstyle plugin documents config/checkstyle/ as the + # default location. + assert {"Java", "linter", "Checkstyle"} \ + <= tags_for("config/checkstyle/rules.xml") + + +class TestGroovy: + + @pytest.mark.parametrize("filename", [ + ".groovylintrc.json", ".groovylintrc.js", ".groovylintrc.yml", + ]) + def test_npm_groovy_lint_config(self, filename): + assert {"Groovy", "linter", "npm-groovy-lint", "config"} \ + <= tags_for(filename) + + +class TestDotNet: + + def test_globalconfig_is_a_roslyn_analyzer_config(self): + assert {"C#", ".NET", "linter", "Roslyn", "config"} \ + <= tags_for(".globalconfig") + + def test_ruleset_extension_is_matched(self): + assert {"linter", "Roslyn"} <= tags_for("CodeAnalysis.ruleset") + + @pytest.mark.parametrize("filename", ["stylecop.json", ".stylecop.json"]) + def test_stylecop_config(self, filename): + assert {"C#", "linter", "StyleCop", "config"} <= tags_for(filename) + + +class TestSqlFluffNormalisation: + """The pre-existing SQLFluff rules move into LINTER_RULES.""" + + def test_sqlfluff_keeps_its_original_tags(self): + # Purely additive: every tag the old literal rule produced is retained. + assert {"SQLFluff", "SQL", "linter"} <= tags_for(".sqlfluff") + + def test_sqlfluff_gains_the_config_role(self): + assert "config" in tags_for(".sqlfluff") + + def test_sqlfluffignore_keeps_its_original_tags(self): + assert {"SQLFluff", "ignore"} <= tags_for(".sqlfluffignore") + + +class TestModuleExtensionFiletypes: + """.mjs/.cjs/.mts/.cts were missing from EXT_FILETYPES entirely.""" + + @pytest.mark.parametrize("filename,expected", [ + ("index.mjs", "JavaScript"), + ("index.cjs", "JavaScript"), + ("index.mts", "TypeScript"), + ("index.cts", "TypeScript"), + ]) + def test_module_extensions_resolve_to_their_language(self, filename, expected): + assert get_language(filename, skip_shebang=True) == expected + + def test_they_are_classified_as_languages(self): + from panopticas import get_languages + + assert "JavaScript" in get_languages() + assert "TypeScript" in get_languages() + + +class TestConfigFiletypeResolution: + """Extensionless config files resolve to a format where one is documented.""" + + @pytest.mark.parametrize("filename,expected", [ + # Documented single format -> that format. + (".editorconfig", "INI"), + (".flake8", "INI"), + (".globalconfig", "INI"), + (".pylintrc", "INI"), + ("pylintrc", "INI"), + (".isort.cfg", "INI"), + (".style.yapf", "INI"), + (".jshintrc", "JSON"), + ("staticcheck.conf", "TOML"), + (".rufo", "Ruby"), + ("Steepfile", "Ruby"), + ("CodeAnalysis.ruleset", "XML"), + # Several accepted formats -> its own name. + (".eslintrc", "ESLintRC"), + (".prettierrc", "Prettierrc"), + (".stylelintrc", "StylelintRC"), + # Pattern files. + (".eslintignore", "ESLintIgnore"), + (".prettierignore", "PrettierIgnore"), + (".stylelintignore", "StylelintIgnore"), + (".jshintignore", "JSHintIgnore"), + (".yapfignore", "YapfIgnore"), + # Data formats new to the table. + ("biome.jsonc", "JSONC"), + (".prettierrc.json5", "JSON5"), + ]) + def test_filetype(self, filename, expected): + assert get_language(filename, skip_shebang=True) == expected + + def test_no_linter_config_resolves_to_unknown(self): + unresolved = [] + for spec in LINTER_RULES.values(): + for filename in spec.get("config", []) + spec.get("ignore", []): + if get_language(filename, skip_shebang=True) == "Unknown": + unresolved.append(filename) + assert not unresolved, ( + f"no filetype for {sorted(unresolved)} — add an EXT_FILETYPES or " + "LANGUAGE_BY_BASENAME entry") + + +class TestRejectedConventions: + """ + Guards for conventions investigated and deliberately rejected. Each of + these would be a wrong rule, and a wrong rule mislabels repositories. + """ + + def test_revive_toml_is_not_detected(self): + # revive auto-discovers only $HOME/revive.toml, a user-level path, not + # a repository artifact. + assert "revive" not in tags_for("revive.toml") + + def test_spotbugs_filter_files_are_not_detected(self): + # SpotBugs filter files are arbitrarily named and passed with + # -exclude/-include; there is no convention to match. + assert "SpotBugs" not in tags_for("spotbugs-exclude.xml") + + def test_pmd_rulesets_are_not_detected(self): + assert "PMD" not in tags_for("ruleset.xml") + + def test_codenarc_rulesets_are_not_detected(self): + assert "CodeNarc" not in tags_for("codenarc.xml") + + def test_setup_cfg_is_not_tagged_as_a_linter_config(self): + # setup.cfg really can hold flake8, isort and yapf config, but which + # tool is inside cannot be known from the path. + tags = tags_for("setup.cfg") + assert "linter" not in tags + assert "Flake8" not in tags + + def test_pyproject_toml_is_not_tagged_as_a_linter_config(self): + tags = tags_for("pyproject.toml") + assert "linter" not in tags + assert "Ruff" not in tags + + def test_tox_ini_is_not_tagged_as_a_linter_config(self): + assert "linter" not in tags_for("tox.ini") + + def test_black_has_no_dedicated_config_file(self): + # Black is configured only in pyproject.toml, so it is undetectable. + assert "Black" not in get_tags() diff --git a/tests/test_panopticas.py b/tests/test_panopticas.py index 6d08227..414522a 100644 --- a/tests/test_panopticas.py +++ b/tests/test_panopticas.py @@ -271,6 +271,89 @@ def test_file_without_shebang(self): os.unlink(f.name) +class TestCheckShebangUnopenablePaths: + """ + check_shebang() must never propagate an exception from open(). + + A shebang is one signal among several in get_language(), and + identify_files() walks whole trees — so a path that cannot be opened has + to fall through to extension-based detection rather than abort the + caller. It caught FileNotFoundError and UnicodeDecodeError but not the + other ways open() fails, so a single unreadable file killed a whole scan. + """ + + needs_non_root = pytest.mark.skipif( + hasattr(os, "geteuid") and os.geteuid() == 0, + reason="root can read a mode-000 file") + + def test_a_directory_returns_none(self, tmp_path): + # open() on a directory raises IsADirectoryError, an OSError that is + # not a FileNotFoundError. + assert check_shebang(str(tmp_path)) is None + + @needs_non_root + def test_an_unreadable_file_returns_none(self, tmp_path): + target = tmp_path / "locked" + target.write_text("#!/bin/bash\n") + target.chmod(0o000) + try: + assert check_shebang(str(target)) is None + finally: + target.chmod(0o644) + + def test_a_path_the_os_rejects_returns_none(self, tmp_path): + # A NUL byte in a path raises ValueError, not OSError. + assert check_shebang("bad\x00name") is None + + def test_an_absurdly_long_name_returns_none(self, tmp_path): + assert check_shebang(str(tmp_path / ("x" * 5000))) is None + + +class TestGetLanguageUnopenablePaths: + """get_language() classifies by path even when the file cannot be read.""" + + needs_non_root = TestCheckShebangUnopenablePaths.needs_non_root + + def test_a_directory_does_not_raise(self, tmp_path): + assert get_language(str(tmp_path)) == "Unknown" + + @needs_non_root + def test_an_unreadable_file_falls_back_to_the_extension(self, tmp_path): + target = tmp_path / "script.py" + target.write_text("x = 1\n") + target.chmod(0o000) + try: + # The extension still identifies it; only the shebang is lost. + assert get_language(str(target)) == "Python" + finally: + target.chmod(0o644) + + @needs_non_root + def test_an_unreadable_unrecognised_file_is_unknown(self, tmp_path): + target = tmp_path / "locked" + target.write_text("#!/bin/bash\n") + target.chmod(0o000) + try: + assert get_language(str(target)) == "Unknown" + finally: + target.chmod(0o644) + + @needs_non_root + def test_one_unreadable_file_does_not_abort_a_scan(self, tmp_path): + # The real-world failure: identify_files() walks a tree and a single + # unreadable file aborted the whole scan with PermissionError. + (tmp_path / "readable.py").write_text("x = 1\n") + locked = tmp_path / "locked" + locked.write_text("#!/bin/bash\n") + locked.chmod(0o000) + try: + results = identify_files(str(tmp_path)) + assert results["readable.py"] == "Python" + assert "locked" in results + finally: + locked.chmod(0o644) + + class TestGetFilenameMetatypes: """Tests for get_filename_metatypes() — metadata tag extraction."""