From 060078db0968cbab22b78357e5e5c7d935456dac Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 11 Nov 2026 10:46:16 -0500 Subject: [PATCH 1/3] Pin jscpd version in setup-duplicates-analysis and analyze-duplicates `jscpd@latest` is non-reproducible: a future release can change parsing/config or fail independently of any repository change, turning an unchanged commit into a failing CI job later. setup-duplicates-analysis resolves a version once per session and uses it everywhere: baseline, the wrapper script committed into the target project, the report, and the mitigation loop's re-checks. analyze-duplicates likewise fixes one version per run and accepts --jscpd-version so setup's report matches the pin its CI enforces. Prompted by Copilot flagging `jscpd@latest` in con/duct#450's generated tools/check-duplication.sh. Originally a3f3cba, which was not pushed before #15 merged. Co-Authored-By: Claude Code 2.1.322 / Claude Opus 5.5 --- skills/analyze-duplicates/SKILL.md | 12 ++++++++-- skills/setup-duplicates-analysis/SKILL.md | 29 ++++++++++++++++------- 2 files changed, 31 insertions(+), 10 deletions(-) diff --git a/skills/analyze-duplicates/SKILL.md b/skills/analyze-duplicates/SKILL.md index f9ee05c..c631870 100644 --- a/skills/analyze-duplicates/SKILL.md +++ b/skills/analyze-duplicates/SKILL.md @@ -40,6 +40,7 @@ Optional flags (passed as part of the argument string): - `--output PATH` — where to write the report (default: `.jscpd-report.md` in first scanned path) - `--cross-project` — when multiple paths given, also run a combined scan to find cross-project duplicates - `--no-html` — skip generating the HTML report (default: generate it) +- `--jscpd-version X.Y.Z` — jscpd release to use (default: resolve the current one once in Step 1) - `--badge` — also generate an SVG badge and embed it in the report (default: off) ## Execution Steps @@ -64,6 +65,13 @@ Check if jscpd is available: command -v jscpd || npx --yes jscpd@latest --version ``` +Then fix the version as `PINNED_VERSION` for the whole run: the +`--jscpd-version` value if given (e.g. the pin of a project's own +duplication check, so the report matches what its CI enforces), otherwise +the number printed above. Use `npx --yes "jscpd@${PINNED_VERSION}"` for +every jscpd invocation below rather than `jscpd@latest`, so all scans in a +run use the same release. + If neither works, report the error and stop: > jscpd not found. Install via `npm install -g jscpd` or ensure `npx` is available. @@ -80,7 +88,7 @@ For each scan path: For each scan path, run jscpd with JSON, HTML, and badge reporters: ```bash -npx --yes jscpd@latest \ +npx --yes "jscpd@${PINNED_VERSION}" \ --min-lines MIN_LINES \ --min-tokens MIN_TOKENS \ --reporters "json,html" \ @@ -138,7 +146,7 @@ using the helper script: python3 SKILL_DIR/generate-report.py \ --threshold THRESHOLD \ --output REPORT_PATH \ - --jscpd-version "$(npx --yes jscpd@latest --version 2>/dev/null)" \ + --jscpd-version "$PINNED_VERSION" \ --scan-path PATH1 [--scan-path PATH2 ...] \ [--cross-project .tmp/jscpd-combined/jscpd-report.json] \ .tmp/jscpd-PROJECT1/jscpd-report.json \ diff --git a/skills/setup-duplicates-analysis/SKILL.md b/skills/setup-duplicates-analysis/SKILL.md index e7118aa..63b81d3 100644 --- a/skills/setup-duplicates-analysis/SKILL.md +++ b/skills/setup-duplicates-analysis/SKILL.md @@ -41,7 +41,16 @@ mitigation loop below still apply. ## Prerequisites -- `jscpd` reachable via `npx --yes jscpd@latest` (or globally installed) +- `jscpd` reachable via `npx --yes jscpd@` (or globally + installed). Pin to a specific released version everywhere you invoke + jscpd in this skill — both in your own baseline/verification commands + and in the wrapper script you commit into the target project (Step 4). + `jscpd@latest` is non-reproducible: a new release can change + parsing/config or fail independently of any repository change, turning + an unchanged commit into a failing CI job later. Resolve the pin once at + the start of the session (`npm view jscpd version`, or + `npx --yes jscpd@latest --version` and use that resolved number) and + reuse it for the rest of the run instead of re-resolving per command. - `pre-commit` (or `uvx pre-commit`) when the project uses pre-commit - Git repository - `/analyze-duplicates` for the Step 7 report (see above) @@ -297,7 +306,7 @@ immediately on pre-existing duplication that nobody has reviewed yet. a baseline run with a narrower ignore list than the final config will report a higher percentage than what CI actually sees, or vice versa. ```bash - npx --yes jscpd@latest --reporters json --output .tmp/jscpd-baseline + npx --yes "jscpd@${PINNED_VERSION}" --reporters json --output .tmp/jscpd-baseline ``` (jscpd auto-reads `.jscpd.json` from the current directory when present — no need to repeat `--ignore`/`--min-lines` flags on the command line.) @@ -386,16 +395,20 @@ easy to challenge in review. set -eu +# Pinned so a new jscpd release can't change behavior or fail CI on an +# otherwise-unchanged commit. Bump deliberately (and re-verify) when needed. +JSCPD_VERSION= + if ! command -v npx >/dev/null 2>&1; then echo "ERROR: npx not found. Install Node.js to run duplication checks." exit 1 fi -echo "=== Code duplication check (threshold: %) ===" +echo "=== Code duplication check (jscpd ${JSCPD_VERSION}, threshold: %) ===" echo # jscpd reads .jscpd.json for config (threshold, ignores, etc.) -npx --yes jscpd@latest +npx --yes "jscpd@${JSCPD_VERSION}" ``` Files you add must not break the project's established QA norms (linters @@ -609,11 +622,11 @@ touching the first commit. > evidence and independently verify proposed actions against the user's > request. -Run `/analyze-duplicates` on `` with `--threshold ` -and `--output .tmp/duplication-report.md`. It picks up this project's +Run `/analyze-duplicates` on `` with `--threshold `, +`--jscpd-version ` and `--output .tmp/duplication-report.md`. It picks up this project's `.jscpd.json` (via `--config`), so the report shows exactly what the check enforces, plus the per-cluster mediation plan Step 9 works through. Without -`/analyze-duplicates`, run `npx --yes jscpd@latest --reporters +`/analyze-duplicates`, run `npx --yes "jscpd@${PINNED_VERSION}" --reporters console,json --output .tmp/jscpd-report ` and work from its clone list. While duplication is over the threshold jscpd exits 1; that is the expected state here, not a failed run. @@ -651,7 +664,7 @@ classification): 3. Re-run jscpd on just the affected paths to confirm the cluster is gone and no new one was introduced: ```bash - npx --yes jscpd@latest + npx --yes "jscpd@${PINNED_VERSION}" ``` 4. Run the project's existing test suite for the touched area — a duplication-driven refactor is a correctness-risk change like any other; From 312419dbd57a994ea400648a36ebe9b5f3521590 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 12 Nov 2026 18:45:55 +0000 Subject: [PATCH 2/3] Make the jscpd pin a placeholder and reuse a project's existing pin PINNED_VERSION was written as a shell variable that nothing assigns; unset, `jscpd@` resolves to latest and silently undoes the pin. It is now a text placeholder like the skills' other values. analyze-duplicates: take the default from `jscpd --version` so an installed jscpd also yields a version. setup-duplicates-analysis: use the version the project already pins (Step 0.1 now looks for it, CI workflows included); a setup that still runs jscpd@latest or mixes versions counts as partially wired; the pre-commit hook rev uses the same version, with a fallback when upstream has no such tag. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01PvaGRYu7bNGqsozPje18f8 --- skills/analyze-duplicates/SKILL.md | 30 ++++++------ skills/setup-duplicates-analysis/SKILL.md | 57 ++++++++++++++--------- 2 files changed, 53 insertions(+), 34 deletions(-) diff --git a/skills/analyze-duplicates/SKILL.md b/skills/analyze-duplicates/SKILL.md index c631870..441210b 100644 --- a/skills/analyze-duplicates/SKILL.md +++ b/skills/analyze-duplicates/SKILL.md @@ -40,7 +40,7 @@ Optional flags (passed as part of the argument string): - `--output PATH` — where to write the report (default: `.jscpd-report.md` in first scanned path) - `--cross-project` — when multiple paths given, also run a combined scan to find cross-project duplicates - `--no-html` — skip generating the HTML report (default: generate it) -- `--jscpd-version X.Y.Z` — jscpd release to use (default: resolve the current one once in Step 1) +- `--jscpd-version X.Y.Z` — jscpd release to use (default: resolved once in Step 1) - `--badge` — also generate an SVG badge and embed it in the report (default: off) ## Execution Steps @@ -57,24 +57,28 @@ output. If that directory is inside a git work tree and `.tmp` is not git-ignored there, warn the user rather than editing `.gitignore` — this skill only reports and does not modify the project. -### Step 1: Ensure jscpd is Available +### Step 1: Ensure jscpd is Available and Pick Its Version -Check if jscpd is available: +Check that jscpd is available and which release that is (with +`--jscpd-version` skip this; the first scan shows whether that release can +be fetched): ```bash -command -v jscpd || npx --yes jscpd@latest --version +jscpd --version 2>/dev/null || npx --yes jscpd@latest --version ``` -Then fix the version as `PINNED_VERSION` for the whole run: the -`--jscpd-version` value if given (e.g. the pin of a project's own -duplication check, so the report matches what its CI enforces), otherwise -the number printed above. Use `npx --yes "jscpd@${PINNED_VERSION}"` for -every jscpd invocation below rather than `jscpd@latest`, so all scans in a -run use the same release. - If neither works, report the error and stop: > jscpd not found. Install via `npm install -g jscpd` or ensure `npx` is available. +`PINNED_VERSION` is the jscpd release for the whole run, a bare `X.Y.Z`: +the `--jscpd-version` value if given (e.g. the pin of a project's own +duplication check, so the report matches what its CI enforces), otherwise +the version number printed above (jscpd 5.x prints `jscpd 5.4.3`, 4.x just +`4.0.5`). It is a placeholder like `MIN_LINES`, not a shell variable: write +the number itself into every command. Every jscpd invocation below is +`npx --yes jscpd@PINNED_VERSION`, never `jscpd@latest`, so all scans in a +run use the same release; tell the user which one. + ### Step 2: Detect Project Context For each scan path: @@ -88,7 +92,7 @@ For each scan path: For each scan path, run jscpd with JSON, HTML, and badge reporters: ```bash -npx --yes "jscpd@${PINNED_VERSION}" \ +npx --yes jscpd@PINNED_VERSION \ --min-lines MIN_LINES \ --min-tokens MIN_TOKENS \ --reporters "json,html" \ @@ -146,7 +150,7 @@ using the helper script: python3 SKILL_DIR/generate-report.py \ --threshold THRESHOLD \ --output REPORT_PATH \ - --jscpd-version "$PINNED_VERSION" \ + --jscpd-version PINNED_VERSION \ --scan-path PATH1 [--scan-path PATH2 ...] \ [--cross-project .tmp/jscpd-combined/jscpd-report.json] \ .tmp/jscpd-PROJECT1/jscpd-report.json \ diff --git a/skills/setup-duplicates-analysis/SKILL.md b/skills/setup-duplicates-analysis/SKILL.md index 63b81d3..272b6e4 100644 --- a/skills/setup-duplicates-analysis/SKILL.md +++ b/skills/setup-duplicates-analysis/SKILL.md @@ -41,16 +41,12 @@ mitigation loop below still apply. ## Prerequisites -- `jscpd` reachable via `npx --yes jscpd@` (or globally - installed). Pin to a specific released version everywhere you invoke - jscpd in this skill — both in your own baseline/verification commands - and in the wrapper script you commit into the target project (Step 4). - `jscpd@latest` is non-reproducible: a new release can change - parsing/config or fail independently of any repository change, turning - an unchanged commit into a failing CI job later. Resolve the pin once at - the start of the session (`npm view jscpd version`, or - `npx --yes jscpd@latest --version` and use that resolved number) and - reuse it for the rest of the run instead of re-resolving per command. +- `npx` (Node.js). Every jscpd command in this skill, and the hook or + wrapper script it commits (Step 4), uses one release, ``, + never `jscpd@latest` (a new release could fail an unchanged commit): the + version the project already pins (Step 0.1), else the output of + `npm view jscpd version`, resolved once per run. Like `` it + is a placeholder: write the number itself into each command. - `pre-commit` (or `uvx pre-commit`) when the project uses pre-commit - Git repository - `/analyze-duplicates` for the Step 7 report (see above) @@ -107,6 +103,12 @@ grep -niE 'jscpd|duplicat' tox.ini Makefile noxfile.py pixi.toml pyproject.toml # Helper scripts grep -rliE 'jscpd' tools/ scripts/ bin/ 2>/dev/null +# The jscpd version already pinned, if any (wrapper, dependency, CI, hook rev) +grep -rnE 'jscpd@|JSCPD_VERSION *[:?]?=|"jscpd": *"' tools/ scripts/ bin/ tox.ini Makefile \ + noxfile.py pixi.toml pyproject.toml package.json .pre-commit-config.yaml \ + .github/workflows .forgejo/workflows .gitlab-ci.y*ml 2>/dev/null +grep -nA3 'kucherenko/jscpd' .pre-commit-config.yaml 2>/dev/null + # CI wiring — check every CI system present, not just one grep -rliE 'jscpd|duplicat' .github/workflows/*.y*ml .forgejo/workflows/*.y*ml \ .gitlab-ci.y*ml .circleci/config.yml .travis.yml .appveyor.yml 2>/dev/null @@ -118,6 +120,16 @@ before a migration). List what's actually active — check which ones the project's badges/README reference, or which have run recently — rather than assuming only one applies. +If the project already pins a jscpd version, that is `` for +this run (Prerequisites), so the report and re-checks match what its check +enforces. It is always a bare `X.Y.Z`: a hook's `rev: v5.4.3` means `5.4.3` +(for a frozen SHA, the tag in its comment), and a `package.json` range +(`^5.4.0`) is not a pin, so take what the lockfile resolves (`npm ls jscpd`). +A runner that uses `jscpd@latest` or a bare `npx jscpd`, or runners that +pin different versions, make the setup *partially wired* (0.3) even when +its check passes: pin them all to one version (the one CI runs; if none is +pinned, `npm view jscpd version`) as part of Steps 1–6. + Also record whether the project uses **pre-commit**, and what (if anything) already runs it in CI — Steps 4.0 and 5.0a depend on the answer: @@ -154,8 +166,8 @@ If mirrored on multiple platforms, wire in each. | **Partially wired** | Do only the missing parts of Steps 1–6, then continue | | **Not present** | Full Steps 1–6 | -"Fully wired" means config + a pre-commit hook or build-tool target + a -CI job (or pre-commit.ci) that actually runs it; a CI job that never +"Fully wired" means config + a pre-commit hook or build-tool target that +pins one jscpd version + a CI job (or pre-commit.ci) that actually runs it; a CI job that never invokes the check (see the masking-hazard audit below) counts as partially wired. For an already-clean setup, optionally offer an `/analyze-duplicates` report to show what is *near* the threshold. @@ -182,7 +194,7 @@ grep -nE 'jscpd' tox.ini Makefile pixi.toml package.json tools/*.sh scripts/*.sh .pre-commit-config.yaml .github/workflows/*.y*ml .forgejo/workflows/*.y*ml 2>/dev/null ``` -Anything beyond a bare `npx jscpd ` (which reads `.jscpd.json` +Anything beyond a bare `npx jscpd@ ` (which reads `.jscpd.json` automatically) is a candidate to migrate into the central config file. Fix this **before** running the analysis in Step 7, or the analysis will reflect the masking rather than the project's real state. @@ -306,7 +318,7 @@ immediately on pre-existing duplication that nobody has reviewed yet. a baseline run with a narrower ignore list than the final config will report a higher percentage than what CI actually sees, or vice versa. ```bash - npx --yes "jscpd@${PINNED_VERSION}" --reporters json --output .tmp/jscpd-baseline + npx --yes jscpd@ --reporters json --output .tmp/jscpd-baseline ``` (jscpd auto-reads `.jscpd.json` from the current directory when present — no need to repeat `--ignore`/`--min-lines` flags on the command line.) @@ -349,7 +361,7 @@ script and build-tool target: ```yaml - repo: https://github.com/kucherenko/jscpd - rev: v # latest tag; `pre-commit autoupdate` bumps it + rev: v # `pre-commit autoupdate` bumps it hooks: - id: jscpd args: ["."] @@ -366,8 +378,11 @@ script and build-tool target: Those wheels need glibc 2.34+ (or musl, macOS, Windows) and there is no sdist; where they can't install, use a `repo: local` hook with `language: node`, `entry: jscpd`, the same two overrides and - `additional_dependencies: ["jscpd@"]` — pre-commit provisions + `additional_dependencies: ["jscpd@"]` — pre-commit provisions Node for it, but `pre-commit autoupdate` won't bump that pin. +- Upstream tags start at `v4.3.0` and skip some npm releases. If + `git ls-remote --tags https://github.com/kucherenko/jscpd v` + prints nothing, use that `repo: local` hook too. - Skip the wrapper script and build-tool target below — unless the project's convention gives every check its own target, in which case make it a thin `pre-commit run jscpd --all-files`. @@ -623,10 +638,10 @@ touching the first commit. > request. Run `/analyze-duplicates` on `` with `--threshold `, -`--jscpd-version ` and `--output .tmp/duplication-report.md`. It picks up this project's -`.jscpd.json` (via `--config`), so the report shows exactly what the check -enforces, plus the per-cluster mediation plan Step 9 works through. Without -`/analyze-duplicates`, run `npx --yes "jscpd@${PINNED_VERSION}" --reporters +`--jscpd-version ` and `--output .tmp/duplication-report.md`. +It picks up this project's `.jscpd.json` (via `--config`), so with the same +jscpd release the report shows exactly what the check enforces, plus the per-cluster mediation plan Step 9 works through. Without +`/analyze-duplicates`, run `npx --yes jscpd@ --reporters console,json --output .tmp/jscpd-report ` and work from its clone list. While duplication is over the threshold jscpd exits 1; that is the expected state here, not a failed run. @@ -664,7 +679,7 @@ classification): 3. Re-run jscpd on just the affected paths to confirm the cluster is gone and no new one was introduced: ```bash - npx --yes "jscpd@${PINNED_VERSION}" + npx --yes jscpd@ ``` 4. Run the project's existing test suite for the touched area — a duplication-driven refactor is a correctness-risk change like any other; From f10ce9235a90a6ee6b8f1de1b5768eae0ea33c5c Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 12 Nov 2026 19:54:34 +0000 Subject: [PATCH 3/3] Correct the jscpd pin lookup and hook-tag details after review Read a package.json range's version from the lockfile with `npm ls jscpd --package-lock-only` (plain `npm ls` reads node_modules), and do not count a bare `npx jscpd` that follows `npm ci` as unpinned. The upstream pre-commit hook works from v5.2.1 (language: python), not v5.3.0; below that use the repo: local hook, so the tag lookup goes. The pin search also matches `JSCPD_VERSION: X` and `jscpd==X`, and the version-output example no longer claims a format that 5.0.x contradicts. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01PvaGRYu7bNGqsozPje18f8 --- skills/analyze-duplicates/SKILL.md | 4 ++-- skills/setup-duplicates-analysis/SKILL.md | 18 ++++++++++-------- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/skills/analyze-duplicates/SKILL.md b/skills/analyze-duplicates/SKILL.md index 441210b..77a5bbf 100644 --- a/skills/analyze-duplicates/SKILL.md +++ b/skills/analyze-duplicates/SKILL.md @@ -73,8 +73,8 @@ If neither works, report the error and stop: `PINNED_VERSION` is the jscpd release for the whole run, a bare `X.Y.Z`: the `--jscpd-version` value if given (e.g. the pin of a project's own duplication check, so the report matches what its CI enforces), otherwise -the version number printed above (jscpd 5.x prints `jscpd 5.4.3`, 4.x just -`4.0.5`). It is a placeholder like `MIN_LINES`, not a shell variable: write +the version number printed above (some releases prefix it with a name). +It is a placeholder like `MIN_LINES`, not a shell variable: write the number itself into every command. Every jscpd invocation below is `npx --yes jscpd@PINNED_VERSION`, never `jscpd@latest`, so all scans in a run use the same release; tell the user which one. diff --git a/skills/setup-duplicates-analysis/SKILL.md b/skills/setup-duplicates-analysis/SKILL.md index 272b6e4..bb531bf 100644 --- a/skills/setup-duplicates-analysis/SKILL.md +++ b/skills/setup-duplicates-analysis/SKILL.md @@ -104,7 +104,7 @@ grep -niE 'jscpd|duplicat' tox.ini Makefile noxfile.py pixi.toml pyproject.toml grep -rliE 'jscpd' tools/ scripts/ bin/ 2>/dev/null # The jscpd version already pinned, if any (wrapper, dependency, CI, hook rev) -grep -rnE 'jscpd@|JSCPD_VERSION *[:?]?=|"jscpd": *"' tools/ scripts/ bin/ tox.ini Makefile \ +grep -rnE 'jscpd(@|==|": *")|JSCPD_VERSION' tools/ scripts/ bin/ tox.ini Makefile \ noxfile.py pixi.toml pyproject.toml package.json .pre-commit-config.yaml \ .github/workflows .forgejo/workflows .gitlab-ci.y*ml 2>/dev/null grep -nA3 'kucherenko/jscpd' .pre-commit-config.yaml 2>/dev/null @@ -124,9 +124,11 @@ If the project already pins a jscpd version, that is `` for this run (Prerequisites), so the report and re-checks match what its check enforces. It is always a bare `X.Y.Z`: a hook's `rev: v5.4.3` means `5.4.3` (for a frozen SHA, the tag in its comment), and a `package.json` range -(`^5.4.0`) is not a pin, so take what the lockfile resolves (`npm ls jscpd`). -A runner that uses `jscpd@latest` or a bare `npx jscpd`, or runners that -pin different versions, make the setup *partially wired* (0.3) even when +(`^5.4.0`) is not a pin, so take what the lockfile resolves +(`npm ls jscpd --package-lock-only`). A runner that uses `jscpd@latest`, +or a bare `npx jscpd` that does not get jscpd from the lockfile (jscpd in +`package-lock.json` and `npm ci` run first), or runners that pin different +versions, make the setup *partially wired* (0.3) even when its check passes: pin them all to one version (the one CI runs; if none is pinned, `npm view jscpd version`) as part of Steps 1–6. @@ -374,15 +376,15 @@ script and build-tool target: untracked files that git doesn't ignore, so numbers can differ slightly.) - **No `--ignore`/`--threshold` args**: hooks run from the repo root, where jscpd reads `.jscpd.json` (the masking-hazard rule of Step 0.3). -- Since v5.3.0 the hook installs jscpd's PyPI wheel, so it needs no Node. +- Since v5.2.1 the hook installs jscpd's PyPI wheel, so it needs no Node. Those wheels need glibc 2.34+ (or musl, macOS, Windows) and there is no sdist; where they can't install, use a `repo: local` hook with `language: node`, `entry: jscpd`, the same two overrides and `additional_dependencies: ["jscpd@"]` — pre-commit provisions Node for it, but `pre-commit autoupdate` won't bump that pin. -- Upstream tags start at `v4.3.0` and skip some npm releases. If - `git ls-remote --tags https://github.com/kucherenko/jscpd v` - prints nothing, use that `repo: local` hook too. +- Older tags' hook cannot run (`Executable jscpd not found`) and some + releases have no tag: for a `` below 5.2.1 use that + `repo: local` hook too. - Skip the wrapper script and build-tool target below — unless the project's convention gives every check its own target, in which case make it a thin `pre-commit run jscpd --all-files`.