Help wanted: edge case lockfile fixtures for regression testing

[sonukapoor]

What we need

We recently fixed two remediation bugs that were only discovered because a real-world project exposed them. The fixes are in — but we have no automated tests that would catch regressions if the same logic breaks again.

We want to build a set of small, crafted lockfile fixtures in examples/ that each test one specific edge case. These are not case studies — they do not need write-ups or blog content. Just a minimal package.json and lockfile that reproduces a specific scanner behavior.

Why this matters

Every fixture becomes a permanent regression test. If a future change breaks the behavior, the test catches it before it ships.

Fixtures we need

Each one is a standalone contribution — claim one by commenting below.

One fixture at a time. Please claim only one fixture. Once your PR is merged, you are welcome to claim another.

#	Fixture name	What it tests	Assignee	Status	PR
Open
3	multi-path-same-vuln	Same vulnerable package reachable through two different intermediate parents with different ranges — both paths should get correct commands	—	open	—
8	no-fix-available	Vulnerable package with no published safe version — scanner should report no fix, not suggest a wrong one	—	open	—
9	workspace-transitive	Vulnerable package pulled in through an npm workspace scope — fix command should include the correct workspace flag	—	open	—
12	pre-release-fix	Advisory fix hint points to a pre-release version — scanner should not suggest a pre-release as the safe target	—	open	—
In progress
2	exact-pinned-intermediate	Intermediate parent pins vulnerable package to an exact version (not a range) — within-range fix should not be suggested	@Ayush7614	in progress	—
11	multiple-versions-same-pkg	Same package installed at two different versions (e.g. lodash@3 and lodash@4) — both should be reported as separate findings if both are vulnerable	@coder-Yash886	in progress	#633
Completed
1	deep-chain-no-fix	3+ level chain where the intermediate parent range does NOT cover the fix — resolver should fall back to best-effort parent upgrade, not suggest npm update	@Ayush7614	completed	#558
4	pnpm-within-range	pnpm equivalent of wrong-parent — immediate parent range covers the fix, should suggest pnpm update <package>	@Ayush7614	completed	#557
5	yarn-within-range	yarn equivalent — should suggest yarn upgrade <package>	@coder-Yash886	completed	#537
6	pnpm-aliased-chain	pnpm v9 aliased dep in a deep chain — aliased intermediate should not break path resolution	@Ayush7614	completed	#559
7	bun-within-range	bun equivalent — should suggest bun update <package>	@coder-Yash886	completed	#562
10	dev-only-finding	Vulnerable package that only appears in devDependencies — scanner should classify it correctly and not include it in prod-only scans	@coder-Yash886	completed	#613

Have an edge case in mind that is not on this list? Comment below with a description and we will add it.

How to contribute a crafted fixture

1. Comment below to claim a fixture number
2. Create a minimal lockfile that reproduces the scenario (pinned to a specific vulnerable version that OSV knows about)
3. Open a PR adding examples/<fixture-name>/ with package.json and the lockfile
4. Run cve-lite examples/<fixture-name> and paste the output in the PR description so we can verify the behavior

If you have cloned the repo and are running from source:

npm install && npm run build
node dist/index.js examples/<fixture-name>

Don't have a crafted fixture? Scan a real project instead

You can find edge cases by scanning a project you already work on. Here is the workflow:

Step 1 — Install the CVE Lite CLI AI skill (once)

CVE Lite CLI ships with an AI assistant skill that teaches Claude Code how to interpret scan output — direct vs transitive findings, fix command logic, parent upgrade paths, confidence levels. Install it once:

cve-lite install-skill

Full guide: https://owasp.org/cve-lite-cli/docs/ai-assistant-integration#installing-skills

Step 2 — Scan your project and save the JSON output

npm install -g cve-lite-cli
cve-lite . --json

Step 3 — Verify the fix commands with Claude Code

Share the generated cve-lite-scan-*.json file with Claude Code (or any AI assistant with the skill installed) and ask:

Here is my CVE Lite CLI scan output . Does each suggested fix 
command actually resolve the vulnerability given the dependency path shown? 
Are there any cases where a command might not work or might fix less than 
it claims?

Because the skill is installed, the AI already understands the JSON schema — dependency paths, fix targets, confidence levels — and can give a precise answer without any extra context.

Step 4 — Compare the answers

• If CVE Lite and the AI agree the fix is correct — no action needed.
• If they disagree or the AI flags an issue — you may have found an edge case. Open an issue describing the discrepancy and we will investigate.

This workflow makes you both a tester and an early adopter. Any real discrepancy you find is a direct contribution to improving the tool.

Questions?

Ask in this thread — we will help you set up the fixture or interpret a scan result.

[sonukapoor]

@Ayush7614 @coder-Yash886 — tagging you both since you have been the most active contributors recently. Would love your help claiming some of these fixtures, and if you have scanned any real projects recently the verification workflow in Step 3 is a great way to turn that into a contribution. Let us know if you have any questions.

[Ayush7614]

Claiming a few fixtures — happy to help turn these into permanent regression examples.

#	Fixture	Notes
1	deep-chain-no-fix	Complements existing examples/wrong-parent/ (within-range → npm update). I will craft a 3+ level npm chain where the intermediate range does not cover the fix so the resolver must fall back to best-effort parent upgrade.
4	pnpm-within-range	pnpm equivalent of wrong-parent — immediate parent range already covers safe child; expect pnpm update <pkg> not a wrong parent bump.
6	pnpm-aliased-chain	Deep chain with a pnpm v9 aliased intermediate (dep name ≠ real package name) — path resolution should stay correct end-to-end.

I will open separate PRs per fixture (minimal package.json + lockfile, scan output in PR description) unless maintainers prefer one combined PR.

Also ran the real-project verification workflow on several lockfile snapshots recently (pnpm/npm monorepos) — will comment here if any scan disagrees with expected fix logic before opening issues.

@sonukapoor — let me know if you would rather I take different numbers from the table.

[sonukapoor]

Thanks @Ayush7614 — great picks, and your notes show you understand exactly what each one needs to exercise. Separate PRs per fixture is the right call — easier to review and merge independently.

Table updated with your assignments. Looking forward to the PRs.

[coder-Yash886]

Picking up a few fixtures — would love to see these edge cases covered with solid regression tests.

#	Fixture	Notes
5	yarn-within-range	yarn version of the wrong-parent scenario — parent range already includes the patched version, so the correct output should be yarn upgrade <pkg> rather than bumping the parent.
7	bun-within-range	same logic but for bun — fix sits within the existing parent range, scanner should emit bun update <package> and nothing more.
10	dev-only-finding	package lives exclusively in devDependencies — goal is to confirm the scanner flags it under the right category and skips it in production-only scan mode.
11	multiple-versions-same-pkg	two installs of the same package at different major versions side by side — each vulnerable version should show up as its own independent finding, not get merged or skipped.

I will open separate PRs per fixture (minimal package.json + lockfile, scan output in PR description) unless maintainers prefer one combined PR.

@sonukapoor — let me know if you would rather I take different numbers from the table.

[sonukapoor]

Thanks @coder-Yash886 — great picks. Your four complement what @Ayush7614 is working on nicely, so we will have solid coverage across npm, pnpm, yarn, and bun. Separate PRs per fixture is the right call.

Table updated with your assignments.

[Ayush7614]

Status update on my claimed fixtures from #528:

#	Fixture	Status	PR
1	deep-chain-no-fix	✅ Merged	#558
4	pnpm-within-range	✅ Merged	#557
6	pnpm-aliased-chain	🟡 Submitted — rebased against main, merge conflict in examples/readme.md resolved	#559

Scan highlights (verified in each PR):

• 1 — express → body-parser → qs@6.14.2; intermediate range ~6.14.0 does not cover fix → npm install express@4.22.2 (not npm update qs)
• 4 — body-parser → qs@6.15.1; parent range ~6.15.1 covers fix → pnpm update --no-save qs
• 6 — deep chain through pnpm v9 alias @remix-run/dev → @vercel/remix-run-dev → vm2@3.9.19; path resolves correctly → pnpm add vercel@32.0.2

---

Also claiming the remaining open fixtures from the table — will open separate PRs per fixture:

#	Fixture	Notes
2	exact-pinned-intermediate	Intermediate parent pins vulnerable package to an exact version — within-range fix must not be suggested
3	multi-path-same-vuln	Same vulnerable package via two intermediate parents with different ranges — both paths need correct commands
8	no-fix-available	Vulnerable package with no published safe version — scanner should report no fix, not a wrong command
9	workspace-transitive	Vulnerable package via npm workspace scope — fix command must include the correct workspace flag
12	pre-release-fix	OSV fix hint points to a pre-release — scanner must not suggest a pre-release as the safe target

@sonukapoor — let me know if you'd prefer I prioritize a different order. #559 should be ready to merge once CI passes.

[sonukapoor]

#559 is already merged — you are all caught up. #2 is a great next pick, go for it.

[Ayush7614]

@sonukapoor — could you update the main tracker table for #1, #4, and #6? Suggested rows:

#	Fixture name	Assignee	Status	PR
1	deep-chain-no-fix	@Ayush7614	completed	#558
4	pnpm-within-range	@Ayush7614	completed	#557
6	pnpm-aliased-chain	@Ayush7614	completed (PR merged pending / submitted)	#559

Merge status: #557 and #558 are merged; #559 is rebased and ready to merge per your review.

Full replacement lines for the main table (What it tests column unchanged):

| 1 | `deep-chain-no-fix` | 3+ level chain where the intermediate parent range does NOT cover the fix — resolver should fall back to best-effort parent upgrade, not suggest npm update | @Ayush7614 | completed | #558 |
| 4 | `pnpm-within-range` | pnpm equivalent of `wrong-parent` — immediate parent range covers the fix, should suggest `pnpm update <package>` | @Ayush7614 | completed | #557 |
| 6 | `pnpm-aliased-chain` | pnpm v9 aliased dep in a deep chain — aliased intermediate should not break path resolution | @Ayush7614 | completed | #559 |

[sonukapoor]

Table updated — #1, #4, #6 are all marked completed. Great work @Ayush7614, and thanks for the detailed scan highlights on each one.

One process note going forward: please claim one fixture at a time. Once your PR is merged, you are welcome to pick up the next one. This keeps the table accurate and ensures open fixtures stay available for other contributors.

For your next one, #2 (exact-pinned-intermediate) is assigned to you. Once that is merged, comment here and claim another from the open list.

@coder-Yash886 — same applies: please focus on #10 before picking up #11. Once #10 is merged, #11 is yours to claim.

[coder-Yash886]

@sonukapoor i looking to work on fixture #11

[sonukapoor]

@coder-Yash886 assigned. Thank you

[coder-Yash886]

@sonukapoor I raised the Pr #633