Skip to content

Verify keyring deletes actually removed the credential (Refs #3011) - #3040

Merged
acoliver merged 2 commits into
mainfrom
issue3011
Aug 5, 2026
Merged

Verify keyring deletes actually removed the credential (Refs #3011)#3040
acoliver merged 2 commits into
mainfrom
issue3011

Conversation

@acoliver

@acoliver acoliver commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

TLDR

@napi-rs/keyring erases backend errors into the boolean returned by deleteCredential(), so a failed delete is indistinguishable from "there was nothing to delete" and a secret can survive a delete that was reported as successful. This adds a read-back probe after every delete and rejects if the credential is still there.

The erasure is worse than issue #3011 described. The issue assumed failures collapse to false. On macOS they collapse to true.

Dive Deeper

The macOS path destroys the delete status three times over

Layer Code Effect
security-framework os/macos/passwords.rs:79-85 pub fn delete(self) { unsafe { SecKeychainItemDelete(...); } } returns unit; drops the OSStatus
apple-native-keyring-store-1.0.1 keychain.rs:87-93 item.delete(); Ok(()) Ok unconditionally
@napi-rs/keyring .is_ok() -> true

So on macOS a refused delete resolves true. A caller that correctly checks the return value is still told the credential was removed while it is still in the Keychain. Checking the boolean is not sufficient on any platform, and on macOS it is actively misleading.

Note the ordering in layer 2: find_generic_password(...)? runs before the delete, so a genuinely missing credential returns Err(NoEntry) early and still yields false. The discard applies specifically when the item was found and the deletion then failed — the case that matters.

Per-platform erasure

Platform Delete succeeded Delete failed (OS refused) Nothing to delete
macOS true true false
Linux / Windows true false false

What this does

Read the credential back after every delete. If it is still present, reject. Only once absence is confirmed is the native boolean returned unchanged, so true still means "a credential was deleted" and false still means "there was nothing to delete".

There is deliberately no if (deleted) return true fast path — that would skip verification on exactly the macOS case above.

Why the error message is fixed and interpolation-free

SecureStore.classifyError() re-derives the error code from the message text and ignores SecureStoreError.code. validateKey() only rejects /, \, \0, . and .. — so a key literally named not found is legal. Interpolating service/account into the message would let such a key re-classify the failure as NOT_FOUND, which the delete path is designed to treat as benign, silently defeating the throw.

The message is therefore the constant Credential remains after keyring deletion, and service/account go to a debug log instead. The read-back value is never logged or thrown.

The wording is also postcondition-phrased ("remains after") rather than causation-phrased ("delete did not take effect"), because the delete and the probe are two separate native calls and a concurrent writer between them could produce the same state.

Scope and honest limitations

Please read this section — the fix is real but bounded.

  • Observable today via MCP. KeychainTokenStorage.deleteCredentials() calls this adapter directly and propagates the rejection.
  • Not yet observable via SecureStore.delete(). deleteLocked() still wraps the call in a bare catch {}. The in-flight PR Harden secure-store fallback: surface delete() keyring failures and align has() with get() (Fixes #1985) #3010 (issue1985, CI green) replaces that with classification and a rethrow of anything that is not NOT_FOUND. secure-store.ts is owned by that PR, so it is deliberately untouched here — the two changes are textually disjoint and will not conflict.
  • Cannot detect a failure that also breaks the read-back. A fully locked store, or an Ambiguous collision, erases the read to null too, so the probe reports absent. Detecting those requires the upstream fix.
  • Upstream cannot fully fix macOS either. I filed keyring-node#137 and PR #138 to propagate the errors instead of erasing them. That fixes Linux and Windows delete, but on macOS there is no error left to propagate by the time the binding sees it — the fix has to go into apple-native-keyring-store.

This is why the PR says Refs, not Fixes#3011 should stay open until #3010 lands and the upstream work resolves.

Dependency decision (asked for by the issue)

There is no earlier version to pin to that propagates these errors; vendoring or replacing the binding is disproportionate while a focused upstream PR is open; the local read-back mitigation is worth having now. Recorded in project-plans/issue3011/plan.md.

Reviewer Test Plan

bun test ./test-bun/keyring-delete-verification.bun.ts --preload ./test-setup-storage-isolation.ts   # from packages/storage

11 cases. The important ones are 7 and 8 (credential survives a false / true delete -> rejects) and 10 (service/account named not found still produce the fixed message).

To confirm they are real regression tests rather than tests of existing behaviour, revert deletePassword to return entry.deleteCredential(); and re-run — case 8 fails with expect(caught).toBeInstanceOf(SecureStoreError). Restoring makes it pass. I verified this both ways.

The fake @napi-rs/keyring controls the native boolean and whether the entry is actually removed independently, which is what allows the macOS true-but-survives case to be modelled at all.

Testing Matrix

🍏 🪟 🐧
npm run
npx
Docker
Podman - -
Seatbelt - -

Verified locally on macOS: npm run test, lint, lint:eslint-guard, typecheck, format, build, and the stepfun-37 smoke test all pass. The change is platform-independent TypeScript; the storage suite is 33/33.

Linked issues / bugs

Refs #3011. Related to #3010 (which makes this observable through SecureStore.delete()).

Upstream: keyring-node#137, keyring-node#138.

Summary by CodeRabbit

  • Bug Fixes
    • Improved secure credential deletion verification.
    • Detects when credentials remain after a deletion attempt, including when the underlying operation reports success.
    • Provides clearer failure handling while protecting credential details from appearing in error messages.
    • Propagates errors encountered while verifying deletion.

@napi-rs/keyring erases backend errors into the boolean returned by
deleteCredential(), so SecureStore cannot tell a failed delete from
"there was nothing to delete" and a secret can survive a delete that
was reported as successful.

The erasure is worse on macOS than the issue described. The delete
status is discarded at three layers: security-framework's macOS
SecKeychainItem::delete() returns unit and drops the OSStatus,
apple-native-keyring-store calls item.delete() then returns Ok(())
unconditionally, and the binding maps that to true. So on macOS a
refused delete resolves TRUE, not false. Checking the boolean is not
enough on any platform, and on macOS it is actively misleading.

Read the credential back after every delete and reject if it is still
there. The native boolean is returned unchanged only once absence has
been confirmed, so true still means "a credential was deleted" and
false still means "there was nothing to delete".

The thrown message is fixed and interpolation-free on purpose:
SecureStore.classifyError() re-derives the code from message text and
ignores SecureStoreError.code, and validateKey() permits a key literally
named "not found" — interpolating one would classify the failure as
NOT_FOUND and get it swallowed downstream. Service and account go to a
debug log instead; the read-back value is never logged or thrown.

This closes the gap at the layer the adapter owns. It is not yet
observable through SecureStore.delete(), whose bare catch is rewritten
by the separate in-flight PR #3010, and it cannot detect a failure that
also makes the read-back fail (a fully locked store, or an Ambiguous
collision, both of which the binding erases to null). Refs rather than
Fixes for that reason.
@github-actions github-actions Bot added the maintainer:e2e:ok Trusted contributor; maintainer-approved E2E run label Aug 4, 2026
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 28 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b800f2bd-acda-4466-b0e5-59626e5f96f6

📥 Commits

Reviewing files that changed from the base of the PR and between b243c54 and 90c10d7.

📒 Files selected for processing (1)
  • packages/storage/test-bun/keyring-delete-verification.bun.ts
📝 Walkthrough

Walkthrough

The secure-store keyring adapter now verifies credential removal after native deletion. A new helper classifies read-back results, while tests cover deletion outcomes, surviving credentials, probe errors, stable error messages, runtime guards, and manifest registration.

Changes

Secure-store deletion verification

Layer / File(s) Summary
Read-back verification contract
packages/storage/src/secure-store/keyring-delete-verification.ts
Adds KeyringDeleteOutcome and verifyKeyringDelete. null means absent. Empty strings remain present. Read errors propagate.
Adapter deletion enforcement
packages/storage/src/secure-store/default-keyring-adapter.ts
The adapter reads the credential after native deletion. It throws SecureStoreError with UNAVAILABLE when the credential remains.
Deletion behavior validation
packages/storage/test-bun/keyring-delete-verification.bun.ts, scripts/bun-test-manifest-data-storage.ts
Adds fake keyring behavior and tests for deletion results, surviving credentials, errors, message stability, runtime replacement, and test manifest registration.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the keyring deletion verification added by the pull request.
Description check ✅ Passed The description covers the change, rationale, limitations, test plan, testing results, and linked issues; some matrix entries remain unverified.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue3011

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

Before this change, the default keyring adapter treated @napi-rs/keyring's deleteCredential() result as the final answer: a failed native delete could be reported back as false, or on macOS even as true, with no way to tell whether the credential was actually removed or had simply survived in the OS keyring. After this PR, the adapter performs a read-back probe after every delete and only returns the native boolean once the credential is confirmed absent; if the secret is still present, the delete path rejects with a fixed, interpolation-free error while logging service and account at debug level.

Release Notes

Bug Fixes

  • Verifies that keyring deletes actually removed the credential by probing the keyring after every delete and rejecting if the secret remains.
  • Prevents a macOS-specific failure mode where a refused delete previously resolved as true, making a surviving credential appear successfully deleted.
  • Uses a fixed error message for delete-verification failures so downstream classification cannot be bypassed by specially crafted key names.

Tests

  • Adds behavioral tests for keyring delete verification outcomes, including read-back cases for absent credentials, surviving credentials, empty strings, and probe rejections.
  • Adds adapter-level delete tests covering successful deletes, no-op deletes, macOS-style silent failures, probe error propagation, and runtime-replaced guard behavior.
  • Registers the new delete-verification test in the Bun test manifest so it runs in CI.

Documentation

  • Adds an issue plan documenting the root cause, per-platform delete erasure behavior, proposed verification approach, and behavioral test plan.

Changes

Layer File(s) Summary
core packages/storage/src/secure-store/default-keyring-adapter.ts, packages/storage/src/secure-store/keyring-delete-verification.ts Implements keyring delete verification and updates the default keyring adapter to confirm credential removal after deletion
tests packages/storage/test-bun/keyring-delete-verification.bun.ts Adds behavioral tests covering keyring delete verification outcomes and adapter delete behavior
ci scripts/bun-test-manifest-data-storage.ts Updates the bun test manifest to include the new keyring delete verification test
docs project-plans/issue3011/plan.md Documents the issue analysis, proposed solution, and behavioral test plan for keyring delete verification

Magnitude

🎯 1 (S)
711 additions, 1 deletions, 5 changed files across 1 package, 0 acceptance criteria

Related

Pre-merge Checks

Check Status Note
Title ...
Description ...
Linked Issues ...
Out of Scope ...

Walkthrough generated by LLxprt PR Review. Planner issue: #2256

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/storage/test-bun/keyring-delete-verification.bun.ts`:
- Around line 293-301: Update the case 11 test around adapter.deletePassword to
track the existing mocks for deleteCredential and getPassword, then assert both
call counts remain zero after the rejection. Preserve the SecureStoreError
assertion while verifying the runtime-replaced guard runs before either native
operation.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: be040f2a-81d8-44e8-9b41-890b86c05573

📥 Commits

Reviewing files that changed from the base of the PR and between fd4c82f and b243c54.

⛔ Files ignored due to path filters (1)
  • project-plans/issue3011/plan.md is excluded by !project-plans/**
📒 Files selected for processing (4)
  • packages/storage/src/secure-store/default-keyring-adapter.ts
  • packages/storage/src/secure-store/keyring-delete-verification.ts
  • packages/storage/test-bun/keyring-delete-verification.bun.ts
  • scripts/bun-test-manifest-data-storage.ts

Comment thread packages/storage/test-bun/keyring-delete-verification.bun.ts
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

OpenCodeReview — PR #3040

  • Reviewed head SHA: 90c10d74507d820674bf2188f96b8b9eafd1eb8b
  • Merge base: 7cf0a3825609510352713e01c7890aa78191b0af
  • Range: incremental from b243c54950db81944626e32bb54e77d2b84f95ca
  • Range fallback: none
  • Scope: selected 1 file(s), +25/-2; cumulative 5 file(s), +711/-1
  • Tokens: 9418 total (5601 input, 3817 output, 0 cache)
  • OCR version: open-code-review v1.8.4 (e78474478) linux/amd64 built at: 2026-08-01T03:27:37Z https://github.com/alibaba/open-code-review
  • Phase: review
  • Exit code: 0
  • Run: https://github.com/vybestack/llxprt-code/actions/runs/30961831513
  • No findings.
  • Artifacts: ocr-review-output contains raw JSON, stdout, stderr, preview, phase, and exit-code diagnostics.
  • WARNING: Changed-file coverage 0/1 preview files covered is below the 90% threshold.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Summary

Package Lines Statements Functions Branches
CLI N/A% N/A% N/A% N/A%
Core N/A% N/A% N/A% N/A%
CLI Package - Full Text Report
CLI full-text-summary.txt not found at: coverage_cli/packages/cli/coverage/full-text-summary.txt
Core Package - Full Text Report
Core full-text-summary.txt not found at: coverage_core/packages/core/coverage/full-text-summary.txt

For detailed HTML reports, please see the 'coverage-reports-24.x-ubuntu-latest' artifact from the main CI run.

The case only asserted that deletePassword() rejects, which also passes
if deleteCredential() or getPassword() ran before the guard threw.

Prove it behaviourally rather than with call counters: seed a credential
and stage the fake so a native delete would remove it and a native read
would reject with a distinctive probe error. Asserting the
RUNTIME_REPLACED identity and the surviving credential shows neither
native call happened.

Verified by removing the delete guard: the case then fails.
@acoliver
acoliver merged commit bcdeeba into main Aug 5, 2026
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer:e2e:ok Trusted contributor; maintainer-approved E2E run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant