Skip to content

fix(skills): correct stale EQL v3 rollout guidance in bundled skills#706

Merged
tobyhede merged 2 commits into
mainfrom
fix/skills-accuracy
Jul 21, 2026
Merged

fix(skills): correct stale EQL v3 rollout guidance in bundled skills#706
tobyhede merged 2 commits into
mainfrom
fix/skills-accuracy

Conversation

@tobyhede

@tobyhede tobyhede commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

skills/*/SKILL.md ship inside the stash tarball and are copied into customer repos by installSkills(). Nothing type-checks them. Three had guidance that was not merely out of date but unrunnable as written.

The main defect

stash-drizzle and stash-supabase documented the v2 rename-cutover as the only rollout path, while their schema-add examples used v3 domains. Under EQL v3 there is no rename step — stash encrypt cutover exits "not applicable" (cutover.ts:117). A customer following either skill end to end hit a command that refuses to run.

Both skills also carried "#648 hasn't landed, write your own backfill script" notes. #648 closed 2026-07-17@cipherstash/migrate and stash encrypt * are EQL v3 compatible.

Corrected across both files: the v2-only callouts became accurate version-dependent notes (auto-detection; v3 = backfill → switch reads by name → drop), the cutover step now leads with the v3 path and marks v2 legacy, and the drop step reflects a real behavioural difference — v3 drops the original column (no _plaintext twin exists), v2 drops <col>_plaintext (drop.ts:151, :45).

Deleted: the 43-line "Interim path until #648: the v2 encrypted twin" section in stash-supabase, which existed solely as a workaround for the now-closed issue.

Also found

  • stash-cli/SKILL.md:607 claimed EQLInstaller's eqlVersion defaults to 2. It is 3DEFAULT_EQL_VERSION: EqlVersion = 3 (installer/index.ts:69), pinned by a test asserting toBe(3).
  • stash-cli/SKILL.md:511 cited EQL v3: make it the default, and stop the CLI recommending db push (a v2/Proxy-only step) #585 as open tracking. Closed 2026-07-13. The underlying v2 gap is real but was resolved by making v3 the default rather than by the decoupling the skill described — reworded rather than deleted.

Scope note

Fixing only the #648 sentences would have left both skills instructing v3 users to run a cutover that exits "not applicable", so this extends to the cutover and drop steps. Larger diff than a pure staleness fix, but scoped to correctness. No EQL v2 content was removed except the interim-twin section — v2 is still shipped and supported.

What was checked and found correct

This was a full sweep, not a targeted patch:

  • Issue states, all via gh issue view: Make @cipherstash/migrate (and stash encrypt *) compatible with EQL v3 #648 and EQL v3: make it the default, and stop the CLI recommending db push (a v2/Proxy-only step) #585 closed → fixed. stack#690, stack#654, stack#657, protectjs-ffi#137, encrypt-query-language#407 all still open and correctly described → left alone.
  • stash-prisma-next already correct for 4923c0acipherstashFromStack is the v3 name, cipherstashFromStackV2 the v2 one, and the runtime Encrypted* envelopes were deliberately left prefixed by that commit.
  • CLI manifest cross-check — every command and flag named in stash-cli resolves against stash manifest --json. Remaining flags are third-party (pnpm --config.*, drizzle-kit --custom, supabase --env-file) or documented as explicitly not existing.
  • All 15 referenced subpath exports resolve; all 12 named exported symbols exist; all 14 EQL v3 domain names match the catalog.
  • No Linear IDs present.

Unverified

The stash-supabase claim about the v2 wrapper's dual-write shape, and stash-encryption:819's "tracked separately" for cutover/EQL-config decoupling (no issue number to check). Both left unchanged rather than guessed at.

Changeset included — skills/ ships in the stash tarball, so a skills-only change still needs one.

🤖 Generated with Claude Code

@cipherstash/migrate and `stash encrypt *` gained EQL v3 support (#648,
closed 2026-07-17), but the shipped skills still said the rollout tooling
was v2-only. These skills are copied into customer repos, so the stale
text steered users away from v3 toward workarounds they no longer need.

- stash-drizzle, stash-supabase: replace the "v3 not supported
  end-to-end" callouts with an accurate EQL version note (auto-detection
  by Postgres domain type; v3 has no cut-over rename).
- stash-supabase: drop the "Interim path until #648: the v2 encrypted
  twin" section — a v2 twin is no longer needed for CLI-managed backfill.
- stash-drizzle, stash-supabase: document that `stash encrypt drop`
  targets the original column under v3, `<col>_plaintext` under v2.
- stash-cli: EQLInstaller `eqlVersion` defaults to 3, not 2
  (DEFAULT_EQL_VERSION, installer/index.ts:69); reword the v2 cut-over
  gap note that cited #585 as open tracking.

Verified `stash manifest --json` resolves every command/flag named in
stash-cli. code:check clean (errors); changeset added.
@tobyhede
tobyhede requested a review from a team as a code owner July 20, 2026 02:48
@changeset-bot

changeset-bot Bot commented Jul 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: e2d76b7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages
Name Type
stash Patch
@cipherstash/basic-example Patch
@cipherstash/e2e Patch
@cipherstash/stack Patch
@cipherstash/stack-drizzle Patch
@cipherstash/stack-supabase Patch
@cipherstash/prisma-next Patch
@cipherstash/wizard Patch
@cipherstash/bench Patch
@cipherstash/test-kit Patch
@cipherstash/prisma-next-example Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@tobyhede, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 7 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9c544c89-09a2-4277-98d7-02c80098fe47

📥 Commits

Reviewing files that changed from the base of the PR and between 4888995 and e2d76b7.

📒 Files selected for processing (4)
  • .changeset/skills-eql-v3-accuracy.md
  • skills/stash-cli/SKILL.md
  • skills/stash-drizzle/SKILL.md
  • skills/stash-supabase/SKILL.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/skills-accuracy

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@auxesis auxesis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict

Docs-only PR (3 bundled skills/*/SKILL.md + a stash patch changeset). The core corrections check out against the code: DEFAULT_EQL_VERSION: EqlVersion = 3 (packages/cli/src/installer/index.ts:69), stash encrypt drop targets the original column under v3 / <col>_plaintext under v2 with backfilled vs cut-over phase gates plus a live coverage check (packages/cli/src/commands/encrypt/drop.ts:112-152), and backfill auto-detects the EQL version. Removing the #648 interim-v2-twin path is right — the tooling supports both generations now.

One claim is over-stated and repeated four times; see the inline comment. Otherwise good to merge.

Review stats

Source Raw findings Kept
claude (claude-opus-4-8) [test-gap] 0 inline (1 substantive note in summary) 1
codex (gpt-5.5) [test-gap] 0 0
  • Cross-model overlap: 0 kept findings corroborated by 2+ models (codex reported nothing).
  • Kept findings: 1 (single-source, verified against packages/cli/src/commands/encrypt/cutover.ts).
  • Dropped as unverifiable: 0.

Additional findings not posted inline

  • Follow-up, not blocking (single-source: claude). The v3 branch of stash encrypt cutover — the exit-0 "not applicable" path, the dropped terminal path, and the exit-1 not-yet-backfilled path — has no test under packages/cli/src/__tests__/ or packages/cli/tests/. These skills now assert those exit codes as documented contract, so a small unit test over the three v3 phases would keep the skill text honest as the code moves. Worth an issue rather than blocking this PR.

Comment thread skills/stash-drizzle/SKILL.md Outdated
Comment thread skills/stash-supabase/SKILL.md Outdated
@tobyhede

Copy link
Copy Markdown
Contributor Author

Re: the non-blocking follow-up about the v3 stash encrypt cutover branch having no test coverage — that coverage already exists. The reviewer checked packages/cli/src/__tests__/ and packages/cli/tests/, but the encrypt tests are co-located with the command at packages/cli/src/commands/encrypt/__tests__/encrypt-v3.test.ts. It pins all three v3 exit-code contracts the skills now document:

  • backfilled → exit 0short-circuits on a backfilled v3 column: guidance, no rename, no config machine, exit 0 (asserts process.exit not called)
  • not-backfilled → exit 1v3 mid-backfill: does NOT tell the user to switch; exits 1 (asserts exit(1))
  • dropped → exit 0v3 already dropped: terminal phase is "nothing to do", not "finish the backfill"; exit 0 (asserts process.exit not called)

All 16 tests in that file pass on the current branch. No new test needed — the contract is already guarded. Closing out the follow-up.

@auxesis auxesis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the feedback @tobyhede 😻

@tobyhede
tobyhede merged commit 5edef3b into main Jul 21, 2026
9 of 10 checks passed
@tobyhede
tobyhede deleted the fix/skills-accuracy branch July 21, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants