Skip to content

test(expo): add the Expo end-to-end test package and the verify-clerk-expo skill - #10087

Draft
mikepitre wants to merge 7 commits into
mike/expo-verify-hostfrom
mike/expo-verify-remote
Draft

mikepitre wants to merge 7 commits into
mike/expo-verify-hostfrom
mike/expo-verify-remote

Conversation

@mikepitre

@mikepitre mikepitre commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds end-to-end tests that prove a @clerk/expo change on an iOS simulator or an Android emulator, and the CLI that runs them, as the Node package integration/expo-native/. The CLI builds the expo-native fixture, launches it with the inputs from #10052, runs the tests against a Clerk application that it creates and deletes, and keeps a video and screenshots of each run. The device is on the machine that runs the CLI. The verify-clerk-expo skill is the instructions an agent or a developer reads, and they point at the package.

It sits on #10052. #10090, on top of this, runs these tests in CI and deletes the older tests in integration/tests/expo-native/, which this pull request leaves as they are. #10131, on top of #10090, lets a machine that cannot run a device borrow one on a CI runner.

The seven commits are in dependency order, and each adds one part. Paths below that begin src/, specs/, or .verify/ are inside integration/expo-native/.

Commit Lines added What it does
1. Package, entry point, repo wiring 78, and 1,783 of generated package-lock.json Sets integration/expo-native/ up as a small Node package: its dependencies, the control-clerk-expo command, and the .prettierignore and eslint.config.mjs entries that keep the repo's formatter and linter off its files.
2. Shared core 3,960 The tool itself. It gives each worktree its own simulator or emulator so two agents never drive the same one, starts the app with the inputs a test asks for, signs test users in without showing any key to the tests, and keeps the video, screenshots, and log of every run.
3. Throwaway Clerk application 1,754 Creates a real Clerk application for the session to test against, sets it up the same way every time, and deletes it with all its users when the session ends.
4. Platform host and the local devices 2,135 The part that knows this platform: how to build the expo-native fixture, install it on the simulator or emulator, and launch it.
5. End-to-end tests, fixtures, feature files 1,006 The tests themselves, grouped by feature, and one short page per feature that says how a user reaches it and what on screen proves it works.
6. Unit tests and their CI job 6,125 Tests of the tool's own code. They run with no simulator or emulator, no network, and no key, and a CI job runs them.
7. Docs 355 The skill, which is the instructions an agent reads to use the tool, and the notes that point to it from the repo's contributor docs.

Commits 2 and 3, specs/fixtures.ts in commit 5, and most of commit 6 are the same files, byte for byte, in clerk/clerk-ios#629 and clerk/clerk-android#1046. src/core/MANIFEST lists the core files with their hashes, and a unit test fails when one drifts. Review them once, in whichever of the three pull requests you read first. The unit tests for all of the code arrive together in commit 6, so the commits before it do not pass a test run by themselves.

The skill is SKILL.md and references/ in .claude/skills/verify-clerk-expo/, and .cursor/skills/verify-clerk-expo is a symlink to that directory. To review what is this repository's own, read SKILL.md, then src/host.ts and src/fixture.ts, then specs/golden/.

To try it on a Mac with Xcode, Node 24.8 or newer, and the team's Clerk Platform API key:

$ pnpm install
$ npm ci --prefix integration/expo-native
$ integration/expo-native/bin/control-clerk-expo doctor --platform ios
$ integration/expo-native/bin/control-clerk-expo run native-auth-view --platform ios
$ integration/expo-native/bin/control-clerk-expo down

doctor only reads, and prints a fix for each thing the machine lacks. run builds the fixture as a Debug dev client, creates the application, takes a simulator that the CLI cloned for itself, starts Metro and tsdown --watch in packages/expo, and runs the tests. A later JS edit reaches the app on the next run with no native build. down releases the device and deletes the application. The evidence stays in .verify/runs/<run-id>/.

The 17 end-to-end tests are in specs/golden/, in seven groups. Two are for iOS only, so Android runs 15.

Group What its tests cover
native-auth-view Opening and closing AuthView, its React Native logo, and a sign-in through it
user-button-and-profile The UserButton, the profile it opens, the home's sign-out, and an inline UserProfileView with onHostBack and a custom page
custom-flow-sign-in An email code sign-in on useSignIn
custom-flow-sign-up An email and password sign-up on useSignUp
token-cache-persistence The same session after a relaunch, and a signed-out start with new storage
native-js-sync A native sign-in and a native sign-out reaching useAuth, useUser, and useSession
native-modules useSignInWithGoogle opening the native Google sign-in and, on iOS, reporting a cancel, and useBiometricCredentials giving the native module's answer

Every test starts at the fixture's home and taps to the screen it needs, as a user would. The options of host.launch choose who is signed in, the mode of AuthView, and whether storage is kept from the last launch. specs/native.ts holds the locators of the home's buttons, and a unit test fails when they differ from the fixture's.

The tests assert on what a user sees, in the prebuilt views first and on the fixture's home for the outcome of a flow. Android tests find the prebuilt views by text, because the clerk-android release that @clerk/expo pins has no test tags. iOS tests use the clerk.* accessibility identifiers. Tests type only +clerk_test addresses and the test code 424242.

The two native-modules tests sign no one in. The Google test proves that the hook reads the fixture's placeholder client IDs and that the native module opens Google's sign-in. On iOS it also proves that a cancel comes back as Google sign-in was cancelled. On Android it stops once a page of Google Play services is on screen, because what that page is and whether Back closes it differ from run to run on an emulator with no Google account. It does not prove that a Google account can sign in. The biometrics test turns biometric sign-in on for the instance through the settings file beside it and expects the answer the native module gives on a simulator or an emulator. It does not prove enrolling or signing in with a biometric.

Each worktree gets one Clerk application in a workspace that holds nothing else, with the settings in src/core/instances/base.json. The CLI creates it with the Platform API key, which comes from the environment, a file, or a 1Password reference kept outside the repository. No file holds the application's secret key. After a run, the CLI searches the run directory for every secret the run used, and attach refuses to post a run that holds one.

The CLI also configures e2e's built-in agent, with anthropic/claude-haiku-5.5 and openai/gpt-6-luna-fast as its backup, only when the machine has a Vercel AI Gateway key. No committed test uses the agent, and no workflow passes a key.

The package installs its dependencies with npm ci from its own lockfile, outside the pnpm workspace. The Expo Native Runner Tests job added to ci.yml runs the package's 385 unit tests and tsc on Linux, with no device and no secret, when a pull request changes the package or a path its tests read.

Nothing in this pull request runs a test on a device in CI. The workflow in #10090 runs the 17 tests on a commit that contains this one.

Checklist

  • pnpm test runs as expected.
  • pnpm build runs as expected.
  • (If applicable) JSDoc comments have been added or updated for any package exports
  • (If applicable) Documentation has been updated

Type of change

  • 🐛 Bug fix
  • 🌟 New feature
  • 🔨 Breaking change
  • 📖 Refactoring / dependency upgrade / documentation
  • other: test tooling

🤖 Generated with Claude Code

@changeset-bot

changeset-bot Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 454052f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
clerk-js-sandbox Ready Ready Preview Oct 8, 2026 4:52pm UTC
swingset Ready Ready Preview Oct 8, 2026 4:52pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 7cd1585a-20df-484f-a882-1d122173b8d5
📥 Commits

Reviewing files that changed from the base of the PR and between e645244 and 7672257.

📒 Files selected for processing (1)
  • .claude/skills/verify-clerk-expo/test/github-report.test.ts
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

Adds the verify-clerk-expo skill for running and documenting @clerk/expo verification on iOS and Android. The implementation includes CLI commands, test-instance management, E2E execution and evidence handling, local simulator and emulator backends, and remote GitHub Actions sessions. It adds feature-specific specs and guides, shared skill access through a Cursor symlink, and CI jobs for skill checks and remote sessions.

Priority: ➖ Normal

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: 🔵 Low · up to 76722

The verification tool can lose run evidence or encounter cleanup, duplicate-runner, and proxy-check failures in specific conditions. These are bounded workflow risks; merge is possible with owner awareness and follow-up.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 257 functions across 52 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main changes: adding Expo end-to-end test tooling and the verify-clerk-expo skill.
Description check ✅ Passed The description is directly related to the changeset and explains the verification CLI, skill, device flows, tests, workflows, and evidence handling.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@mikepitre
mikepitre force-pushed the mike/expo-verify-remote branch from 160b44a to 729ecd1 Compare October 6, 2026 06:03
@mikepitre
mikepitre force-pushed the mike/expo-verify-remote branch from 729ecd1 to 2d05137 Compare October 6, 2026 06:41
@mikepitre
mikepitre changed the base branch from mike/expo-verify-skill to mike/expo-verify-host October 6, 2026 16:30
@mikepitre
mikepitre force-pushed the mike/expo-verify-remote branch from 2d05137 to 43ca8cc Compare October 6, 2026 16:55
@mikepitre
mikepitre force-pushed the mike/expo-verify-remote branch from 43ca8cc to e31b477 Compare October 6, 2026 17:50
@mikepitre mikepitre changed the title test(expo): borrow a simulator or emulator on a CI runner for the verify skill test(expo): add a verify skill that drives the expo-native fixture on a local or borrowed device Oct 6, 2026
@mikepitre
mikepitre added this pull request to stack #10100 October 6, 2026 19:57
@mikepitre
mikepitre marked this pull request as ready for review October 6, 2026 19:59
@mikepitre
mikepitre force-pushed the mike/expo-verify-remote branch from caefb95 to 8647439 Compare October 7, 2026 17:01
@mikepitre
mikepitre force-pushed the mike/expo-verify-remote branch from b0d1508 to 249bffc Compare October 7, 2026 18:02
@mikepitre
mikepitre force-pushed the mike/expo-verify-remote branch from 9b925c9 to e0f90bd Compare October 7, 2026 20:31
mikepitre and others added 4 commits October 8, 2026 00:17
…nd repo wiring

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…roker, secrets, and evidence

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on per worktree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
return { source: readFileSync(join(packageDir, specPath), 'utf8'), declaration: existsSync(file) ? readFileSync(file, 'utf8') : null };
}

export const sourceHash = (text: SpecText): string => createHash('sha256').update(JSON.stringify([text.source, text.declaration])).digest('hex');

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a false positive, so no code changed. sourceHash hashes the text of a test file and its settings file to tell whether the test was edited since a run was planned (verbs.ts compares the two hashes). The input is source code that is already in the repository, and no password or other secret reaches it. CodeQL appears to treat the MFA settings text in this file as a credential. The alert can be dismissed as a false positive in code scanning.

🤖 Reviewed by Claude Code

mikepitre and others added 3 commits October 8, 2026 12:48
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – swingset — 454052fe Deployed Oct 8, 2026 by vercel[bot]
Preview – clerk-js-sandbox — 454052fe Deployed Oct 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants