Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 166 additions & 0 deletions .claude/skills/ios-simulator/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
---
name: ios-simulator
description: Use when validating BlockNote behavior that Vitest browser emulation cannot reproduce, especially iOS Safari software-keyboard input, native selection, autocapitalization, focus, or viewport behavior. Includes Appium setup, model/DOM verification, and simulator recordings.
---

# iOS simulator testing for BlockNote

## Choose the right layer

Try a red-first Vitest browser-mode regression in Docker first; follow
`../testing-skill/SKILL.md`. Never run the repository's browser suite natively.
Use the simulator for the OS-owned half that emulation cannot establish, or when
the user explicitly requests native validation. A simulator recording is evidence,
not automated regression coverage or proof of physical-device/iPad/IME behavior.

## Prerequisites

- macOS, full Xcode selected, an installed iOS runtime, and Node.js/pnpm.
- Docker for browser tests. Use only `vp` / `pnpm`, not npm/yarn.
- Optional for recording review: FFmpeg (`ffmpeg` and `ffprobe`, available via
`brew install ffmpeg`) for the contact-sheet commands in the debugging reference.
- A local Safari-accessible dev server from **this checkout**. Another worktree
may own the usual port; verify the URL rather than assuming port 5173.
- An approved external temporary directory for sessions, logs, captures and driver
installation. Keep generated evidence out of the PR checkout.

Read `references/native-debugging.md` for live editor inspection, input controls, and
failure diagnosis. Check current Appium/XCUITest documentation when capabilities
or versions differ; the versions below are a verified pairing, not a latest claim.

## 1. Use the dev server and minimal example

Reuse a dev server already running for this checkout, or start one in a Paseo
terminal (or a managed background process):

```sh
pnpm exec vp run --filter @blocknote/example-editor dev --host
```

Read the actual URL/port from the server output; let the server choose an available
port. This serves `playground`. Open its `basic/minimal` example, or the existing example
that demonstrates the reported behavior. Preserve its real UI and styling. Do not
create a diagnostic page or modify the example to expose globals.

Set `PAGE_URL` to that example's actual URL. Verify source imports point to this
worktree. Do not stop another checkout's server. Record owned processes/sessions.

The running editor is already accessible through TipTap's DOM reference:

```js
document.querySelector(".ProseMirror").editor.schema.cached.blockNoteEditor;
```

The class name is case-sensitive. Use a specific editor selector on multi-editor
pages. Inspect through Appium execution without changing the application source.
This lookup was validated against mounted editors in Docker Chromium, Firefox,
and WebKit (six checks, including framed and unframed custom blocks), returning
the exact BlockNote editor instance. That check was not an iOS simulator run.

## 2. Boot and start automation

Set `IOS_ARTIFACTS` to a new directory under the approved temporary location, and
`SKILL_DIR` to the absolute path of this skill. Create the artifacts directory.

```sh
xcrun simctl list devices available
xcrun simctl boot "$UDID"
xcrun simctl bootstatus "$UDID" -b
open -a Simulator
```

Choose `UDID` from the list; skip `boot` if already booted. First boot can take
minutes for migration. Do not start duplicate sessions while it is initializing.

Appium **3.9.0 + XCUITest 12.22.8** worked with Xcode 26.4.1 / iOS 26.4:

```sh
export APPIUM_HOME="$IOS_ARTIFACTS/appium"
pnpm dlx appium@3.9.0 driver install xcuitest@12.22.8
pnpm dlx appium@3.9.0 server -p 4723 --log "$IOS_ARTIFACTS/appium.log"
```

Run the server in an owned terminal/background process; wait until listening.
Do not install drivers into the repository. Latest driver and Appium majors may
be incompatible (one latest driver required Appium 4 beta).

```sh
node "$SKILL_DIR/scripts/device.mjs" create "$UDID" "$PAGE_URL"
```

The helper disables hardware-keyboard connection, sets Safari's initial URL,
allows 60 seconds for webviews, and sets script timeout to 30 seconds. Session
IDs and their Appium endpoint are saved outside the checkout. Subsequent commands
use that saved endpoint, including cleanup. Set `APPIUM_URL` before creation if
not using `http://127.0.0.1:4723`. For an older session file without an endpoint,
add its confirmed URL as the `server` field before running commands.

## 3. Drive real input, inspect through WebKit

**Serialize all commands to a session.** Save the WebView context, switch to
`NATIVE_APP` for gestures, then restore it for JS/model inspection. The helper's
native commands do this with `try/finally`.

```sh
node "$SKILL_DIR/scripts/device.mjs" exec 'return !!document.querySelector(".ProseMirror")?.editor?.schema?.cached?.blockNoteEditor'
node "$SKILL_DIR/scripts/device.mjs" tap 110 145
node "$SKILL_DIR/scripts/device.mjs" native-source
node "$SKILL_DIR/scripts/device.mjs" key Return
node "$SKILL_DIR/scripts/device.mjs" type After
node "$SKILL_DIR/scripts/device.mjs" key Delete
node "$SKILL_DIR/scripts/device.mjs" state "$IOS_ARTIFACTS/after.json"
node "$SKILL_DIR/scripts/device.mjs" screenshot "$IOS_ARTIFACTS/after.png"
```

Coordinates above are illustrative, **not portable**. Inspect native source or a
screenshot for this device/page. Native coordinates are logical points, not PNG
pixels or CSS coordinates; Safari chrome changes after the keyboard opens.

A real editor tap must open the software keyboard before precise editor-API
selection setup. DOM `focus()` / synthetic clicks do not reliably open it.
`key` finds the native key's accessibility rectangle and taps its center. Use
actual Return/Delete/Shift taps for keyboard-origin investigations. `type` uses
native `/keys` for convenient text, not for proving modifier origin.
`mobile: keys` is not the iPhone substitute (documented as iPad-only).

Wait for the editor DOM reference after navigating; navigation completion is not
module readiness. Use bounded polling/observable conditions, not arbitrary sleeps.
Capture immediate and settled states when investigating delayed DOM observation.

## 4. Prove the outcome

- Compare fixed and unfixed code with identical fixtures/input. Use isolated
snapshots or read-only Docker mounts; never discard user work.
- Check the **model**, DOM, selection, native events and errors. Visible suffix text
can be stranded outside the document model; a screenshot alone misses data loss.
- Count the intended wrapper boundary, not all descendant node-view elements:
nested editable inline views legitimately introduce additional wrappers.
- Continue typing/deleting after the action, not merely checking that it did not
throw. Verify text, marks, props, nesting and caret destination.
- Include a native paragraph/plain contenteditable control where relevant. Repeat
intermittent cases and report failures as well as passes; do not declare them
fixed after a passing rerun.
- Add the cheapest red-first regression test that can express the identified
contract. Explicit Shift in a test covers the handler, not OS autocapitalization.

## 5. Record and hand off

See the reference for recording commands and review checks. Record only the
simulator display, pause so results are readable, and keep paired model traces.
Do not upload videos, create PRs, or publish issues without authorization.

Report device/iOS, exact native vs API/synthetic actions, before/after outcomes,
test results, limitations, and artifact paths. Keep unrelated failures visible.

Respect the user's lifecycle preference: retain services for ongoing testing if
requested. Once cleanup is authorized, archive useful scripts and evidence
externally first. Remove only your temporary instrumentation/files; leave
production fixes/tests. Delete your Appium session with `device.mjs close`, stop
only owned servers, and shut down the **specific** device:

```sh
xcrun simctl shutdown "$UDID"
```

Do not use `shutdown all`, kill unrelated processes, or close Simulator while
someone else is using it.
134 changes: 134 additions & 0 deletions .claude/skills/ios-simulator/references/native-debugging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Live editor inspection and native-input debugging

## Inspect the existing example without modifying it

Use the dev server's minimal example or an existing example with the relevant
schema/spec/skin. Do not create a new page or expose globals for inspection. A
minimal example does not reproduce bugs in UI/extensions it does not load.

TipTap sets `view.dom.editor` when mounting; BlockNote caches itself on that
editor's schema. Both links were source-verified and the complete lookup returned
the exact mounted BlockNote instance in Docker Chromium, Firefox, and WebKit
(six checks across framed/unframed custom blocks). This validates the DOM lookup,
not the Appium/iOS execution path. Execute this read-only snapshot in Appium:

```js
const root = document.querySelector(".ProseMirror");
const tiptap = root.editor;
const editor = tiptap.schema.cached.blockNoteEditor;
const { from, to } = tiptap.state.selection;
return JSON.stringify({
document: editor.document,
selection: { from, to },
focused: tiptap.isFocused,
dom: root.innerHTML,
});
```

The helper's `state` command performs this lookup; pass an editor selector as its
second argument on multi-editor pages. No `window.probe` is required.

When a bug needs event tracing, temporarily instrument the running page through
Appium, not source edits. Record capture/bubble
`keydown`, `keyup`, `beforeinput`, `input`: key, code, shiftKey, CapsLock state
(`getModifierState`), inputType, data, isTrusted, defaultPrevented, and composition
state. Bound traces and remove listeners on teardown. An investigation may need
render/mount counters and DOM Selection anchor/focus offsets as well.

Return JSON strings from remote execution and parse them
on the host: returning nested objects directly has sometimes produced WebDriver
window-handle serialization rather than the intended data.

For readiness after navigation, the helper supports raw W3C session requests:

```sh
node "$SKILL_DIR/scripts/device.mjs" request POST /execute/async \
'{"script":"const done = arguments[arguments.length - 1]; const end = Date.now() + 10000; function check() { if (document.querySelector(\".ProseMirror\")?.editor?.schema?.cached?.blockNoteEditor) return done(true); if (Date.now() >= end) return done(false); setTimeout(check, 50); } check();","args":[]}'
```

Require the result to be true before input. The helper sets the script timeout;
a fresh session once defaulted to zero, breaking otherwise valid async checks.

## Investigating keyboard modifiers

On one verified iOS 26.4 simulator, native Return changed the keyboard from `a`
to `A`; Return **keyup** and the next Delete **keydown** had `shiftKey: true`
with CapsLock false. Native default editing controls reproduced it. This was
software-keyboard autocapitalization, not BlockNote synthesizing Shift.

Do not assume this explains every modifier bug. Use these controls:

1. Native Return then native Delete in BlockNote; capture keydown **and keyup**.
2. Repeat in a plain contenteditable and textarea, without editor plugins.
3. Set `autocapitalize="none"` **before focus**, then repeat.
4. With default capitalization, tap the actual native Shift key off after Return,
then Delete. Read keyboard accessibility letter labels (`a` vs `A`).
5. Compare editor-command Enter as a separate control, clearly labeled non-native.

Do not inject Shift or use `/keys` in these origin controls. Explicit Shift is
appropriate later for a deterministic regression test of a missing shortcut alias.
Do not disable capitalization in production merely to hide a handler bug.

## ProseMirror diagnostics

If model and DOM selections disagree, record both before attributing the problem
to keyboard shortcuts. For plugin-handler tracing, wrap each handler preserving
`this`, arguments, return values and thrown errors. Record transaction/filter
activity and composition state, bound memory, and restore every original method.
Instrumentation must not change event dispatch behavior.

Missing `beforeinput` is not enough to identify the culprit: ProseMirror can
prevent default in a boundary-deletion fallback after every plugin declines the
event. Inspect modifier fields and actual handler outcomes first.

When isolating splitting behavior, compare native paragraph, custom React block,
editable inline node, framed/nested block, and any truly relevant chrome updates.
Check valid split boundaries, attributes/marks, suffix preservation, and subsequent
typing. Nested inline `data-node-view-*` descendants are not automatically clones.

## Setup failures and remedies

| Symptom | Check / remedy |
| ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| WDA never answers on 8100 | Inspect Appium/Xcode logs with `showXcodeLog`; wait for boot migration. Do not repeatedly create sessions while an earlier create is still running. |
| No webviews during startup | Supply a real `safariInitialUrl`, allow `webviewConnectTimeout: 60000`, confirm the server is reachable. |
| Safari tooltip/alert blocks automation | Inspect native source/screenshot and dismiss the actual first-run UI. Use `request POST /alert/accept '{}'` only when an alert is present. |
| Keyboard absent after `editor.focus()` | Tap the editor natively; verify keyboard exists, hardware keyboard is disconnected, then set the desired selection. |
| Wrong native key | Inspect source; key labels/localization and Return/Search/Go change with input type. `key` accepts the observed name. |
| Session no longer found | Create a new session. Saved JSON is not a persistent browser connection. |
| Broken content with no obvious exception | Check the actual example's CSS and viewport; don't substitute an unstyled diagnostic page for the application. |
| JS returns an opaque window handle | Serialize state to JSON in the WebView before returning it. |

## Simulator-only recordings

Record the selected device, not the desktop. The simulator must remain running:

```sh
xcrun simctl io "$UDID" recordVideo --codec=h264 --force "$IOS_ARTIFACTS/before.mp4"
```

Run recording in its own terminal/owned process, wait for “Recording started”,
perform the real reproduction in another process, leave a readable final pause,
then send **SIGINT** to that recorder and await exit so the MP4 finalizes. Do not
SIGKILL it. Repeat with the fixed variant and identical input. Check actual state
and event traces alongside video; successful-looking pixels can hide model loss.

```sh
ffprobe -v error -show_entries format=duration -of default=nw=1 "$IOS_ARTIFACTS/before.mp4"
ffmpeg -hide_banner -loglevel error -y -i "$IOS_ARTIFACTS/before.mp4" \
-vf 'fps=1,scale=280:-1,tile=4x3' -frames:v 1 "$IOS_ARTIFACTS/review.png"
```

Inspect the contact sheet and important individual frames. Unused cells on short
clips are black, not necessarily corrupt video. Never claim review from metadata
alone. Use the screen-recording skill for conversion/publication if requested.

## What a report must not imply

- A fix for native Enter does not prove an unrelated insertion freeze was fixed.
- A passing fixture does not clear a different extension, skin or node-view layout.
- One simulator/iOS version does not cover physical devices, iPad, keyboards,
dictation, composition/IME, or native selection handles that were not exercised.
- Repeated passes after one failure do not erase the unresolved observation.
- A failing existing test is not “preexisting” just because new tests pass. Compare
the relevant baseline under the same machinery and report its exact outcome.
Loading
Loading