Repository navigation
fix(ios): touch-control UX fixes, a Return key and 8× local zoom - #27
Draft
raiseCatError wants to merge 12 commits into
Draft
raiseCatError wants to merge 12 commits into
raiseCatError wants to merge 12 commits into
Conversation
…ves it Avoid Notch lifts the iPhone tray off the home indicator (portrait) or the notch (landscape), but the palette was avoided on its own and stayed where the tray used to be, covering half the tray on every Face ID iPhone. Carry the palette by the tray's offset, as the reserved-region pass already does.
… and Switch Control The Main Tray, keyboard bar and iPad chips are drawn views whose taps arrive through one hit-testing gesture, so assistive technologies saw labelled text with no button trait and no action. Give each control the button trait and its tap action directly (a modifier latches, as on a quick tap); palette keys fire their shortcut. The Explore Demo card wrapped its button in a separate accessibility element, hiding the button trait (and the button from UI tests). Adds a Demo Mode UI tour that captures the controls per device and orientation and checks the tray controls are buttons.
Nine 40pt chips need 396pt, wider than a 390pt or 375pt iPhone in portrait and taller than it in landscape, so Command and Settings ran off the screen and out of the tray's touch area (the geometry clamps the touch frame to the screen). Shrink the chips just enough to fit, never below 30pt; trays that already fit are unchanged.
Demo Mode passed the keyboard's own frame where the control overlay expects the visible area above it, so the modifier/esc/tab bar was laid out behind the keyboard and the trays disappeared while typing. Hand the overlay the same geometry a live session does.
A fresh iPad install (and Demo Mode) now opens on the radial Custom layout around a bottom-right Settings anchor instead of Strip. Strip and Overlay remain in Settings > Controls > Control Layout, and saved preferences are never replaced. iPhone is unchanged.
At accessibility text sizes the fixed-height banner's text overflowed and covered Exit Demo, and the status chips wrapped mid-word over the desktop. Cap the banner at xxLarge and the chips at xxxLarge.
Feature inventory, findings with before/after simulator screenshots, the Control Surface assessment and a hardware-only test checklist.
Return (HID usage 40, no modifiers of its own) is now a built-in Function Tray action in its own group after Undo/Redo, a key in the keyboard bar between tab and Hide Keyboard, and part of the radial iPad layout's Two-Hand Assist arc. A latched modifier is held on the Mac and merges into the press, so ⌘ then ↵ sends ⌘↵. Fresh installs show it; saved Function Tray profiles keep their layout and gain Return hidden, and saved Custom layouts keep their own assist arc. The bar's keys shrink to fit eight (35 pt on 375 pt, 37 pt on 390 pt). The bar's Return is sent only while input is allowed; the Mac enforces the grant too. Function Tray chips and the iPad Wheel palette's wedges now perform their action for VoiceOver, Voice Control and Switch Control instead of only carrying a label; the iPad chip's ignored children had dropped its tap.
…creen Manual viewport zoom now goes to 8× instead of 3×. It stays a layer transform: stream resolution, decoding and networking are unchanged, and Zoom While Typing keeps its own 1.5× limit. Pan limits came from the rotated picture's bounding box, so a tilted picture dragged along a diagonal could leave the screen entirely, more so the further it was zoomed. The clamp now projects the view's center onto the picture in the picture's own rotated frame, which is identical at 0°, 90° and 180°. An in-range state is kept exactly, so restore and relayout never nudge it, and the at-rest snap is skipped when it would step the center off a tilted picture's edge. Tests cover 1×, 3×, 4×, 6× and 8×, continuous and anchored pinching, extreme and non-finite pans, an angle grid with idempotence, reset and restore, pointer round trips, and orientation, window and keyboard changes; a Demo Mode UI tour pinches to 8×, rotates and pans.
Adds findings 8 (Return) and 9 (8× zoom and the rotated-pan clamp) to the control surface review, with iPhone SE, iPhone 17e and iPad Pro 13 screenshots and new hardware checklist items. A new performance audit lists the existing telemetry, software-only measurements (local gesture math ~0.13 µs per update at any zoom), code-level findings, what needs a device, and candidates for a separate performance PR.
raiseCatError
changed the title
The Settings walk in DemoModeSmokeTests fails in landscape on a 375 pt iPhone SE on the shipped base and on this branch alike (portrait 3/3, landscape 0/3 each, same list state at failure): the Gestures row sits under the pinned Search field and the test's single swipe flings past it. Also states that the Wheel and chip accessibility actions are checked in source only.
This was referenced Oct 26, 2026
Saving any receiver setting also writes the Function Tray profiles, so a v1.20.0 tray nobody customized was stored and, on upgrade, treated like a customized one: Return arrived hidden. A saved tray that still matches v1.20.0's starting tray exactly (action IDs, order, visibility and groups, with no renamed or restyled built-in) now takes today's default, with Return shown. Any other tray keeps its layout and gains Return hidden, as before. An upgraded tray contains Return, so it never matches again: the upgrade is idempotent and a later hide sticks. Tested against a literal v1.20.0 stored profile in every slot.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
iPhone/iPad touch-control fixes found by a simulator UX review of the shipped v1.20.0 iOS source. The branch is based on
release/app-store-rc(cbbad7f), not on #24, #25 or #26. No protocol, networking, pairing, streaming or security changes. Every change is iOS UI code or the pure layout and preferences helpers inShared/.The full write-up (feature inventory, findings, Control Surface assessment, hardware checklist):
ux-review/ios-control-surface.mdWhat changes
xxLargeand the chips atxxxLarge.Tests
ReceiverControlsTourTests: a Demo Mode tour plustestTrayControlsAreAccessibilityButtons. On the shipped build that test fails with "Command is not exposed as a button"; on this branch it passes.Follow-up: Return and 8× zoom
Commits
af961c5(Return) and1cc4930(8× zoom and the rotated-pan clamp), plus docs (7410a74,2ea0025). New tests: Return's default group, single plain press, saved-profile upgrade (an untouched v1.20.0 tray shows Return in every slot; a customized one keeps its layout and gains Return hidden; idempotent; a later hide sticks; commitf81898b), keyboard-bar order and fit at 375/390/402/440 pt, HID 40 →kVK_Returnwith latched ⌘/⇧, the radial assist arc; for zoom, 1×/3×/4×/6×/8×, continuous anchored pinching, extreme and NaN/∞ pans, a 7.5° angle grid with exact idempotence, the tilted-diagonal regression, reset/restore, pointer round trips, and orientation/window/keyboard rebasing. UI teststestReturnSendsAPressWithKeyboardClosedAndOpen(taps Return in the tray and the keyboard bar and checks Demo received it) andtestHighMagnificationTour.Verified (iOS 26.5 simulators, Xcode 27.1 beta, one simulator at a time):
OpenSidecarMacfull unit suite: 1,875 tests, 0 failures.ReceiverControlsTourTests(5 tests incl. the new two): pass on iPad Pro 13, iPhone 17e (390 pt), iPhone SE 3rd gen (375 pt) and iPhone 17 (402 pt).DemoModeSmokeTests: pass on iPad Pro 13, iPhone 17 and iPhone 17e. On 17e, portrait failed once on the first launch after boot (the Explore Demo tap didn't register) and passed on rerun. On iPhone SE (375 pt), landscape can't walk Settings. This predates the branch: it's a test scrolling issue. The Gestures row sits under the pinned Search field, so the test's single swipe flings the list past it. With the same test on the same iOS 26.5 SE simulator, the shipped basecbbad7fand this head behaved identically: portrait 3/3 pass, landscape 0/3, the same list state at failure. For that comparison both located Explore Demo by identifier; on the base the unchanged test can't find the Explore Demo button at all (finding 5). Test left unchanged; details in the review doc.ux-review/ios-control-surface-performance.md. Local gesture math costs ~0.13 µs per update at any zoom; whether iOS composites a scaled/rotated video layer differently needs hardware. No streaming changes.Verification (first round)
OpenSidecarMacfull unit suite: 1,854 tests, 0 failures (run before the Dynamic Type commit, which is iOS view code only).DemoModeSmokeTestspass in portrait and landscape; on iOS 27.1 they couldn't find Explore Demo before the card fix.git diff --checkis clean. Only the final tree was built; the intermediate commits weren't built one by one.Notes