Repository navigation
Stop, rotation, placing the drawing by hand, and recordings that travel - #11
Merged
Merged
Conversation
**Stop did nothing, and the reason is that there was nothing left to stop.** Every wait inside a stroke happens on the phone - that is the entire point of the injector - so writing a stroke costs the host almost nothing. The loop therefore handed the whole drawing to the device's pipe in a fraction of a second, finished, and sat there while the phone drew on out of a buffer that nothing on this end can reach. `should_continue` was being polled faithfully between strokes; by the time anyone could press the button there were no strokes left to skip. The host now refuses to run more than a quarter of a second ahead of the device: it counts the milliseconds it has queued and waits for the injector to catch up when it goes over. Stopping is then immediate to a person, the backlog being at most that. It costs a round trip per 250 ms of drawing - about eleven of them across a three-second drawing, which is lost in the noise. Two tests hold it down: one that the whole drawing is not queued at once, and one that stopping leaves the rest unsent. **A phone held sideways was squeezed into a portrait frame.** `wm size` reports the orientation the phone was manufactured in and does not change when it is turned, so it cannot be asked. The captured frame knows, being a picture of the screen as it is now, so the engine compares the two: if the frame is the other way round from the natural size, so is the phone. That correction now feeds the placement overlay and the drawing itself, not just the preview - a drawing placed from the natural size lands rotated on a phone in landscape. The window reshapes its phone from every frame whose proportions differ from the last, so turning the device mid-session is followed rather than ignored. Also: the device dropdown showed "Looking for a device…" over a device that was connected and mirroring, because setting SelectedIndex on a ComboBox whose items were added moments earlier does not reliably take. Selecting the item does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
**Placing by hand.** The drawing was always fitted to the screen with a fixed margin, and the margin was a slider nobody wanted. Now it is dragged: pull it about over the live view, wheel to resize, Shift and wheel to turn, double-click to fit again. Placement is held in fractions of the screen and in degrees rather than pixels, so it survives the phone being turned and means the same thing on a different device; a scale of one is "as large as it goes with a margin", whatever the drawing and whatever the phone, so the number a person sees is a multiplier on something recognisable. Rotation turns about the drawing's own middle and fits the rotated bounding box - without that a drawing turned forty-five degrees quietly grows past the edges of the screen. The front end coalesces the gestures rather than asking for an overlay per mouse report. **Recordings that replay somewhere else.** The old format was faithful and useless anywhere but the phone it came from, twice over: the numbers are digitizer coordinates, whose range has little to do with any display, and replaying them means writing to /dev/input, which every recent Pixel denies to the shell. So a recording could not be replayed on a different phone, and on a current one it could not be replayed at all. mthread.gestures decodes those events into strokes of (time, x, y) with the coordinates as fractions of the screen, and Device.play_gestures scales them to whatever screen it is handed and plays them through the injector. A fraction means the same thing everywhere, so the file travels. Nine tests cover the decoding: one finger, two fingers, a finger that is held rather than moved, a panel whose axes are swapped, and the three ways a file can be the wrong version. The window gets Record, Play, Open and Save alongside the drawing controls. Known and not yet chased: replaying a two-second recording takes longer than two seconds. Some of that is the injector starting and stopping, which is a fixed cost either way, but it has not been measured properly and the timing should be faithful before anyone relies on it for a regression pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
All four things reported from using the application.
1. Stop did nothing, because there was nothing left to stop
Every wait inside a stroke happens on the phone — that is the whole point of
the injector — so writing a stroke costs the host almost nothing. The loop
handed the entire drawing to the device's pipe in a fraction of a second,
finished, and sat there while the phone went on drawing out of a buffer nothing
on this end can reach.
should_continuewas polled faithfully between strokes;by the time anyone could reach the button there were no strokes left to skip.
The host now refuses to run more than 250 ms ahead of the device. Stopping is
immediate to a person, the backlog being at most that. It costs one round trip
per 250 ms of drawing. Two tests hold it down.
2. A phone held sideways was squeezed into a portrait frame
wm sizereports the orientation the phone was manufactured in and does notchange when it is turned, so it cannot be asked. The captured frame knows, being
a picture of the screen as it is now — so the engine compares the two.
That correction feeds the placement overlay and the drawing itself. This was
worse than cosmetic: a drawing placed from the natural size lands rotated on a
phone in landscape.
3. The drawing can be placed by hand
Drag it over the live view, wheel to resize, Shift and wheel to turn,
double-click to fit again.
Placementis held in fractions of the screen and indegrees rather than pixels, so it survives the phone being turned and means the
same thing on a different device. A scale of one is "as large as it goes with a
margin", whatever the drawing and whatever the phone.
Rotation turns about the drawing's own middle and fits the rotated bounding
box — without that a drawing turned forty-five degrees quietly grows past the
edges of the screen. The front end coalesces the gestures rather than asking for
an overlay per mouse report.
The fixed margin slider is gone; it was the least interesting control in the
window and it was in the way.
4. Recordings that replay on a different phone
The old format was faithful and useless anywhere but the phone it came from,
twice over:
any display — which is why replaying one elsewhere was refused rather than
attempted;
/dev/input, which every recent Pixeldenies to the shell — so it could not be replayed on the phone that made
it either.
mthread.gesturesdecodes those events into strokes of(time, x, y)with thecoordinates as fractions of the screen, and
Device.play_gesturesscales themto whatever screen it is handed and plays them through the injector. A fraction
means the same thing everywhere, so the file travels.
Nine tests cover the decoding: one finger, two fingers, a finger held rather
than moved, a panel whose axes are swapped, and the three ways a file can be the
wrong version.
Device.modelwas added to label a recording with the phone itcame from.
The window gets Record, Play, Open and Save alongside the drawing controls.
Checked
the same file scales correctly to different screen sizes and plays.
Known, and honestly not finished
Replay timing is not yet faithful. A two-second recording takes longer than
two seconds to play. Some of that is the injector starting and stopping, which
is a fixed cost either way, but it has not been measured properly and it should
be right before anyone leans on this for a regression pass.
Recording itself has not been exercised end to end, because injected touches
do not appear in
/dev/input— verifying it needs a human finger on the glass.The decoder is covered by tests against captured event sequences; the path from
geteventto those events is the part still unproven in this change.🤖 Generated with Claude Code