Skip to content

Stop, rotation, placing the drawing by hand, and recordings that travel - #11

Merged
MAXAWER merged 2 commits into
mainfrom
fix/stop-and-rotation
Aug 27, 2026
Merged

MAXAWER merged 2 commits into
mainfrom
fix/stop-and-rotation

Conversation

@MAXAWER

@MAXAWER MAXAWER commented Aug 26, 2026 •

Copy link
Copy Markdown
Owner

All four things reported from using the application.

1. Stop did nothing, because there was nothing left to stop

Every wait inside a stroke happens on the phone — that is the whole point of
the injector — so writing a stroke costs the host almost nothing. The loop
handed the entire drawing to the device's pipe in a fraction of a second,
finished, and sat there while the phone went on drawing out of a buffer nothing
on this end can reach. should_continue was polled faithfully between strokes;
by the time anyone could reach the button there were no strokes left to skip.

The host now refuses to run more than 250 ms ahead of the device. Stopping is
immediate to a person, the backlog being at most that. It costs one round trip
per 250 ms of drawing. Two tests hold it down.

2. A phone held sideways was squeezed into a portrait frame

wm size reports the orientation the phone was manufactured in and does not
change when it is turned, so it cannot be asked. The captured frame knows, being
a picture of the screen as it is now — so the engine compares the two.

That correction feeds the placement overlay and the drawing itself. This was
worse than cosmetic: a drawing placed from the natural size lands rotated on a
phone in landscape.

3. The drawing can be placed by hand

Drag it over the live view, wheel to resize, Shift and wheel to turn,
double-click to fit again. Placement is held in fractions of the screen and in
degrees rather than pixels, so it survives the phone being turned and means the
same thing on a different device. A scale of one is "as large as it goes with a
margin", whatever the drawing and whatever the phone.

Rotation turns about the drawing's own middle and fits the rotated bounding
box — without that a drawing turned forty-five degrees quietly grows past the
edges of the screen. The front end coalesces the gestures rather than asking for
an overlay per mouse report.

The fixed margin slider is gone; it was the least interesting control in the
window and it was in the way.

4. Recordings that replay on a different phone

The old format was faithful and useless anywhere but the phone it came from,
twice over:

  • the numbers are digitizer coordinates, whose range has little to do with
    any display — which is why replaying one elsewhere was refused rather than
    attempted;
  • replaying them means writing to /dev/input, which every recent Pixel
    denies to the shell — so it could not be replayed on the phone that made
    it either.

mthread.gestures decodes those events into strokes of (time, x, y) with the
coordinates as fractions of the screen, and Device.play_gestures scales them
to whatever screen it is handed and plays them through the injector. A fraction
means the same thing everywhere, so the file travels.

Nine tests cover the decoding: one finger, two fingers, a finger held rather
than moved, a panel whose axes are swapped, and the three ways a file can be the
wrong version. Device.model was added to label a recording with the phone it
came from.

The window gets Record, Play, Open and Save alongside the drawing controls.

Checked

  • 177 tests pass, nine of them new.
  • The WinUI project builds clean, no warnings.
  • Replay was exercised against a real Pixel 8 Pro with a synthetic recording:
    the same file scales correctly to different screen sizes and plays.

Known, and honestly not finished

Replay timing is not yet faithful. A two-second recording takes longer than
two seconds to play. Some of that is the injector starting and stopping, which
is a fixed cost either way, but it has not been measured properly and it should
be right before anyone leans on this for a regression pass.

Recording itself has not been exercised end to end, because injected touches
do not appear in /dev/input — verifying it needs a human finger on the glass.
The decoder is covered by tests against captured event sequences; the path from
getevent to those events is the part still unproven in this change.

🤖 Generated with Claude Code

MAXAWER and others added 2 commits August 26, 2026 23:15
**Stop did nothing, and the reason is that there was nothing left to stop.**
Every wait inside a stroke happens on the phone - that is the entire point of
the injector - so writing a stroke costs the host almost nothing. The loop
therefore handed the whole drawing to the device's pipe in a fraction of a
second, finished, and sat there while the phone drew on out of a buffer that
nothing on this end can reach. `should_continue` was being polled faithfully
between strokes; by the time anyone could press the button there were no
strokes left to skip.

The host now refuses to run more than a quarter of a second ahead of the
device: it counts the milliseconds it has queued and waits for the injector to
catch up when it goes over. Stopping is then immediate to a person, the backlog
being at most that. It costs a round trip per 250 ms of drawing - about eleven
of them across a three-second drawing, which is lost in the noise.

Two tests hold it down: one that the whole drawing is not queued at once, and
one that stopping leaves the rest unsent.

**A phone held sideways was squeezed into a portrait frame.** `wm size` reports
the orientation the phone was manufactured in and does not change when it is
turned, so it cannot be asked. The captured frame knows, being a picture of the
screen as it is now, so the engine compares the two: if the frame is the other
way round from the natural size, so is the phone. That correction now feeds the
placement overlay and the drawing itself, not just the preview - a drawing
placed from the natural size lands rotated on a phone in landscape.

The window reshapes its phone from every frame whose proportions differ from
the last, so turning the device mid-session is followed rather than ignored.

Also: the device dropdown showed "Looking for a device…" over a device that was
connected and mirroring, because setting SelectedIndex on a ComboBox whose
items were added moments earlier does not reliably take. Selecting the item
does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
**Placing by hand.** The drawing was always fitted to the screen with a fixed
margin, and the margin was a slider nobody wanted. Now it is dragged: pull it
about over the live view, wheel to resize, Shift and wheel to turn,
double-click to fit again. Placement is held in fractions of the screen and in
degrees rather than pixels, so it survives the phone being turned and means the
same thing on a different device; a scale of one is "as large as it goes with a
margin", whatever the drawing and whatever the phone, so the number a person
sees is a multiplier on something recognisable. Rotation turns about the
drawing's own middle and fits the rotated bounding box - without that a drawing
turned forty-five degrees quietly grows past the edges of the screen.

The front end coalesces the gestures rather than asking for an overlay per
mouse report.

**Recordings that replay somewhere else.** The old format was faithful and
useless anywhere but the phone it came from, twice over: the numbers are
digitizer coordinates, whose range has little to do with any display, and
replaying them means writing to /dev/input, which every recent Pixel denies to
the shell. So a recording could not be replayed on a different phone, and on a
current one it could not be replayed at all.

mthread.gestures decodes those events into strokes of (time, x, y) with the
coordinates as fractions of the screen, and Device.play_gestures scales them to
whatever screen it is handed and plays them through the injector. A fraction
means the same thing everywhere, so the file travels. Nine tests cover the
decoding: one finger, two fingers, a finger that is held rather than moved, a
panel whose axes are swapped, and the three ways a file can be the wrong
version.

The window gets Record, Play, Open and Save alongside the drawing controls.

Known and not yet chased: replaying a two-second recording takes longer than
two seconds. Some of that is the injector starting and stopping, which is a
fixed cost either way, but it has not been measured properly and the timing
should be faithful before anyone relies on it for a regression pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MAXAWER MAXAWER changed the title Make Stop stop, and let the phone be held sideways Stop, rotation, placing the drawing by hand, and recordings that travel Aug 26, 2026
@MAXAWER
MAXAWER merged commit 5e51352 into main Aug 27, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant