Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,28 @@

## Unreleased

- **Layers.** Several pictures can be loaded and arranged against each other
before anything is drawn, each with its own placement and its own tracer
settings. Hide one, reorder them, remove one; what is drawn is every visible
layer in order.
- **An eraser.** Drag across the live view with the eraser on and the strokes
under the cursor come out. Which strokes are near enough is decided after
placement, because the cursor is over a picture of the phone rather than over
the tracer's coordinate space. Re-tracing forgets erasures rather than
misapplying them: stroke five of one tracing is not stroke five of the next.
- **Flip, and buttons for the transforms that are not gestures.** `Flip`,
`Fit` and `Undo erase` sit under the phone; `Placement.mirrored` flips rather
than sets, so flipping twice is not flipping.
- **A screenshot taken by hand can stand in for capture.** When ADB capture
fails there is otherwise nothing to place a drawing against. A still picture
does not update, but its proportions are the ones that matter, and placement
now works with no phone attached at all - so a drawing can be arranged before
the cable is anywhere near.
- **Shapes and text from the command line**, with nothing to prepare:
`mthread shape heart`, `mthread shape star --points 7 --rotate 20`,
`mthread text "hello" --font arial.ttf`. Seven shapes, and text in any font
the machine has.

- **Place the drawing by hand.** Drag it over the live view, wheel to resize,
Shift and wheel to turn, double-click to fit again. `Placement` holds the
position in fractions of the screen and degrees, so it survives the phone
Expand Down
124 changes: 104 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,29 @@ does not take. Windows SmartScreen says "unknown publisher" once — *More info*
1. **Turn on USB debugging** on the phone: Settings → About phone → tap *Build
number* seven times → Developer options → *USB debugging*.
2. **Plug it in.** Or go wireless: `adb connect 192.168.1.42:5555`.
3. **Open MThread Draw** → *Connect device* → *Capture screen* → *Load image* → drag
it over the phone preview → **START DRAWING**.
3. **Open MThread Draw.** It finds the phone by itself, shows its screen live,
and lays the drawing over exactly where it will land. Drag to move it, wheel
to resize, Shift and wheel to turn — then **START DRAWING**.

No GUI, no clicking:
### What you can do to a drawing before it is drawn

| | |
|---|---|
| **Move, resize, turn, flip** | Drag it over the live view; wheel resizes, Shift and wheel turns, and `Flip` mirrors it. `Fit` puts it back. |
| **Layers** | Load several pictures and arrange them against each other. Each keeps its own position and its own tracer settings; hide one, reorder them, remove one. |
| **Erase single strokes** | Turn the eraser on and drag across the lines you do not want. `Undo erase` brings them back. |
| **Change the tracing** | The *detail* slider re-traces what is already loaded. There is no need to open the file again. |
| **No live view?** | If capture fails, *Use a screenshot…* takes a picture you copied off the phone by hand. It does not update, but its proportions are what placement needs. |

Nothing to install and nothing to draw with a mouse:

```bash
mthread shape heart # a heart, fitted to the screen
mthread shape star --points 7 # a seven-pointed star, turned however you like
mthread text "hello" --y 0.35 # words, in any font this machine has
```

Or from code:

```python
from mthread import Device
Expand Down Expand Up @@ -211,14 +230,24 @@ This repository is two things:

## Record and replay gestures

Press record, do something on the phone, press stop. You get a JSON file with
every touch event and its timing. Replay it whenever you want, at whatever speed.
Press record, do something on the phone, press stop. You get a JSON file of
strokes and their timing, and you can replay it whenever you want, at whatever
speed.

```bash
mthread record -o login.json # do the thing on the phone, press Enter
mthread play login.json --speed 2 --repeat 5
```

**The file replays on a different phone.** It stores each touch as a fraction of
the screen rather than as a digitizer coordinate, so playing it back on another
device scales it to that screen. This was not true of the old format, and could
not be: a digitizer's range has little to do with any display, which is why
replaying one elsewhere used to be refused rather than attempted. Nor could the
old format be replayed on a *current* phone at all, since it went through
`/dev/input`, which every recent Pixel denies to the shell. Playback now goes
through the injector, like drawing.

Useful for regression passes, for reproducing a bug reliably, or for any
repetitive tapping you would rather not do by hand.

Expand Down Expand Up @@ -261,12 +290,26 @@ python tools/build_app.py --dmg # macOS
```bash
mthread devices # what is attached
mthread info # screen size and digitizer ranges

mthread shape heart # draw a shape, nothing to prepare
mthread shape star --points 7 --rotate 20
mthread shape circle --scale 0.4 --x 0.25 --y 0.3
mthread text "hello world" # any font the machine has
mthread text "signed" --font arial.ttf --scale 0.5 --y 0.8

mthread record -o session.json # record until Enter
mthread record -o session.json -d 30 # record for 30 seconds
mthread play session.json # replay once
mthread play session.json --speed 0.5 --repeat 3
```

Every drawing command takes the same placement options: `--scale`, `--rotate`,
`--flip-x`, `--flip-y`, `--x`, `--y`, `--margin`, and `--speed`/`--human` for
how it is drawn. The shapes are `heart`, `star`, `circle`, `square`, `polygon`,
`spiral` and `wave`.

Text is rendered with a real font and then traced, which is why any font on the
machine works and why the letters come out as outlines: a filled glyph is a
shape with an inside and an outside, and this draws with one finger.

## Library

```python
Expand Down Expand Up @@ -323,13 +366,18 @@ one of them, while leaving genuine closed shapes like circles intact.

## Known limits

- **Recordings are not portable between phones.** They contain raw digitizer
coordinates, so replaying a recording made on a different panel is refused
rather than silently misfiring.
- **Rotation is not handled.** Record and replay in the same orientation.
- **Some devices expose no touchscreen usable by `sendevent`.** Drawing then fails
with an explicit error instead of a wrong result; `Device.swipe()` still works,
and an automatic fallback is an open task.
- **Recordings are portable now**, holding fractions of a screen rather than
digitizer coordinates. A file written by an older version is not, and says so
rather than misfiring.
- **A recording does not know which way up the phone was.** Drawing follows the
orientation - the engine reads it from the captured frame, since `wm size` does
not change when a phone is turned - but a recording holds fractions of the
screen it was made on, so replaying a portrait one in landscape lands sideways.
Replay in the orientation you recorded in.
- **Replay timing carries a fixed overhead.** Starting and stopping the on-device
injector costs a second or two, so a two-second recording takes longer than two
seconds. The strokes and the gaps between them are faithful; the total is not,
and that matters if you are timing something.
- **`mthread info` is the first thing to check** when touches land in the wrong
place.

Expand All @@ -343,13 +391,15 @@ are the easiest way in. Things worth doing:

- SVG input, so line art skips edge detection entirely.
- Auto-detect swapped X/Y axes (the `swap_xy` flag exists but nothing sets it).
- Rotation-aware coordinate mapping.
- An automatic `input swipe` fallback when raw touch is unavailable.
- Rotation in recordings: store which way up the phone was, and turn a replay to
match. Drawing already follows the orientation; replay does not.
- Take the fixed second or two out of replay by keeping the injector alive
between runs.
- Trim recordings visually in the app; cut dead time at the start and end.
- Assertions during replay — wait for a screenshot to match before continuing,
which is what turns this into a real test runner.
- Skeletonise edges instead of halving contours, for cleaner line art.
- Pressure-sensitive strokes from image darkness.
- Shed OpenCV. Five of its functions are used and it is half the download.

---

Expand Down Expand Up @@ -412,8 +462,19 @@ explained in plain English and Russian in **[TERMS.md](TERMS.md)**.
1. **Включить отладку по USB**: Настройки → О телефоне → семь раз по «Номер
сборки» → Для разработчиков → Отладка по USB.
2. **Подключить телефон.** Или по Wi-Fi: `adb connect 192.168.1.42:5555`.
3. **Открыть MThread Draw** → *Connect device* → *Capture screen* → *Load image* →
перетащить картинку на превью экрана → **START DRAWING**.
3. **Открыть MThread Draw.** Телефон находится сам, его экран показывается
живьём, а рисунок ложится поверх того места, куда он попадёт. Тащите мышью,
колесо меняет размер, Shift с колесом поворачивает — затем **START DRAWING**.

Что можно сделать с рисунком до отрисовки:

| | |
|---|---|
| **Двигать, размер, поворот, отражение** | Перетаскивание по живому экрану; колесо — размер, Shift с колесом — поворот, `Flip` — отражение, `Fit` — вписать заново. |
| **Слои** | Несколько картинок, расставленных друг относительно друга. У каждой своё положение и свои настройки трассировки; слой можно спрятать, переставить, удалить. |
| **Ластик** | Включите ластик и проведите по ненужным линиям. `Undo erase` возвращает их. |
| **Пересчёт** | Ползунок детализации перетрассирует уже загруженное. Открывать файл заново не нужно. |
| **Нет живого экрана?** | Если захват не работает, *Use a screenshot…* берёт снимок, снятый на телефоне вручную. Он не обновляется, но его пропорции — это то, что нужно для размещения. |

<a name="из-исходников"></a>

Expand Down Expand Up @@ -507,16 +568,39 @@ python tools/fetch_platform_tools.py # ~7 МБ, прямо от Google
```bash
mthread devices # какие устройства подключены
mthread info # разрешение экрана и диапазоны тачскрина

mthread shape heart # нарисовать фигуру, ничего не готовя
mthread shape star --points 7 --rotate 20
mthread text "привет" # текст любым шрифтом, что есть в системе

mthread record -o session.json # запись до нажатия Enter
mthread play session.json --speed 2 --repeat 5
```

У всех команд рисования одни и те же параметры размещения: `--scale`,
`--rotate`, `--flip-x`, `--flip-y`, `--x`, `--y`, `--margin`, а также
`--speed`/`--human` — как рисовать. Фигуры: `heart`, `star`, `circle`,
`square`, `polygon`, `spiral`, `wave`.

Текст рисуется настоящим шрифтом и затем трассируется — поэтому доступен любой
шрифт системы, и поэтому буквы выходят контурами: залитая глифа это фигура с
внутренней и внешней границей, а здесь рисует один палец.

Если `adb` установлен в нестандартное место — укажите путь в переменной
окружения `ADB_PATH`.

## Ограничения

- Записи **не переносятся между разными телефонами** — внутри сырые координаты
- Записи **переносятся между телефонами**: внутри доли экрана, а не сырые
координаты дигитайзера. Файлы старого формата не переносятся и честно об этом
сообщают, а не рисуют мимо.
- **Запись не знает, как был повёрнут телефон.** Рисование ориентацию учитывает —
движок читает её из снятого кадра, поскольку `wm size` при повороте не
меняется, — но запись хранит доли того экрана, на котором сделана, поэтому
портретная запись в горизонтальной ориентации ляжет набок.
- **У воспроизведения есть постоянная накладная стоимость** — секунда-две на
запуск и остановку инжектора. Штрихи и паузы между ними точны, а общая
длительность нет. Старые записи — сырые координаты
тачскрина. Попытка воспроизвести запись на панели другого размера будет
отклонена, а не выполнена криво.
- Поворот экрана не учитывается: записывайте и воспроизводите в одной ориентации.
Expand Down
6 changes: 0 additions & 6 deletions examples/CREDITS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,3 @@ every line is real and none of them tells you what you are looking at.
The three from Wikimedia Commons arrived there through its Unsplash import and
carry the CC0 waiver on their file pages. The Met's photograph is part of that
museum's Open Access release, also CC0.

## castle.png

Drawn from scratch by `tools/make_sample.py`. No third-party rights, and useful
as a test case precisely because it is synthetic: the tracer's output can be
compared against lines whose true positions are known.
Binary file removed examples/castle.png
Binary file not shown.
Loading
Loading