Device GUI Verification

Device & GUI verification — 🚧 just started

Source: docs/DEVICE_GUI_VERIFICATION_ROADMAP.md. ← Overview

A follow-up to the language roadmap: the targets left out of scope there — mobile, embedded, native desktop GUI. Each slice is "same pipeline, new adapters".

Legend: ✅ done · 🟢 partial · 🚧 in progress · ⬜ planned

Verification levels

Level	Meaning
L0	Builds cleanly
L1	Host-side unit tests pass
L2	Runs in emulator/simulator, assertions pass
L3	Screenshot verification (agent visually confirms UI)
L4	Scripted interaction (agent drives input and asserts)
L5	On physical device (out of scope except serial harness)

Current state

Target	Detection	Test discovery	Interaction	Best level
Android (Kotlin/Java)	✅	🟢 instrumented tests (#422)	❌	L1→L2
Flutter	✅	🟡 flutter test	❌	L1
iOS (Swift)	🟡 Package.swift only	🟡 swift test	❌	L1
React Native	✅	🟡 jest	❌	L1
Embedded (PlatformIO/Zephyr/ESP-IDF)	❌	❌	❌	L0
Desktop GUI (Qt/GTK/native)	✅	✅ ctest	❌	L1
Tauri	✅	🟡 jest/vitest	❌	L1

---

Tracks

Track A — Android (highest value per effort)

• 🟢 A1 instrumented test discovery: Gradle Managed Virtual Devices + connectedAndroidTest fallback (#422) → L2.
• ⬜ A2 adb tools (android_screenshot/android_input/android_logcat) → L3/L4.
• ⬜ A3 Maestro flows (YAML, LLM-friendly, kept as regression tests).

Track B — Flutter

• ⬜ B1 flutter drive / integration_test + emulator lifecycle → L2.
• ⬜ B2 reuse A2 adb tools → L3/L4.

Track C — Embedded (emulation; hardware never assumed)

Honest ceiling without hardware is L2. ⬜ C1 detect+allowlist+discovery (PlatformIO pio test, Zephyr west twister -p qemu, ESP-IDF); ⬜ C2 Renode harness; ⬜ C3 serial harness run_serial_session (the only sanctioned L5); ⬜ C4 QA honesty ("verified in QEMU, not on hardware").

Track D — Native desktop GUI (screenshots first)

• ⬜ D1 launch + screenshot (L3): macOS screencapture, Windows System.Drawing, Linux grim/import.
• ⬜ D2 accessibility-tree read (Windows UIAutomation → Linux AT-SPI → macOS AXUIElement).
• ⬜ D3 input injection (L4).
• ⬜ D4 Tauri fast-path via tauri-driver (WebDriver), independent of D1–D3.

Track E — iOS (last: macOS-only runners)

• ⬜ E1 detect *.xcodeproj/*.xcworkspace + xcodebuild test on the simulator (is_macos()).
• ⬜ E2 screenshots simctl io booted screenshot (L3).
• ⬜ E3 input via idb or Maestro.

PR order (impact/effort)

1. A1 ✅ → 2. A2 adb tools → 3. C1 embedded detect → 4. C3 serial → 5. B1 flutter drive → 6. D1 desktop_screenshot → 7. C2 Renode → 8. E1/E2 iOS → 9. Maestro (A3/E3) → 10. D2/D3 → 11. D4 tauri-driver.

Non-goals

Device farms (Firebase/BrowserStack), hardware-in-the-loop beyond the serial harness, commercial GUI automation (Squish/TestComplete).