Skip to content

artemis batch ignores the requested device serial (no --device-serial; ADB_DEVICE_SERIAL / ARTEMIS_DEVICE_ID silently dropped) and can run goals on another phone #135

Description

@LunarECL

Summary

With several devices attached, artemis batch ignores the requested device and can run its goals on a different phone. There is no --device-serial option (unlike artemis run -s), and the documented ADB_DEVICE_SERIAL / ARTEMIS_DEVICE_ID environment variables are silently ignored on both execution paths:

  • Daemon path: batch.py calls submit_batch_to_daemon() without a device_serial, even though the client function accepts one. The queue then picks a device itself via device_pool.select_device_async() — the first idle ready device (or the first ready one if all are busy).
  • Standalone path: AgentConfigBuilder.build() reads the serial from the environment into device_id but leaves device_platform as None; Agent._init_internal() then treats the config as "no device configured" and calls get_first_device(), i.e. literally the first device listed.

artemis run is not affected because it calls config.for_device(DevicePlatform.ANDROID, serial), which sets both fields.

Observed

Six devices attached over wireless adb, all idle; 10.60.0.101:5555 is the first one listed. On an idle farm both paths therefore land on 101:

$ adb devices
10.60.0.101:5555   device
10.60.0.102:5555   device
10.60.0.103:5555   device
10.60.0.105:5555   device
10.60.0.107:5555   device
10.60.0.108:5555   device

$ ADB_DEVICE_SERIAL=10.60.0.108:5555 uv run artemis batch -f goals.txt -p flash
...
Waiting for Task 1/2 ... (Session: 352be33c-...)
│ 1 │ ... │ PASS │
$ grep 'Device ID' traces/352be33c-*/stdout.log
Device ID: 10.60.0.101:5555          # <- not 108

$ ADB_DEVICE_SERIAL=10.60.0.108:5555 uv run artemis batch -f goals.txt -p flash --standalone
Device ID: 10.60.0.101:5555          # <- standalone path, same result

$ ARTEMIS_DEVICE_ID=10.60.0.108:5555 uv run artemis batch -f goals.txt -p flash --standalone
Device ID: 10.60.0.101:5555          # <- same

$ uv run artemis run "..." --profile flash -s 10.60.0.108:5555
Device ID: 10.60.0.108:5555          # <- `run` works

The batch reports PASS, so nothing signals that the goals ran on the wrong phone. With one device attached the bug is invisible.

Mechanism (at 371aa6d)

File Line What happens
artemis/interfaces/cli/commands/batch.py 108-165 batch_command options: --file/-f, --profile, --delay, --standalone, --verification-level, --explorer-pro-mode. No device option.
artemis/interfaces/cli/commands/batch.py 222-228 Daemon path: submit_batch_to_daemon(task_list, profile=..., verification_level=..., explorer_mode=..., base_url=...) — device_serial never passed.
artemis/runtime/daemon_client.py 340-344 submit_batch_to_daemon(..., device_serial: str | None = None, ...) does accept it.
apps/admin_console/services/task_queue_service.py 1083-1088 With no serial on the task, the queue calls device_pool.select_device_async().
artemis/runtime/device_pool.py 461-468 _pick_device() without a preferred serial returns the first idle ready device, else the first ready device.
artemis/interfaces/cli/commands/batch.py 57-62 Standalone path builds the config with Builders.AgentConfig.with_default_profile(profile) and never calls for_device().
artemis/sdk/builders/agent_config_builder.py 597-601 build() sets device_id = self._device_id or ARTEMIS_DEVICE_ID or ADB_DEVICE_SERIAL.
artemis/sdk/builders/agent_config_builder.py 608 ...but device_platform=self._device_platform, which is still None when the serial came from the environment.
artemis/sdk/agent.py 227-228 elif not self._config.device_id or not self._config.device_platform: → get_first_device(logger=logger). The env-provided serial is discarded.
artemis/interfaces/cli/commands/run.py 138-152 For contrast, run resolves -s / ADB_DEVICE_SERIAL and calls config.for_device(DevicePlatform.ANDROID, target_serial), so both fields are set.

.env.example:21 documents # ADB_DEVICE_SERIAL= under "Android ADB Connectivity", so users reasonably expect it to bind the batch to a device.

Steps to reproduce

  1. Attach two or more authorized devices (adb devices shows ≥2 rows in device state).
  2. Create goals.txt with any goal, e.g. Tell me what app is currently on screen (just look, do nothing).
  3. Run ADB_DEVICE_SERIAL=<second serial> uv run artemis batch -f goals.txt -p flash (with or without --standalone).
  4. Check Device ID: in traces/<session_id>/stdout.log (or the device shown in the Web Console history).

Observed: the goal runs on a device chosen by the tool (first idle device on the daemon path, first listed device standalone) instead of the requested one, and the batch still reports PASS.

Expected: the goal runs on the requested device; ideally batch accepts --device-serial/-s like run does.

Suggested fix

Verified locally (both paths now report Device ID: 10.60.0.108:5555):

  1. agent_config_builder.py build(): when device_id came from the environment and device_platform is None, default the platform to DevicePlatform.ANDROID (the same default for_device(str) already applies).
  2. batch.py: add --device-serial/-s (falling back to ADB_DEVICE_SERIAL / ARTEMIS_DEVICE_ID), pass it as device_serial= to submit_batch_to_daemon(), and call config_builder.for_device(serial) in run_batch_tasks().

Happy to open a PR with this if it matches the intended design (an alternative would be for Agent._init_internal to only fall back to get_first_device() when device_id itself is missing).

Environment

  • Commit 371aa6d (current main at the time of filing)
  • macOS 27.0 (arm64), Python 3.12.13 in the uv venv, adb 1.0.41 (37.0.1)
  • 6 physical Samsung devices (SM-F731N / SM-F721N / SM-S901W, Android 15/16) attached over wireless adb; Gemini provider

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions