diff --git a/examples/egocap/capture_server/CALIBRATION.md b/examples/egocap/capture_server/CALIBRATION.md new file mode 100644 index 0000000000..048c6f0fb0 --- /dev/null +++ b/examples/egocap/capture_server/CALIBRATION.md @@ -0,0 +1,123 @@ + + +# Manus glove calibration in CaptureServer + +CaptureServer can run the Manus SDK's glove calibration on its host, on the same WebSocket connection as +capture, and write the resulting MCAL files into the server's MCAL directory. The SDK work happens in the separate `manus_glove_calibrator` process (see +`src/plugins/manus/calibration/README.md`); CaptureServer supervises it and stores its output. + +This is vendor glove calibration only. It needs no camera, OpenXR session or capture session. + +## Setup + +```sh +capture-server --manus-calibration-dir /var/lib/egocap/mcal \ + [--manus-calibrator /path/to/manus_glove_calibrator] +``` + +- `--manus-calibration-dir` enables the feature. New files are written there as plain `.mcal` files. It is the + same folder the server offers its MCAL files from, so a file written here is found wherever that folder is + read. It is created on first use. +- The worker is the explicit `--manus-calibrator` path (an unusable path is a startup error), else + `$ISAAC_ROOT/install/plugins/manus/manus_glove_calibrator`, where the Manus component installs it. When no + worker is found, ordinary capture is unaffected and only a calibration request is rejected with + `INVALID_CONFIG` saying so. +- Stop every other Manus consumer first: the dongles are not shared. + +## Ownership and admission + +CaptureServer admits one session at a time, from acceptance through cleanup. A calibration and a capture +session are mutually exclusive: a request for one while the other is active (starting, running, or stopping) +is rejected `BUSY` and never interrupts it. The same single owner connection and single outbound writer are +used. Capture `Stop` addresses capture sessions and `StopCalibration` addresses calibration sessions; neither +stops the other kind. + +The station stays unavailable (`BUSY` for both) while a worker could not be reaped, because it may still hold +the gloves. Restart the server once the process has exited. A terminal error is not evidence the hardware +was released. + +If the owner connection closes or the server shuts down, the calibration is cancelled with a bounded cleanup. +There is no reconnect grace period. + +## Python API + +```python +client = CaptureClient("ws://host:8082") +await client.start_calibration( + sides=(HandSide.LEFT, HandSide.RIGHT), + on_started=..., + on_status=..., + on_stopped=..., +) +await client.confirm_calibration_step(step_token=status.step.token) # on an operator action only +await client.stop_calibration() # idempotent +``` + +- `start_calibration` returns when the server accepts, not when the calibration finishes. A rejected start + raises `RequestRejectedError` and calls no callback. +- Every accepted attempt calls `on_stopped` exactly once. `CalibrationStopped.reason` is the server's + (`COMPLETED`, `CANCELLED`, `FAILED`, `UNCERTAIN`), or `None` if the connection or protocol failed first: the + outcome is then unknown, no files are offered, and the gloves may still be in use. +- `confirm_calibration_step` returns after the worker accepted starting the step. That is not completion of + the step or the calibration. Repeating an accepted token returns at once and starts nothing; a stale or + unoffered token raises `RequestRejectedError` and leaves the attempt unchanged. A pending confirm is + released with `CalibrationEndedError` if the calibration ends first. +- Callbacks run one at a time on the client's loop. A callback that raises cancels the calibration and its + exception is named in the terminal `error_message`. +- A client has one active attempt of either kind. + +## Steps + +`CalibrationStatus.step` carries what the SDK provides, never hardcoded: `title`, `instruction`, `index` and +`count` (steps for that glove), and `mode`: + +- `HOLD`: the operator is in the pose, then confirms. `duration_seconds` is 0. +- `MOTION`: confirming starts a timed sampling window of `duration_seconds`. + +`continuous` is true for an SDK step whose time is negative, a continuous step with an estimated duration. +`estimated_seconds` is that estimate, guidance for the operator only. The vendor documents no completion +signal for such steps, so neither CaptureServer nor the worker claims one: confirming starts the SDK step +and the next prompt follows. It is not an instantaneous capture and not a guaranteed timed completion. + +## Files + +A calibration that completes returns one `.mcal` basename per requested side. The worker writes the files +straight into the managed directory as `_.mcal` and never overwrites an earlier file; a failed, +cancelled or uncertain attempt removes whatever it wrote. The clients never choose paths or upload files. + +Calibration only writes into the directory. It does not list, read or select files; those belong to whatever +reads the folder. + +## Wire + +The existing numeric envelope is extended; existing capture values and the protocol version are unchanged. + +| Value | Message | Direction | +| --- | --- | --- | +| 13 | `StartCalibration` (`sides`: array of `HandSide`, `LEFT=1`, `RIGHT=2`) | client to server | +| 14 | `ConfirmCalibrationStep` (`step_token`) | client to server | +| 15 | `StopCalibration` | client to server | +| 16 | `CalibrationStarted` (`sides`) | server to client | +| 17 | `CalibrationStatus` (`phase`, `gloves`, `step`, `step_elapsed_seconds`) | server to client | +| 18 | `CalibrationStepAccepted` (`step_token`) | server to client | +| 19 | `CalibrationStopped` (`reason`, `result`, `error_message`) | server to client | + +All carry `version` and `session_id` like capture messages. `RequestRejected.request_type` may be 13, 14 or +15, and a rejected `ConfirmCalibrationStep` echoes `step_token`. `CalibrationPhase`: `CONNECTING=1`, +`AWAITING_STEP=2`, `CAPTURING_STEP=3`, `SAVING=4`, `STOPPING=5`. `CalibrationStopReason`: `COMPLETED=0`, +`CANCELLED=1`, `FAILED=2`, `UNCERTAIN=3` (the worker's outcome could not be established, for example it +stopped answering and was terminated). `result` is present exactly for `COMPLETED`. + +Message names in the design for this feature were symbolic (`startCalibration`, and so on); the numeric +values above are their wire form here, so the existing numeric discriminators are kept. + +## Status + +The CaptureServer side is exercised only against a scripted stand-in for the worker. The worker itself has +not completed a calibration with real gloves: with the real SDK and no working Manus license on the test +machine, the SDK call that starts a step did not return within the supervisor's bound, and the cause is not +established. An end-to-end calibration and MCAL export are not yet demonstrated. diff --git a/examples/egocap/capture_server/README.md b/examples/egocap/capture_server/README.md index bfaebbdde3..b2ede85419 100644 --- a/examples/egocap/capture_server/README.md +++ b/examples/egocap/capture_server/README.md @@ -88,6 +88,12 @@ a degraded camera. There is no automatic reconnection: retry by starting a new s If a release times out the server refuses further sessions (`BUSY`) until it is restarted, because a timeout does not prove the device was released. +## Manus calibration + +CaptureServer can calibrate Manus gloves on its host and keep the MCAL files in a managed directory for +later capture. See [`CALIBRATION.md`](CALIBRATION.md) for setup, the `CaptureClient` calibration API, the +wire messages and the file-selection rules. + ## Validation status Implemented and exercised on a Pico 4 Ultra and an OAK-D-W through the normal server diff --git a/examples/egocap/capture_server/capture_server/calibration.py b/examples/egocap/capture_server/capture_server/calibration.py new file mode 100644 index 0000000000..4169e0c6e0 --- /dev/null +++ b/examples/egocap/capture_server/capture_server/calibration.py @@ -0,0 +1,310 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""Manus glove calibration inside CaptureServer. + +A :class:`CalibrationSession` occupies the same admission gate as a capture session, so the two are +mutually exclusive from acceptance through cleanup. It runs the standalone ``manus_glove_calibrator`` +process through :mod:`isaaccapture_manus_calibration` and translates the worker's events into calibration +messages on the connection's single outbound writer. It emits control and status messages only; no +acquisition, OpenXR session or tracking source is created for it. +""" + +from __future__ import annotations + +import asyncio +import contextlib +import logging +import os +import shutil +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Protocol + +from isaaccapture_manus_calibration import protocol as wp +from isaaccapture_manus_calibration.worker import ( + AttemptError, + CalibrationAttempt, + CalibrationOutcome, + CalibrationWorker, + OutcomeKind, +) + +from . import protocol as p + +logger = logging.getLogger("isaaccapture.capture_server.calibration") + +WORKER_NAME = "manus_glove_calibrator" + +_SIDE_TO_WIRE = {wp.Side.LEFT: p.HandSide.LEFT, wp.Side.RIGHT: p.HandSide.RIGHT} +_SIDE_FROM_WIRE = {value: key for key, value in _SIDE_TO_WIRE.items()} +_PHASE_TO_WIRE = { + wp.Phase.DISCOVERING: p.CalibrationPhase.CONNECTING, + wp.Phase.AWAITING_STEP: p.CalibrationPhase.AWAITING_STEP, + wp.Phase.CAPTURING_STEP: p.CalibrationPhase.CAPTURING_STEP, + wp.Phase.SAVING: p.CalibrationPhase.SAVING, + wp.Phase.STOPPING: p.CalibrationPhase.STOPPING, +} +_REASON_TO_WIRE = { + OutcomeKind.COMPLETED: p.CalibrationStopReason.COMPLETED, + OutcomeKind.CANCELLED: p.CalibrationStopReason.CANCELLED, + OutcomeKind.FAILED: p.CalibrationStopReason.FAILED, + OutcomeKind.UNCERTAIN: p.CalibrationStopReason.UNCERTAIN, +} + + +@dataclass(frozen=True) +class CalibrationConfig: + """The managed MCAL directory on this host and, when one is available, the worker that generates files. + + Generated files are written into this directory as plain ``.mcal`` files, the same folder the device query + lists. Calibration only writes there; it never reads or selects files back. + """ + + directory: Path + worker_path: Path | None = None + + +def find_worker(override: str | os.PathLike[str] | None) -> Path | None: + """The worker executable: the configured override, else the Manus install under ``ISAAC_ROOT``. + + ``None`` when no executable is found, so calibration is simply unavailable and capture is unaffected. + """ + if override is not None: + candidate = Path(override) + return candidate if candidate.is_file() and os.access(candidate, os.X_OK) else None + root = os.environ.get("ISAAC_ROOT") + found = shutil.which(WORKER_NAME, path=str(Path(root) / "install" / "plugins" / "manus") if root else "") + return Path(found) if found else None + + +class _Outbox(Protocol): + """The part of the connection's single writer a calibration session uses.""" + + closed: bool + + def bind_session(self, stream_ids: list[int], *, on_sent: Any, on_fatal: Any) -> None: ... + + def publish_control(self, data: dict[str, Any]) -> bool: ... + + def publish_status(self, data: dict[str, Any]) -> None: ... + + async def quiesce(self) -> None: ... + + +class ConfirmRefused(Exception): + """A confirmation was not accepted; the active calibration is unchanged.""" + + +class CalibrationSession: + """One accepted calibration attempt; implements the admission surface a capture session has.""" + + def __init__( + self, + request: p.StartCalibrationRequest, + worker: CalibrationWorker, + outbox: _Outbox, + owner_ws: Any, + ) -> None: + self.session_id = request.session_id + self.owner_ws = owner_ws + self.request = request + self.done = asyncio.Event() + self.terminal: p.CalibrationStopped | None = None + self._worker = worker + self._outbox = outbox + self._phase = p.CalibrationPhase.CONNECTING + self._gloves: tuple[p.CalibrationGlove, ...] = () + self._step: p.CalibrationStep | None = None + self._elapsed: float | None = None + self._accepted: set[str] = set() + self._tasks: set[asyncio.Task[Any]] = set() + self._stop_requested = False + # Taken synchronously, before any await, so the station is occupied the moment the request is accepted. + self._attempt: CalibrationAttempt = worker.new_attempt(self._on_event) + + @property + def cleanup_budget_s(self) -> float: + """The longest a cancelled attempt can take to finish: cancel grace, then terminate, kill and reap.""" + w = self._worker + return w.cancel_grace_s + w.exit_grace_s + w.term_grace_s + w.kill_wait_s + 2.0 + + @property + def cleanup_ok(self) -> bool: + """False while the gloves may still be in use, so the station stays unavailable for every session. + + The supervisor sets the latch for a worker that could not be reaped and for one that exited after + reporting the SDK was not released: a process exit alone is not evidence the gloves are free. + """ + return self._worker.unreleased is None + + def request_stop(self, reason: Any = None, message: str | None = None) -> None: + """Cancel the attempt; idempotent. Used for StopCalibration, owner loss and shutdown.""" + if self._stop_requested: + return + self._stop_requested = True + self._spawn(self._attempt.cancel()) + + async def run(self) -> None: + """Run the attempt to its terminal outcome and publish the one terminal message after cleanup.""" + outcome: CalibrationOutcome | None = None + try: + self._outbox.bind_session([], on_sent=lambda _stream_id: None, on_fatal=self._on_fatal) + self._outbox.publish_control( + p.CalibrationStarted(session_id=self.session_id, sides=self.request.sides).to_json() + ) + self._publish_snapshot() + try: + await self._attempt.start(tuple(_SIDE_FROM_WIRE[side] for side in self.request.sides)) + except AttemptError: + # Stopped before the worker started: the attempt is already settled as cancelled. + pass + except Exception: + # A failed start settled the attempt; its outcome below carries the reason. + logger.exception("calibration start failed") + outcome = await self._attempt.outcome() + except asyncio.CancelledError: + await self._attempt.cancel() + outcome = await self._attempt.outcome() + raise + finally: + await self._finalize(outcome) + + async def handle_confirm(self, token: str) -> None: + """Start the offered step; the acceptance message itself is published from the worker's event, in order.""" + if token in self._accepted: + # An accepted token is acknowledged again and never starts a second step. + self._outbox.publish_control(p.CalibrationStepAccepted(self.session_id, token).to_json()) + return + prompt = self._attempt.prompt + if prompt is None or prompt.token != token: + raise ConfirmRefused("that step is not awaiting confirmation") + try: + await self._attempt.confirm(token) + except AttemptError as exc: + raise ConfirmRefused(str(exc)) from None + + def spawn_confirm(self, token: str, on_refused: Any) -> None: + """Run a confirmation without holding up the connection's read loop (a stop must still get through).""" + + async def _run() -> None: + try: + await self.handle_confirm(token) + except ConfirmRefused as exc: + on_refused(str(exc)) + + self._spawn(_run()) + + def replay(self) -> None: + """Re-send acceptance and the current snapshot for an identical repeated start.""" + self._outbox.publish_control(p.CalibrationStarted(self.session_id, self.request.sides).to_json()) + self._publish_snapshot() + + def _on_event(self, event: wp.WorkerEvent) -> None: + """Fold one worker event into the snapshot; called in worker event order.""" + if isinstance(event, wp.Status): + self._phase = _PHASE_TO_WIRE[event.phase] + self._gloves = tuple(p.CalibrationGlove(_SIDE_TO_WIRE[g.side], g.glove_id) for g in event.gloves) + if self._phase in ( + p.CalibrationPhase.CONNECTING, + p.CalibrationPhase.SAVING, + p.CalibrationPhase.STOPPING, + ): + self._step = None + self._elapsed = None + # The prompt that follows carries the step; a snapshot without it would show a prompt-less wait. + if self._phase is not p.CalibrationPhase.AWAITING_STEP: + self._publish_snapshot() + elif isinstance(event, wp.Prompt): + self._phase = p.CalibrationPhase.AWAITING_STEP + self._elapsed = None + self._step = p.CalibrationStep( + token=event.token, + side=_SIDE_TO_WIRE[event.side], + index=event.index, + count=event.count, + title=event.title, + instruction=event.instruction, + mode=p.CalibrationStepMode.MOTION + if event.mode is wp.StepMode.MOTION + else p.CalibrationStepMode.HOLD, + duration_seconds=event.duration_s, + continuous=event.continuous, + estimated_seconds=event.estimated_s, + ) + self._publish_snapshot() + elif isinstance(event, wp.StepAccepted): + self._accepted.add(event.token) + # Published here, in the worker's own event order, so the next prompt cannot overtake it. + self._outbox.publish_control(p.CalibrationStepAccepted(self.session_id, event.token).to_json()) + elif isinstance(event, wp.StepProgress): + self._elapsed = event.elapsed_s + self._publish_snapshot() + + def _publish_snapshot(self) -> None: + """Coalesced: each status is a complete replacement, so only the latest needs to arrive.""" + self._outbox.publish_status( + p.CalibrationStatus( + session_id=self.session_id, + phase=self._phase, + gloves=self._gloves, + step=self._step, + step_elapsed_seconds=self._elapsed, + ).to_json() + ) + + def _on_fatal(self, message: str) -> None: + """The connection's writer died: nobody is listening, so cancel.""" + logger.warning("calibration connection lost: %s", message) + self.request_stop() + + def _spawn(self, coro: Any) -> None: + task = asyncio.ensure_future(coro) + self._tasks.add(task) + task.add_done_callback(self._tasks.discard) + + async def _finalize(self, outcome: CalibrationOutcome | None) -> None: + """Quiesce the writer, publish the terminal message, and mark the session done.""" + try: + with contextlib.suppress(Exception): + await self._outbox.quiesce() + self.terminal = self._terminal_of(outcome) + self._outbox.publish_control(self.terminal.to_json()) + finally: + for task in list(self._tasks): + task.cancel() + self.done.set() + + def _terminal_of(self, outcome: CalibrationOutcome | None) -> p.CalibrationStopped: + """The server's honest account: files only for COMPLETED, never invented for any other outcome.""" + if outcome is None: + return p.CalibrationStopped( + self.session_id, + p.CalibrationStopReason.UNCERTAIN, + None, + "the calibration ended without an outcome", + ) + reason = _REASON_TO_WIRE[outcome.kind] + if outcome.kind is OutcomeKind.COMPLETED: + if not outcome.files: + # A completed result is promised only with files; never leave the client without a terminal. + return p.CalibrationStopped( + self.session_id, + p.CalibrationStopReason.UNCERTAIN, + None, + "the calibration reported completion but saved no files", + ) + files = tuple(p.McalFile(_SIDE_TO_WIRE[f.side], f.file) for f in outcome.files) + return p.CalibrationStopped(self.session_id, reason, p.CalibrationResult(files), None) + message = ( + None if outcome.kind is OutcomeKind.CANCELLED else (outcome.message or "calibration did not complete") + ) + return p.CalibrationStopped(self.session_id, reason, None, message) + + +def build_worker(config: CalibrationConfig) -> CalibrationWorker | None: + """The supervisor for the configured executable and managed MCAL directory; None without a worker.""" + if config.worker_path is None: + return None + return CalibrationWorker(config.worker_path, config.directory) diff --git a/examples/egocap/capture_server/capture_server/client.py b/examples/egocap/capture_server/capture_server/client.py index 9776eb38ce..03893ede4a 100644 --- a/examples/egocap/capture_server/capture_server/client.py +++ b/examples/egocap/capture_server/capture_server/client.py @@ -67,6 +67,8 @@ FieldDescriptor = p.FieldDescriptor __all__ = [ + "CalibrationEndedError", + "CalibrationStopped", "CaptureClient", "CaptureFrame", "CaptureStarted", @@ -290,6 +292,46 @@ class CaptureStopped: streams: tuple[p.StreamCounters, ...] | None +class CalibrationEndedError(Exception): + """A pending calibration confirmation was released because the calibration ended first.""" + + +@dataclass(frozen=True) +class CalibrationStopped: + """The client's calibration completion value. + + A non-None ``reason`` confirms the server's terminal message. ``None`` means the connection or protocol + failed before one arrived: the server-side outcome is unknown, no files are offered, and the gloves may + still be in use until the server reports otherwise. + """ + + session_id: str + reason: p.CalibrationStopReason | None + result: p.CalibrationResult | None + error_message: str | None + + +@dataclass +class _CalibrationAttempt: + """All state of one ``start_calibration`` call, owned by its supervisor task.""" + + session_id: str + sides: tuple[p.HandSide, ...] + callbacks: dict[str, Callback[Any] | None] + ws: Any = None + stop_requested: asyncio.Event = dataclass_field(default_factory=asyncio.Event) + # Set when on_started has been attempted, or the attempt failed before acceptance. + ready: asyncio.Event = dataclass_field(default_factory=asyncio.Event) + accepted: bool = False + start_exception: BaseException | None = None + events: asyncio.Queue[tuple[str, Any] | None] = dataclass_field(default_factory=asyncio.Queue) + pending: dict[str, asyncio.Future[None]] = dataclass_field(default_factory=dict) + accepted_tokens: set[str] = dataclass_field(default_factory=set) + stop_sent: bool = False + callback_error: BaseException | None = None + task: asyncio.Task[None] | None = None + + def _is_awaitable(value: Any) -> bool: return inspect.isawaitable(value) @@ -364,6 +406,7 @@ def __init__(self, url: str) -> None: raise ValueError("CaptureClient url must be a nonempty ws:// or wss:// string") self._url = url self._attempt: _Attempt | None = None + self._calibration: _CalibrationAttempt | None = None async def start( self, @@ -377,7 +420,7 @@ async def start( on_capture_image: Callback[CameraImage] | None = None, on_stopped: Callback[CaptureStopped] | None = None, ) -> None: - if self._attempt is not None: + if self._attempt is not None or self._calibration is not None: raise RuntimeError( "CaptureClient.start called while an attempt is active; the active attempt continues" ) @@ -434,6 +477,243 @@ async def stop(self) -> None: return # idle, or the previous attempt already finished attempt.stop_requested.set() # idempotent: a no-op if already set + # -- calibration ------------------------------------------------------- + + async def start_calibration( + self, + *, + sides: Sequence[p.HandSide], + on_started: Callback[p.CalibrationStarted] | None = None, + on_status: Callback[p.CalibrationStatus] | None = None, + on_stopped: Callback[CalibrationStopped] | None = None, + ) -> None: + """Start a Manus glove calibration on the server; returns once the server accepts it. + + Acceptance does not mean the gloves are connected or the calibration finished. Every accepted + attempt ends with exactly one ``on_stopped`` call, including after a lost connection (then with + ``reason=None``). A rejected start raises :class:`RequestRejectedError` and calls no callback. Callbacks + run one at a time on this loop; never wait inside one for a later callback. + """ + if self._attempt is not None or self._calibration is not None: + raise RuntimeError( + "CaptureClient.start_calibration called while an attempt is active; the active attempt continues" + ) + try: + request = p.StartCalibrationRequest(session_id=str(uuid.uuid4()), sides=tuple(sides)) + except (ValueError, TypeError) as exc: + raise ValueError(f"invalid CaptureClient.start_calibration arguments: {exc}") from exc + callbacks = {"on_started": on_started, "on_status": on_status, "on_stopped": on_stopped} + for name, value in callbacks.items(): + if value is not None and not callable(value): + raise ValueError(f"CaptureClient.start_calibration {name} must be callable or None") + attempt = _CalibrationAttempt(session_id=request.session_id, sides=request.sides, callbacks=callbacks) + self._calibration = attempt + attempt.task = asyncio.ensure_future(self._run_calibration(attempt, request)) + try: + await attempt.ready.wait() + except asyncio.CancelledError: + # Same contract as start(): signal a stop, let the supervisor tidy up, never cancel it. + attempt.stop_requested.set() + if not attempt.accepted and attempt.task is not None: + await asyncio.wait({attempt.task}, timeout=_ACCEPT_TIMEOUT_S) + raise + if attempt.start_exception is not None: + raise attempt.start_exception + + async def confirm_calibration_step(self, *, step_token: str) -> None: + """Start the step offered under ``step_token``; returns once the worker accepted it. + + Acceptance means the SDK accepted starting the step, not that the step or calibration finished. A + repeat of an accepted token returns at once and never starts another step. A stale or unoffered token + raises :class:`RequestRejectedError` and leaves the attempt unchanged; if the calibration ends first, + :class:`CalibrationEndedError` is raised. + """ + attempt = self._calibration + if attempt is None or not attempt.accepted: + raise RuntimeError("no accepted calibration is active") + if not isinstance(step_token, str) or not step_token: + raise ValueError("step_token must be a nonempty string") + if step_token in attempt.accepted_tokens: + return + future = attempt.pending.get(step_token) + if future is None: + future = asyncio.get_running_loop().create_future() + attempt.pending[step_token] = future + request = p.ConfirmCalibrationStepRequest(session_id=attempt.session_id, step_token=step_token) + await self._send_calibration(attempt, request.to_json()) + # Shielded: a cancelled caller must not cancel the future a retry (or the reader) still uses. + await asyncio.shield(future) + + async def stop_calibration(self) -> None: + """Ask the server to cancel the active calibration. Idempotent; wait for ``on_stopped`` for the result.""" + attempt = self._calibration + if attempt is None: + return + attempt.stop_requested.set() + if attempt.accepted and not attempt.stop_sent: + attempt.stop_sent = True + await self._send_calibration( + attempt, p.StopCalibrationRequest(session_id=attempt.session_id).to_json() + ) + + async def _send_calibration(self, attempt: _CalibrationAttempt, data: dict[str, Any]) -> None: + """Send one request; a dead transport is reported by the reader as an unconfirmed ending.""" + ws = attempt.ws + if ws is None: + return + try: + await asyncio.wait_for(ws.send(_to_json_text(data)), timeout=_STOP_TIMEOUT_S) + except (ConnectionClosed, OSError, asyncio.TimeoutError): + pass + + async def _run_calibration(self, attempt: _CalibrationAttempt, request: p.StartCalibrationRequest) -> None: + """The sole owner of one calibration attempt, from connect through finalization.""" + try: + started = await self._calibration_handshake(attempt, request) + except BaseException as exc: # noqa: BLE001 -- reported through start_exception, not raised here + await self._close_calibration_ws(attempt) + attempt.start_exception = ( + StoppedBeforeAcceptedError(str(exc)) if isinstance(exc, _StopWonRace) else exc + ) + attempt.ready.set() + self._calibration = None + return + attempt.accepted = True + dispatcher = asyncio.ensure_future(self._dispatch_calibration(attempt)) + attempt.events.put_nowait(("on_started", started)) + terminal: p.CalibrationStopped | None = None + diagnostic: BaseException | None = None + try: + if attempt.stop_requested.is_set(): + await self.stop_calibration() + terminal = await self._read_calibration(attempt) + except BaseException as exc: # noqa: BLE001 -- becomes the unconfirmed ending's diagnostic + diagnostic = exc + finally: + stopped = self._calibration_stopped(attempt, terminal, diagnostic) + for future in attempt.pending.values(): + if not future.done(): + future.set_exception( + CalibrationEndedError("the calibration ended before the step was accepted") + ) + attempt.pending.clear() + # The connection is closed and the client idle before on_stopped runs, so a caller (including the + # callback itself) can start the next attempt at once: the server frees its owner slot only when + # this connection is gone. + await self._close_calibration_ws(attempt) + if self._calibration is attempt: + self._calibration = None + attempt.events.put_nowait(("on_stopped", stopped)) + attempt.events.put_nowait(None) + await asyncio.gather(dispatcher, return_exceptions=True) + + async def _calibration_handshake( + self, attempt: _CalibrationAttempt, request: p.StartCalibrationRequest + ) -> p.CalibrationStarted: + """Connect, send StartCalibration, and wait for acceptance or rejection.""" + attempt.ws = await self._race( + websockets.connect(self._url, compression=None, max_size=_MAX_MESSAGE_BYTES), + attempt.stop_requested, + _ACCEPT_TIMEOUT_S, + ) + await self._race( + attempt.ws.send(_to_json_text(request.to_json())), attempt.stop_requested, _ACCEPT_TIMEOUT_S + ) + raw = await self._race(attempt.ws.recv(), attempt.stop_requested, _ACCEPT_TIMEOUT_S) + if isinstance(raw, bytes): + raise p.ProtocolError("received a binary message before CalibrationStarted") + event = p.parse_server_message(_from_json_text(raw)) + if isinstance(event, p.RequestRejected): + raise RequestRejectedError(event.session_id, event.reason, event.error_message) + if not isinstance(event, p.CalibrationStarted) or event.session_id != attempt.session_id: + raise p.ProtocolError(f"expected CalibrationStarted first, got {type(event).__name__}") + if event.sides != attempt.sides: + raise p.ProtocolError("CalibrationStarted names different sides than requested") + return event + + async def _read_calibration(self, attempt: _CalibrationAttempt) -> p.CalibrationStopped | None: + """Read until the server's terminal message; None when the transport ended without one.""" + while True: + try: + raw = await attempt.ws.recv() + except ConnectionClosed: + return None + if isinstance(raw, bytes): + raise p.ProtocolError("received a binary message during a calibration") + event = p.parse_server_message(_from_json_text(raw)) + if event.session_id != attempt.session_id: + raise p.ProtocolError(f"received a message for session {event.session_id!r}, expected another") + if isinstance(event, p.CalibrationStopped): + return event + if isinstance(event, p.CalibrationStatus): + attempt.events.put_nowait(("on_status", event)) + elif isinstance(event, p.CalibrationStepAccepted): + attempt.accepted_tokens.add(event.step_token) + future = attempt.pending.pop(event.step_token, None) + if future is not None and not future.done(): + future.set_result(None) + elif isinstance(event, p.RequestRejected): + self._calibration_rejected(attempt, event) + elif not isinstance(event, p.CalibrationStarted): + raise p.ProtocolError(f"unexpected {type(event).__name__} during a calibration") + + @staticmethod + def _calibration_rejected(attempt: _CalibrationAttempt, event: p.RequestRejected) -> None: + """A refused confirmation fails its caller; the attempt itself continues unchanged.""" + if event.request_type != p.MessageType.CONFIRM_CALIBRATION_STEP: + return + error = RequestRejectedError(event.session_id, event.reason, event.error_message) + tokens = [event.step_token] if event.step_token is not None else list(attempt.pending) + for token in tokens: + future = attempt.pending.pop(token, None) + if future is not None and not future.done(): + future.set_exception(error) + + async def _dispatch_calibration(self, attempt: _CalibrationAttempt) -> None: + """Deliver callbacks one at a time, in order, apart from the reader so a callback can await a confirm.""" + while True: + item = await attempt.events.get() + if item is None: + return + name, value = item + try: + await _invoke(attempt.callbacks[name], value) + except (Exception, asyncio.CancelledError) as exc: + if name == "on_stopped": + # Reported to the loop, never repeated: there is no later callback to carry it. + asyncio.get_running_loop().call_exception_handler( + {"message": "on_stopped callback raised", "exception": exc} + ) + elif attempt.callback_error is None: + # A failing callback cancels the calibration; the terminal value carries the diagnostic. + attempt.callback_error = exc + attempt.stop_requested.set() + await self.stop_calibration() + finally: + if name == "on_started": + attempt.ready.set() + + @staticmethod + def _calibration_stopped( + attempt: _CalibrationAttempt, terminal: p.CalibrationStopped | None, diagnostic: BaseException | None + ) -> CalibrationStopped: + """The server's terminal message, or an honest unconfirmed value; files only from a confirmed COMPLETED.""" + notes = [f"a callback raised: {attempt.callback_error!r}"] if attempt.callback_error is not None else [] + if terminal is not None: + message = "; ".join(([terminal.error_message] if terminal.error_message else []) + notes) or None + return CalibrationStopped(attempt.session_id, terminal.reason, terminal.result, message) + why = f"protocol error: {diagnostic}" if diagnostic is not None else "the connection ended" + message = "; ".join([f"{why}; the server's outcome is unknown", *notes]) + return CalibrationStopped(attempt.session_id, None, None, message) + + async def _close_calibration_ws(self, attempt: _CalibrationAttempt) -> None: + ws, attempt.ws = attempt.ws, None + if ws is not None: + try: + await asyncio.wait_for(ws.close(), timeout=_CLOSE_TIMEOUT_S) + except BaseException: # noqa: BLE001 -- best-effort cleanup, must never raise or hang + pass + # -- the supervisor ---------------------------------------------------- async def _run_attempt(self, attempt: _Attempt) -> None: diff --git a/examples/egocap/capture_server/capture_server/protocol.py b/examples/egocap/capture_server/capture_server/protocol.py index 04ef35bd86..361ddefb51 100644 --- a/examples/egocap/capture_server/capture_server/protocol.py +++ b/examples/egocap/capture_server/capture_server/protocol.py @@ -28,6 +28,7 @@ from __future__ import annotations +import re import struct from dataclasses import dataclass from dataclasses import field as dataclass_field @@ -70,6 +71,17 @@ class MessageType(IntEnum): CAPTURE_IMAGE = 8 SESSION_INIT = 9 REQUEST_REJECTED = 10 + # 11 and 12 are reserved for the read-only device query. + # Calibration operations. The numeric names below are the wire form of the symbolic operation names + # startCalibration, confirmCalibrationStep, stopCalibration, CalibrationStarted, CalibrationStatus, + # CalibrationStepAccepted and CalibrationStopped. + START_CALIBRATION = 13 + CONFIRM_CALIBRATION_STEP = 14 + STOP_CALIBRATION = 15 + CALIBRATION_STARTED = 16 + CALIBRATION_STATUS = 17 + CALIBRATION_STEP_ACCEPTED = 18 + CALIBRATION_STOPPED = 19 class SourceType(IntEnum): @@ -113,6 +125,33 @@ class RejectionReason(IntEnum): UNKNOWN_SESSION = 3 +class HandSide(IntEnum): + LEFT = 1 + RIGHT = 2 + + +class CalibrationPhase(IntEnum): + CONNECTING = 1 + AWAITING_STEP = 2 + CAPTURING_STEP = 3 + SAVING = 4 + STOPPING = 5 + + +class CalibrationStepMode(IntEnum): + HOLD = 0 + MOTION = 1 + + +class CalibrationStopReason(IntEnum): + COMPLETED = 0 + CANCELLED = 1 + FAILED = 2 + # The server could not establish what the worker did (for example it stopped answering and was + # terminated); no files are offered. + UNCERTAIN = 3 + + class StreamKind(IntEnum): TRACKING = 1 IMAGE = 2 @@ -925,6 +964,31 @@ def _require_session_id(data: dict) -> str: return session_id +_MCAL_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}\.mcal") +_MAX_TOKEN_LEN = 128 + + +def _require_mcal_name(value: Any, context: str) -> str | None: + """An optional ``.mcal`` basename: no separators, no leading dot, bounded length.""" + if value is None: + return None + if not isinstance(value, str) or not _MCAL_NAME_RE.fullmatch(value): + raise ProtocolError(f"{context} must be an .mcal basename or null") + return value + + +def _require_token(value: Any, context: str) -> str: + if not isinstance(value, str) or not value or len(value) > _MAX_TOKEN_LEN: + raise ProtocolError(f"{context} must be a nonempty string of at most {_MAX_TOKEN_LEN} characters") + return value + + +def _require_seconds(value: Any, context: str) -> float: + if isinstance(value, bool) or not isinstance(value, (int, float)) or not (0 <= value < float("inf")): + raise ProtocolError(f"{context} must be a finite nonnegative number") + return float(value) + + def require_envelope(data: dict, expected_type: MessageType) -> str: """Envelope-level validation shared by every message kind that needs it: message type, protocol version, and the canonical session_id rule @@ -1799,23 +1863,413 @@ def from_json(cls, data: Any) -> SessionStopped: ) +# --------------------------------------------------------------------------- +# Calibration. A calibration session is exclusive with a capture session and emits control/status +# messages only. Operation names map to numeric discriminators (see MessageType). +# --------------------------------------------------------------------------- + + +def _require_hand_sides(value: Any, context: str) -> tuple[HandSide, ...]: + if not isinstance(value, list) or not value: + raise ProtocolError(f"{context} must be a nonempty list") + sides = tuple(_require_int_enum(item, HandSide, f"{context}[]") for item in value) + if len(sides) != len(set(sides)): + raise ProtocolError(f"{context} has duplicate entries: {[int(side) for side in sides]}") + return sides + + +@dataclass(frozen=True) +class StartCalibrationRequest: + session_id: str + sides: tuple[HandSide, ...] + + def __post_init__(self) -> None: + if not self.sides or len(self.sides) != len(set(self.sides)): + raise ValueError("sides must be a nonempty sequence of unique HandSide values") + if not all(type(side) is HandSide for side in self.sides): + raise ValueError("sides must contain only HandSide members") + + def to_json(self) -> dict[str, Any]: + return { + "type": int(MessageType.START_CALIBRATION), + "version": PROTOCOL_VERSION, + "session_id": self.session_id, + "sides": [int(side) for side in self.sides], + } + + +def parse_start_calibration(data: dict) -> StartCalibrationRequest: + _require_keys(data, {"type", "version", "session_id", "sides"}, "StartCalibration") + session_id = require_envelope(data, MessageType.START_CALIBRATION) + return StartCalibrationRequest( + session_id=session_id, sides=_require_hand_sides(data.get("sides"), "StartCalibration.sides") + ) + + +@dataclass(frozen=True) +class ConfirmCalibrationStepRequest: + session_id: str + step_token: str + + def to_json(self) -> dict[str, Any]: + return { + "type": int(MessageType.CONFIRM_CALIBRATION_STEP), + "version": PROTOCOL_VERSION, + "session_id": self.session_id, + "step_token": self.step_token, + } + + +def parse_confirm_calibration_step(data: dict) -> ConfirmCalibrationStepRequest: + _require_keys(data, {"type", "version", "session_id", "step_token"}, "ConfirmCalibrationStep") + session_id = require_envelope(data, MessageType.CONFIRM_CALIBRATION_STEP) + return ConfirmCalibrationStepRequest( + session_id=session_id, + step_token=_require_token(data.get("step_token"), "ConfirmCalibrationStep.step_token"), + ) + + +@dataclass(frozen=True) +class StopCalibrationRequest: + session_id: str + + def to_json(self) -> dict[str, Any]: + return { + "type": int(MessageType.STOP_CALIBRATION), + "version": PROTOCOL_VERSION, + "session_id": self.session_id, + } + + +def parse_stop_calibration(data: dict) -> StopCalibrationRequest: + _require_keys(data, {"type", "version", "session_id"}, "StopCalibration") + return StopCalibrationRequest(session_id=require_envelope(data, MessageType.STOP_CALIBRATION)) + + +@dataclass(frozen=True) +class CalibrationGlove: + side: HandSide + glove_id: int + + def to_json(self) -> dict[str, Any]: + return {"side": int(self.side), "glove_id": self.glove_id} + + @classmethod + def from_json(cls, data: Any) -> CalibrationGlove: + _require_keys(data, {"side", "glove_id"}, "CalibrationGlove") + glove_id = data.get("glove_id") + if type(glove_id) is not int or glove_id < 0: + raise ProtocolError("CalibrationGlove.glove_id must be a nonnegative integer") + return cls(side=_require_int_enum(data.get("side"), HandSide, "CalibrationGlove.side"), glove_id=glove_id) + + +@dataclass(frozen=True) +class CalibrationStep: + """One SDK-provided step awaiting confirmation or running. + + ``duration_seconds`` is a timed sampling window (positive only for MOTION). A continuous step + (``continuous``) has no completion signal from the vendor: ``estimated_seconds`` is the SDK's estimate, + guidance for the operator only, and never a guaranteed completion time. + """ + + token: str + side: HandSide + index: int + count: int + title: str + instruction: str + mode: CalibrationStepMode + duration_seconds: float + continuous: bool + estimated_seconds: float + + def __post_init__(self) -> None: + if not self.token or self.count <= 0 or not 0 <= self.index < self.count: + raise ValueError("CalibrationStep needs a token and 0 <= index < count") + if (self.mode is CalibrationStepMode.HOLD) != (self.duration_seconds == 0): + raise ValueError("duration_seconds must be 0 for HOLD and positive for MOTION") + if self.continuous and (self.mode is not CalibrationStepMode.HOLD or self.estimated_seconds <= 0): + raise ValueError("a continuous step is a HOLD with a positive estimated_seconds") + if not self.continuous and self.estimated_seconds != 0: + raise ValueError("estimated_seconds must be 0 unless the step is continuous") + + def to_json(self) -> dict[str, Any]: + return { + "token": self.token, + "side": int(self.side), + "index": self.index, + "count": self.count, + "title": self.title, + "instruction": self.instruction, + "mode": int(self.mode), + "duration_seconds": self.duration_seconds, + "continuous": self.continuous, + "estimated_seconds": self.estimated_seconds, + } + + @classmethod + def from_json(cls, data: Any) -> CalibrationStep: + _require_keys( + data, + { + "token", + "side", + "index", + "count", + "title", + "instruction", + "mode", + "duration_seconds", + "continuous", + "estimated_seconds", + }, + "CalibrationStep", + ) + index, count = data.get("index"), data.get("count") + if type(index) is not int or type(count) is not int: + raise ProtocolError("CalibrationStep.index and count must be integers") + title, instruction = data.get("title"), data.get("instruction") + if not isinstance(title, str) or not isinstance(instruction, str) or not instruction: + raise ProtocolError("CalibrationStep.title must be a string and instruction a nonempty string") + try: + return cls( + token=_require_token(data.get("token"), "CalibrationStep.token"), + side=_require_int_enum(data.get("side"), HandSide, "CalibrationStep.side"), + index=index, + count=count, + title=title, + instruction=instruction, + mode=_require_int_enum(data.get("mode"), CalibrationStepMode, "CalibrationStep.mode"), + duration_seconds=_require_seconds( + data.get("duration_seconds"), "CalibrationStep.duration_seconds" + ), + continuous=_require_strict_bool(data.get("continuous"), "CalibrationStep.continuous"), + estimated_seconds=_require_seconds( + data.get("estimated_seconds"), "CalibrationStep.estimated_seconds" + ), + ) + except ValueError as exc: + raise ProtocolError(str(exc)) from None + + +@dataclass(frozen=True) +class McalFile: + side: HandSide + filename: str + + def to_json(self) -> dict[str, Any]: + return {"side": int(self.side), "filename": self.filename} + + @classmethod + def from_json(cls, data: Any) -> McalFile: + _require_keys(data, {"side", "filename"}, "McalFile") + filename = _require_mcal_name(data.get("filename"), "McalFile.filename") + if filename is None: + raise ProtocolError("McalFile.filename is required") + return cls(side=_require_int_enum(data.get("side"), HandSide, "McalFile.side"), filename=filename) + + +@dataclass(frozen=True) +class CalibrationResult: + files: tuple[McalFile, ...] + + def to_json(self) -> dict[str, Any]: + return {"files": [f.to_json() for f in self.files]} + + @classmethod + def from_json(cls, data: Any) -> CalibrationResult: + _require_keys(data, {"files"}, "CalibrationResult") + files = data.get("files") + if not isinstance(files, list) or not files: + raise ProtocolError("CalibrationResult.files must be a nonempty list") + parsed = tuple(McalFile.from_json(f) for f in files) + if len({f.side for f in parsed}) != len(parsed): + raise ProtocolError("CalibrationResult.files has two files for one side") + return cls(files=parsed) + + +@dataclass(frozen=True) +class CalibrationStarted: + session_id: str + sides: tuple[HandSide, ...] + type: MessageType = dataclass_field(default=MessageType.CALIBRATION_STARTED, init=False) + version: int = dataclass_field(default=PROTOCOL_VERSION, init=False) + + def to_json(self) -> dict[str, Any]: + return { + "type": int(self.type), + "version": self.version, + "session_id": self.session_id, + "sides": [int(side) for side in self.sides], + } + + @classmethod + def from_json(cls, data: Any) -> CalibrationStarted: + _require_keys(data, {"type", "version", "session_id", "sides"}, "CalibrationStarted") + _require_message_type(data, MessageType.CALIBRATION_STARTED) + _require_version(data) + return cls( + session_id=_require_session_id(data), + sides=_require_hand_sides(data.get("sides"), "CalibrationStarted.sides"), + ) + + +@dataclass(frozen=True) +class CalibrationStatus: + """A complete replacement snapshot of calibration progress, never a delta.""" + + session_id: str + phase: CalibrationPhase + gloves: tuple[CalibrationGlove, ...] + step: CalibrationStep | None + step_elapsed_seconds: float | None + type: MessageType = dataclass_field(default=MessageType.CALIBRATION_STATUS, init=False) + version: int = dataclass_field(default=PROTOCOL_VERSION, init=False) + + def to_json(self) -> dict[str, Any]: + return { + "type": int(self.type), + "version": self.version, + "session_id": self.session_id, + "phase": int(self.phase), + "gloves": [g.to_json() for g in self.gloves], + "step": self.step.to_json() if self.step is not None else None, + "step_elapsed_seconds": self.step_elapsed_seconds, + } + + @classmethod + def from_json(cls, data: Any) -> CalibrationStatus: + _require_keys( + data, + {"type", "version", "session_id", "phase", "gloves", "step", "step_elapsed_seconds"}, + "CalibrationStatus", + ) + _require_message_type(data, MessageType.CALIBRATION_STATUS) + _require_version(data) + gloves = data.get("gloves") + if not isinstance(gloves, list): + raise ProtocolError("CalibrationStatus.gloves must be a list") + step = data.get("step") + elapsed = data.get("step_elapsed_seconds") + return cls( + session_id=_require_session_id(data), + phase=_require_int_enum(data.get("phase"), CalibrationPhase, "CalibrationStatus.phase"), + gloves=tuple(CalibrationGlove.from_json(g) for g in gloves), + step=None if step is None else CalibrationStep.from_json(step), + step_elapsed_seconds=None if elapsed is None else _require_seconds(elapsed, "step_elapsed_seconds"), + ) + + +@dataclass(frozen=True) +class CalibrationStepAccepted: + """The worker accepted starting the step; it does not mean the step or the calibration finished.""" + + session_id: str + step_token: str + type: MessageType = dataclass_field(default=MessageType.CALIBRATION_STEP_ACCEPTED, init=False) + version: int = dataclass_field(default=PROTOCOL_VERSION, init=False) + + def to_json(self) -> dict[str, Any]: + return { + "type": int(self.type), + "version": self.version, + "session_id": self.session_id, + "step_token": self.step_token, + } + + @classmethod + def from_json(cls, data: Any) -> CalibrationStepAccepted: + _require_keys(data, {"type", "version", "session_id", "step_token"}, "CalibrationStepAccepted") + _require_message_type(data, MessageType.CALIBRATION_STEP_ACCEPTED) + _require_version(data) + return cls( + session_id=_require_session_id(data), + step_token=_require_token(data.get("step_token"), "CalibrationStepAccepted.step_token"), + ) + + +@dataclass(frozen=True) +class CalibrationStopped: + """The one terminal message of an accepted calibration, sent after cleanup. Files only for COMPLETED.""" + + session_id: str + reason: CalibrationStopReason + result: CalibrationResult | None + error_message: str | None + type: MessageType = dataclass_field(default=MessageType.CALIBRATION_STOPPED, init=False) + version: int = dataclass_field(default=PROTOCOL_VERSION, init=False) + + def __post_init__(self) -> None: + if (self.reason is CalibrationStopReason.COMPLETED) != (self.result is not None): + raise ValueError("a result is present exactly when the calibration COMPLETED") + if self.error_message is not None and not self.error_message: + raise ValueError("error_message must be nonempty or null") + if self.reason is CalibrationStopReason.COMPLETED and self.error_message is not None: + raise ValueError("a COMPLETED calibration has no error_message") + + def to_json(self) -> dict[str, Any]: + return { + "type": int(self.type), + "version": self.version, + "session_id": self.session_id, + "reason": int(self.reason), + "result": self.result.to_json() if self.result is not None else None, + "error_message": self.error_message, + } + + @classmethod + def from_json(cls, data: Any) -> CalibrationStopped: + _require_keys( + data, {"type", "version", "session_id", "reason", "result", "error_message"}, "CalibrationStopped" + ) + _require_message_type(data, MessageType.CALIBRATION_STOPPED) + _require_version(data) + result = data.get("result") + error_message = data.get("error_message") + if error_message is not None and (not isinstance(error_message, str) or not error_message): + raise ProtocolError("CalibrationStopped.error_message must be a nonempty string or null") + try: + return cls( + session_id=_require_session_id(data), + reason=_require_int_enum(data.get("reason"), CalibrationStopReason, "CalibrationStopped.reason"), + result=None if result is None else CalibrationResult.from_json(result), + error_message=error_message, + ) + except ValueError as exc: + raise ProtocolError(str(exc)) from None + + +# Client request types a RequestRejected can answer. +_REJECTABLE_REQUESTS = ( + MessageType.BEGIN, + MessageType.STOP, + MessageType.START_CALIBRATION, + MessageType.CONFIRM_CALIBRATION_STEP, + MessageType.STOP_CALIBRATION, +) + + @dataclass(frozen=True) class RequestRejected: session_id: str request_type: MessageType reason: RejectionReason error_message: str + # The attempted token of a rejected CONFIRM_CALIBRATION_STEP, so the client can tell which prompt failed. + step_token: str | None = None type: MessageType = dataclass_field(default=MessageType.REQUEST_REJECTED, init=False) version: int = dataclass_field(default=PROTOCOL_VERSION, init=False) def __post_init__(self) -> None: - if self.request_type not in (MessageType.BEGIN, MessageType.STOP): - raise ValueError("RequestRejected.request_type must be BEGIN or STOP") + if self.request_type not in _REJECTABLE_REQUESTS: + raise ValueError("RequestRejected.request_type must be a client request type") if not self.error_message: raise ValueError("RequestRejected.error_message must be a nonempty string") + if self.step_token is not None and self.request_type != MessageType.CONFIRM_CALIBRATION_STEP: + raise ValueError("RequestRejected.step_token is only valid for CONFIRM_CALIBRATION_STEP") def to_json(self) -> dict[str, Any]: - return { + data: dict[str, Any] = { "type": int(self.type), "version": self.version, "session_id": self.session_id, @@ -1823,33 +2277,54 @@ def to_json(self) -> dict[str, Any]: "reason": int(self.reason), "error_message": self.error_message, } + # Omitted rather than null, so Begin and Stop rejections keep their exact earlier wire form. + if self.step_token is not None: + data["step_token"] = self.step_token + return data @classmethod def from_json(cls, data: Any) -> RequestRejected: _require_keys( data, - {"type", "version", "session_id", "request_type", "reason", "error_message"}, + {"type", "version", "session_id", "request_type", "reason", "error_message", "step_token"}, "RequestRejected", ) _require_message_type(data, MessageType.REQUEST_REJECTED) _require_version(data) session_id = _require_session_id(data) request_type = _require_int_enum(data.get("request_type"), MessageType, "RequestRejected.request_type") - if request_type not in (MessageType.BEGIN, MessageType.STOP): - raise ProtocolError("RequestRejected.request_type must be BEGIN or STOP") + if request_type not in _REJECTABLE_REQUESTS: + raise ProtocolError("RequestRejected.request_type must be a client request type") error_message = data.get("error_message") if not isinstance(error_message, str) or not error_message: raise ProtocolError("RequestRejected.error_message must be a nonempty string") + step_token = data.get("step_token") + if step_token is not None: + if request_type != MessageType.CONFIRM_CALIBRATION_STEP: + raise ProtocolError("RequestRejected.step_token is only valid for CONFIRM_CALIBRATION_STEP") + step_token = _require_token(step_token, "RequestRejected.step_token") return cls( session_id=session_id, request_type=request_type, reason=_require_int_enum(data.get("reason"), RejectionReason, "RequestRejected.reason"), error_message=error_message, + step_token=step_token, ) # A server-to-client JSON message, dispatched by its envelope's "type". -ServerJsonMessage = SessionInit | SessionStarting | SessionBegun | SessionStatus | SessionStopped | RequestRejected +ServerJsonMessage = ( + SessionInit + | SessionStarting + | SessionBegun + | SessionStatus + | SessionStopped + | RequestRejected + | CalibrationStarted + | CalibrationStatus + | CalibrationStepAccepted + | CalibrationStopped +) _JSON_PARSERS_BY_TYPE: dict[MessageType, Any] = { MessageType.SESSION_INIT: SessionInit.from_json, @@ -1858,6 +2333,10 @@ def from_json(cls, data: Any) -> RequestRejected: MessageType.SESSION_STATUS: SessionStatus.from_json, MessageType.SESSION_STOPPED: SessionStopped.from_json, MessageType.REQUEST_REJECTED: RequestRejected.from_json, + MessageType.CALIBRATION_STARTED: CalibrationStarted.from_json, + MessageType.CALIBRATION_STATUS: CalibrationStatus.from_json, + MessageType.CALIBRATION_STEP_ACCEPTED: CalibrationStepAccepted.from_json, + MessageType.CALIBRATION_STOPPED: CalibrationStopped.from_json, } diff --git a/examples/egocap/capture_server/capture_server/server.py b/examples/egocap/capture_server/capture_server/server.py index 6a46e8c895..a6fc665a2b 100644 --- a/examples/egocap/capture_server/capture_server/server.py +++ b/examples/egocap/capture_server/capture_server/server.py @@ -45,14 +45,17 @@ import sys import time from dataclasses import dataclass +from pathlib import Path from typing import Any, Awaitable, Callable, TypeVar import websockets +from isaaccapture_manus_calibration.worker import AttemptError from websockets.exceptions import ConnectionClosed from websockets.server import ServerConnection from . import protocol as p from .acquisition import Acquisition, CameraSourceError, SourceExited +from .calibration import CalibrationConfig, CalibrationSession, build_worker, find_worker __all__ = ["CaptureServer", "ServerConfig", "main"] @@ -160,6 +163,9 @@ class ServerConfig: # prints the mode's default (the mock's SYNTHETIC-data warning, nothing for live); replay sets it so # recorded data is never announced as synthetic. data_note: str | None = None + # Manus glove calibration: the worker executable and the managed MCAL directory. None leaves calibration + # unavailable (a StartCalibration is rejected) and capture unaffected. + calibration: CalibrationConfig | None = None def __post_init__(self) -> None: """Reject one physical camera assigned to more than one logical name.""" @@ -1131,7 +1137,12 @@ def __init__( # ws.send() directly; everything (rejections, replays, session # lifecycle, status, frames) goes through this. self._outbox: _Outbox | None = None - self._active_session: _Session | None = None + self._active_session: _Session | CalibrationSession | None = None + self._calibration_worker = build_worker(self.config.calibration) if self.config.calibration else None + # Terminal calibration messages of recent attempts on the current owner connection, for stop retries. + self._completed_calibrations: collections.OrderedDict[str, p.CalibrationStopped] = ( + collections.OrderedDict() + ) # Scoped to the CURRENT owner connection, reset in _handle_connection # -- a session ID completed under a since-disconnected owner must # not leak into a brand new connection's admission decisions. @@ -1227,11 +1238,12 @@ async def close(self) -> None: # clean shutdown, same as REQUESTED/OWNER_DISCONNECTED -- # these are benign stop causes, not error diagnostics. self._active_session.request_stop(p.StopReason.SERVER_SHUTDOWN) + timeout = _SOURCE_RELEASE_TIMEOUT_S + _SEND_DEADLINE_S + 1.0 + if isinstance(self._active_session, CalibrationSession): + # A stuck worker needs its full cancel, terminate and kill sequence before it is reaped. + timeout = max(timeout, self._active_session.cleanup_budget_s) with contextlib.suppress(TimeoutError): - await asyncio.wait_for( - self._active_session.done.wait(), - timeout=_SOURCE_RELEASE_TIMEOUT_S + _SEND_DEADLINE_S + 1.0, - ) + await asyncio.wait_for(self._active_session.done.wait(), timeout=timeout) finally: if self._ws_server is not None: self._ws_server.close() @@ -1262,6 +1274,7 @@ async def _handle_connection(self, ws: ServerConnection) -> None: outbox.start() self._outbox = outbox self._completed.clear() + self._completed_calibrations.clear() try: async for raw in ws: await self._handle_message(ws, raw) @@ -1324,6 +1337,12 @@ async def _handle_message(self, ws: ServerConnection, raw: str | bytes) -> None: await self._handle_begin(ws, data) elif data.get("type") == int(p.MessageType.STOP): await self._handle_stop(ws, data) + elif data.get("type") == int(p.MessageType.START_CALIBRATION): + await self._handle_start_calibration(ws, data) + elif data.get("type") == int(p.MessageType.CONFIRM_CALIBRATION_STEP): + await self._handle_confirm_calibration_step(ws, data) + elif data.get("type") == int(p.MessageType.STOP_CALIBRATION): + await self._handle_stop_calibration(ws, data) else: await ws.close(code=1002, reason="unexpected message type from client") @@ -1333,6 +1352,7 @@ def _reject( request_type: p.MessageType, reason: p.RejectionReason, message: str, + step_token: str | None = None, ) -> None: """Enqueued through the shared Outbox's control FIFO, same as every other required message -- not a direct send. There is only @@ -1346,6 +1366,7 @@ def _reject( request_type=request_type, reason=reason, error_message=message, + step_token=step_token, ).to_json(), ) @@ -1422,6 +1443,15 @@ async def _handle_begin(self, ws: ServerConnection, data: dict[str, Any]) -> Non if self._active_session is not None: active = self._active_session + if isinstance(active, CalibrationSession): + # Capture never interrupts a calibration, and a calibration is not replayable as a capture. + self._reject( + begin.session_id, + p.MessageType.BEGIN, + p.RejectionReason.BUSY, + "a calibration session is active", + ) + return # Owner identity, not just session_id, gates replay: a session # can remain the active/admission-blocking one after its owner # disconnected (its resource release failed) -- a DIFFERENT, @@ -1558,6 +1588,7 @@ async def _handle_stop(self, ws: ServerConnection, data: dict[str, Any]) -> None # session_id. if ( self._active_session is not None + and not isinstance(self._active_session, CalibrationSession) and self._active_session.session_id == stop.session_id and self._active_session.owner_ws is ws ): @@ -1585,6 +1616,160 @@ async def _handle_stop(self, ws: ServerConnection, data: dict[str, Any]) -> None "no active or recently completed session with that ID", ) + async def _handle_start_calibration(self, ws: ServerConnection, data: dict[str, Any]) -> None: + try: + session_id = p.require_envelope(data, p.MessageType.START_CALIBRATION) + except p.ProtocolError: + await ws.close(code=1002, reason="invalid StartCalibration envelope") + return + if self._stop_event.is_set() or self._outbox is None or self._outbox.closed: + self._reject( + session_id, + p.MessageType.START_CALIBRATION, + p.RejectionReason.BUSY, + "server is shutting down" if self._stop_event.is_set() else "this connection is no longer usable", + ) + return + try: + request = p.parse_start_calibration(data) + except p.ProtocolError as exc: + self._reject(session_id, p.MessageType.START_CALIBRATION, p.RejectionReason.INVALID_CONFIG, str(exc)) + return + active = self._active_session + if ( + isinstance(active, CalibrationSession) + and active.session_id == request.session_id + and active.owner_ws is ws + ): + # An identical repeat re-sends acceptance and the current snapshot; it never starts a second worker. + if active.request != request: + self._reject( + request.session_id, + p.MessageType.START_CALIBRATION, + p.RejectionReason.INVALID_CONFIG, + "StartCalibration differs from the active calibration", + ) + return + active.replay() + return + if active is not None: + self._reject( + request.session_id, + p.MessageType.START_CALIBRATION, + p.RejectionReason.BUSY, + "another session is already active", + ) + return + if self._calibration_worker is None: + # Say which setting is missing, so the operator knows what to start the server with. + reason = ( + "the server has no calibration directory (--manus-calibration-dir) and no Manus calibration " + "worker (--manus-calibrator)" + if self.config.calibration is None + else "no Manus calibration worker is configured or found" + ) + self._reject( + request.session_id, + p.MessageType.START_CALIBRATION, + p.RejectionReason.INVALID_CONFIG, + f"calibration is not available: {reason}", + ) + return + if request.session_id in self._completed or request.session_id in self._completed_calibrations: + self._reject( + request.session_id, + p.MessageType.START_CALIBRATION, + p.RejectionReason.INVALID_CONFIG, + "session_id was already used by a completed session", + ) + return + assert self._outbox is not None # bound in _handle_connection before any message is handled + try: + session = CalibrationSession(request, self._calibration_worker, self._outbox, ws) + except AttemptError as exc: + # A worker that could not be reaped may still hold the gloves. + self._reject(request.session_id, p.MessageType.START_CALIBRATION, p.RejectionReason.BUSY, str(exc)) + return + # Occupies the gate before any await, exactly as an accepted Begin does. + self._active_session = session + asyncio.ensure_future(self._run_calibration(session)) + + async def _run_calibration(self, session: CalibrationSession) -> None: + try: + await session.run() + finally: + if session.terminal is not None: + self._completed_calibrations[session.session_id] = session.terminal + self._completed_calibrations.move_to_end(session.session_id) + while len(self._completed_calibrations) > _COMPLETED_TOMBSTONE_CAPACITY: + self._completed_calibrations.popitem(last=False) + # Released only when the worker is known to be gone; an unreaped worker keeps the station busy. + if self._active_session is session and session.cleanup_ok: + self._active_session = None + + async def _handle_confirm_calibration_step(self, ws: ServerConnection, data: dict[str, Any]) -> None: + try: + confirm = p.parse_confirm_calibration_step(data) + except p.ProtocolError: + await ws.close(code=1002, reason="invalid ConfirmCalibrationStep envelope") + return + active = self._active_session + if not ( + isinstance(active, CalibrationSession) + and active.session_id == confirm.session_id + and active.owner_ws is ws + and not active.done.is_set() + ): + self._reject( + confirm.session_id, + p.MessageType.CONFIRM_CALIBRATION_STEP, + p.RejectionReason.UNKNOWN_SESSION, + "no active calibration with that ID", + step_token=confirm.step_token, + ) + return + # Run off the read loop: a slow acceptance must not block a StopCalibration behind it. + active.spawn_confirm( + confirm.step_token, + lambda message: self._reject( + confirm.session_id, + p.MessageType.CONFIRM_CALIBRATION_STEP, + p.RejectionReason.INVALID_CONFIG, + message, + step_token=confirm.step_token, + ), + ) + + async def _handle_stop_calibration(self, ws: ServerConnection, data: dict[str, Any]) -> None: + try: + stop = p.parse_stop_calibration(data) + except p.ProtocolError: + await ws.close(code=1002, reason="invalid StopCalibration envelope") + return + active = self._active_session + if ( + isinstance(active, CalibrationSession) + and active.session_id == stop.session_id + and active.owner_ws is ws + ): + if active.done.is_set(): + # Already finished but still holding the gate (its worker was not reaped): replay the terminal. + if active.terminal is not None and self._outbox is not None: + self._outbox.publish_control(active.terminal.to_json()) + return + active.request_stop() + return + if stop.session_id in self._completed_calibrations: + if self._outbox is not None: + self._outbox.publish_control(self._completed_calibrations[stop.session_id].to_json()) + return + self._reject( + stop.session_id, + p.MessageType.STOP_CALIBRATION, + p.RejectionReason.UNKNOWN_SESSION, + "no active or recently completed calibration with that ID", + ) + def _build_arg_parser() -> argparse.ArgumentParser: # server.py never imports capture_server.mock: the mock-only options are @@ -1652,9 +1837,47 @@ def _build_arg_parser() -> argparse.ArgumentParser: parser.add_argument( "--camera-usb-speed", default=None, help="real only: cap the camera USB speed, e.g. HIGH (USB2) or SUPER" ) + parser.add_argument( + "--manus-calibration-dir", + default=None, + metavar="DIR", + help="enable Manus glove calibration: the managed directory on this host where new MCAL files are " + "stored and from which capture selects them by basename", + ) + parser.add_argument( + "--manus-calibrator", + default=None, + metavar="PATH", + help="the manus_glove_calibrator executable (default: the Manus install under " + "$ISAAC_ROOT/install/plugins)", + ) return parser +def _calibration_config(args: argparse.Namespace) -> CalibrationConfig | None: + """The calibration setup from ``--manus-calibration-dir``; None leaves calibration and file selection off. + + The directory is always kept, so capture can select saved MCAL files. An explicit ``--manus-calibrator`` + that is not an executable is a startup error. A missing default worker only disables generating new + calibrations, with a note so the operator knows why. + """ + if args.manus_calibration_dir is None: + if args.manus_calibrator is not None: + raise ValueError("--manus-calibrator needs --manus-calibration-dir") + return None + worker = find_worker(args.manus_calibrator) + if worker is None: + if args.manus_calibrator is not None: + raise ValueError(f"--manus-calibrator {args.manus_calibrator} is not an executable file") + print( + "capture-server: no manus_glove_calibrator found under $ISAAC_ROOT/install/plugins/manus; " + "generating new calibrations is unavailable (pass --manus-calibrator PATH); saved files in " + "the calibration directory can still be selected for capture", + file=sys.stderr, + ) + return CalibrationConfig(directory=Path(args.manus_calibration_dir), worker_path=worker) + + def _station_cameras(args: argparse.Namespace) -> dict[str, StationCamera] | None: """The ``--camera NAME=DEVICE_ID`` entries sharing the global size/fps/quality; None when none given.""" if not args.camera: @@ -1711,6 +1934,7 @@ def _replay_setup(args: argparse.Namespace) -> tuple[ServerConfig, Any]: host=args.host, port=args.port, cameras=replay_station_cameras(episode), + calibration=_calibration_config(args), data_note=( f" -- REPLAY of recorded episode {episode.path} (streams: {', '.join(streams)}; loops every " f"{episode.period:.1f}s). Recorded data, not live hardware; reported to clients as MOCK mode." @@ -1761,6 +1985,7 @@ def _gmsl_setup(args: argparse.Namespace) -> tuple[ServerConfig, Any]: host=args.host, port=args.port, cameras=stations, + calibration=_calibration_config(args), data_note=( " -- GMSL cameras from the frame feed at " f"{args.gmsl_ipc}: " @@ -1788,6 +2013,7 @@ async def _run_cli(args: argparse.Namespace) -> int: port=args.port, connect_delay=args.connect_delay, cameras=_station_cameras(args), + calibration=_calibration_config(args), ) # The one place the implementation is chosen, before any session is accepted. source_factory = build_source_factory( diff --git a/examples/egocap/capture_server/scripts/calibrate.py b/examples/egocap/capture_server/scripts/calibrate.py new file mode 100644 index 0000000000..492291d94a --- /dev/null +++ b/examples/egocap/capture_server/scripts/calibrate.py @@ -0,0 +1,69 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""Calibrate Manus gloves through a CaptureServer. + + python calibrate.py ws://station:8082 + +Prompts are printed, and a step starts only when the operator presses Enter. The calibration files are written +into the server's --manus-calibration-dir and stay on the server host; only their basenames come back. +""" + +from __future__ import annotations + +import asyncio +import sys + +from capture_server.client import CalibrationStopped, CaptureClient +from capture_server.protocol import CalibrationPhase, CalibrationStopReason, HandSide + + +async def calibrate(url: str) -> CalibrationStopped: + """Run one calibration for both gloves; the operator confirms each step at the keyboard.""" + client = CaptureClient(url) + loop = asyncio.get_running_loop() + finished: asyncio.Future[CalibrationStopped] = loop.create_future() + offered: set[str] = set() + + async def ask(token: str) -> None: + await loop.run_in_executor(None, input, " press Enter to start this step (Ctrl-C cancels) ") + await client.confirm_calibration_step(step_token=token) + + def on_status(status) -> None: + step = status.step + if status.phase is CalibrationPhase.AWAITING_STEP and step is not None and step.token not in offered: + offered.add(step.token) + print(f"{step.side.name} glove, step {step.index + 1} of {step.count}: {step.title}") + print(f" {step.instruction}") + if step.continuous: + print(f" (about {step.estimated_seconds:.0f} s; the SDK gives no completion signal)") + # Scheduled, not awaited: a callback must never wait on a later callback. + asyncio.ensure_future(ask(step.token)) + + await client.start_calibration( + sides=(HandSide.LEFT, HandSide.RIGHT), + on_status=on_status, + on_stopped=lambda stopped: finished.set_result(stopped), + ) + try: + return await finished + except asyncio.CancelledError: + await client.stop_calibration() + raise + + +async def main(url: str) -> int: + stopped = await calibrate(url) + if stopped.reason is not CalibrationStopReason.COMPLETED or stopped.result is None: + print(f"calibration did not complete: {stopped.reason!r} {stopped.error_message}") + return 1 + files = {f.side: f.filename for f in stopped.result.files} + print(f"calibration files on the server: {files}") + return 0 + + +if __name__ == "__main__": + if len(sys.argv) != 2: + sys.exit(__doc__) + sys.exit(asyncio.run(main(sys.argv[1]))) diff --git a/examples/egocap/pyproject.toml b/examples/egocap/pyproject.toml index 8789a81fc6..d286386a5a 100644 --- a/examples/egocap/pyproject.toml +++ b/examples/egocap/pyproject.toml @@ -32,6 +32,8 @@ dependencies = [ # decode/encode MP4 files on disk. # --- capture_server --- "websockets>=17.2", + "isaaccapture-manus-calibration", # Manus calibration worker supervisor + MCAL storage (standard library + # only); a path dependency on the library beside the native worker. # --- backend (egocap_web) --- "fastapi>=0.115", "uvicorn[standard]>=0.30", @@ -139,3 +141,8 @@ include = [ # Marks this pyproject.toml as its own workspace root, so `uv sync`/`uv lock` run from inside this # tree do not discover an ancestor pyproject.toml and pull it into the lockfile's manifest metadata. members = [] + +[tool.uv.sources] +# Installed (not editable) from the checkout, so a deployed CaptureServer never needs the checkout's +# examples/ or src/ trees at run time. +isaaccapture-manus-calibration = { path = "../../src/plugins/manus/calibration/python" } diff --git a/examples/egocap/tests/capture_server/calibration_fake_worker.py b/examples/egocap/tests/capture_server/calibration_fake_worker.py new file mode 100644 index 0000000000..edeaa1899c --- /dev/null +++ b/examples/egocap/tests/capture_server/calibration_fake_worker.py @@ -0,0 +1,216 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""A scripted stand-in for ``manus_glove_calibrator`` speaking the worker contract, for CaptureServer tests. + +Behavior comes from the JSON in ``CAL_FAKE_SCENARIO``; ``CAL_FAKE_LOG`` names a file that gets one line per +command the worker reads, so a test can count what reached it. Run as ``python calibration_fake_worker.py +--ipc-fd=N``. Scenario keys: + +``steps`` list of ``{"title", "instruction", "time", "continuous", "estimated"}`` (default: a hold and + a 0.2 s motion). ``time`` > 0 is a timed motion; ``continuous`` models a negative SDK time. +``slow_confirm`` seconds to wait before accepting a confirmed step (still honors cancel). +``slow_cancel`` seconds to take stopping after a cancel. +``fail_at_step`` ``[side, index]``: report a failed result at that step after accepting it. +``unreleased`` true: every result reports ``released: false`` (the SDK could not be shut down). +""" + +from __future__ import annotations + +import hashlib +import json +import os +import select +import socket +import sys +import time + +SCENARIO = json.loads(os.environ.get("CAL_FAKE_SCENARIO", "{}")) +LOG_PATH = os.environ.get("CAL_FAKE_LOG") +STEPS = SCENARIO.get( + "steps", + [ + {"title": "Flat hand", "instruction": "Lay your hand flat.", "time": 0.0}, + {"title": "Wave", "instruction": "Wave your fingers.", "time": 0.2}, + ], +) +RELEASED = not SCENARIO.get("unreleased", False) +GLOVE_IDS = {"left": 11, "right": 22} + + +class Cancelled(Exception): + pass + + +class Channel: + def __init__(self, sock: socket.socket) -> None: + self.sock = sock + self.buffer = b"" + self.attempt: str | None = None + + def send(self, message: dict) -> None: + self.sock.sendall((json.dumps({"v": 1, **message}) + "\n").encode()) + + def status(self, phase: str, gloves: list) -> None: + self.send({"type": "status", "attempt": self.attempt, "phase": phase, "gloves": gloves}) + + def poll(self, timeout: float) -> dict | None: + while b"\n" not in self.buffer: + ready, _, _ = select.select([self.sock], [], [], timeout) + if not ready: + return None + chunk = self.sock.recv(4096) + if not chunk: + raise Cancelled + self.buffer += chunk + line, self.buffer = self.buffer.split(b"\n", 1) + command = json.loads(line) + if LOG_PATH: + with open(LOG_PATH, "a", encoding="utf-8") as log: + log.write(command["type"] + "\n") + return command + + def sleep(self, seconds: float) -> None: + """Sleep while still honoring a cancel; other commands read meanwhile are only logged.""" + deadline = time.monotonic() + seconds + while time.monotonic() < deadline: + command = self.poll(0.05) + if command is not None and command["type"] == "cancel": + raise Cancelled + + def read(self, *types: str) -> dict: + while True: + command = self.poll(30.0) + if command is None: + raise SystemExit("fake worker timed out waiting for a command") + if command["type"] == "cancel": + raise Cancelled + if command["type"] in types: + return command + + +def prompt(channel: Channel, attempt: str, side: str, index: int, step: dict) -> str: + token = f"{side}-{index}" + channel.send( + { + "type": "prompt", + "attempt": attempt, + "token": token, + "side": side, + "glove_id": GLOVE_IDS[side], + "index": index, + "count": len(STEPS), + "title": step["title"], + "instruction": step["instruction"], + "mode": "motion" if step["time"] > 0 else "hold", + "duration_s": float(step["time"]), + "continuous": bool(step.get("continuous", False)), + "estimated_s": float(step.get("estimated", 0.0)), + } + ) + return token + + +def confirm(channel: Channel, attempt: str, token: str, step: dict) -> None: + while True: + command = channel.read("confirm") + if command["token"] != token: + channel.send( + { + "type": "rejected", + "attempt": attempt, + "command": "confirm", + "reason": "stale_token", + "message": "not the offered prompt", + "token": command["token"], + } + ) + continue + if SCENARIO.get("slow_confirm"): + channel.sleep(SCENARIO["slow_confirm"]) + channel.send({"type": "step_accepted", "attempt": attempt, "token": token}) + break + if step["time"] > 0: + channel.status("capturing_step", []) + elapsed = 0.0 + while elapsed < step["time"]: + channel.sleep(0.1) + elapsed = min(step["time"], elapsed + 0.1) + channel.send({"type": "step_progress", "attempt": attempt, "token": token, "elapsed_s": elapsed}) + + +def export(channel: Channel, attempt: str, staging: str, side: str) -> dict: + data = (b"MCAL-" + side.encode()) * 64 + name = f"{attempt}_{side}.mcal" + path = os.path.join(staging, name) + with open(path + ".part", "wb") as handle: + handle.write(data) + os.rename(path + ".part", path) + event = { + "type": "exported", + "attempt": attempt, + "side": side, + "glove_id": GLOVE_IDS[side], + "file": name, + "bytes": len(data), + "sha256": hashlib.sha256(data).hexdigest(), + } + channel.send(event) + return event + + +def run(channel: Channel, start: dict) -> int: + attempt = start["attempt"] + channel.attempt = attempt + sides = start["sides"] + gloves = [{"side": s, "glove_id": GLOVE_IDS[s]} for s in sides] + channel.status("discovering", []) + channel.status("discovering", gloves) + exported = [] + for side in sides: + for index, step in enumerate(STEPS): + token = prompt(channel, attempt, side, index, step) + channel.status("awaiting_step", gloves) + confirm(channel, attempt, token, step) + if SCENARIO.get("fail_at_step") == [side, index]: + channel.send( + { + "type": "result", + "attempt": attempt, + "outcome": "failed", + "released": RELEASED, + "error": { + "code": "step_failed", + "message": "the SDK refused the step", + "side": side, + "step": index, + }, + } + ) + return 1 + exported.append(export(channel, attempt, start["staging_dir"], side)) + channel.status("saving", gloves) + files = [{k: e[k] for k in ("side", "file", "bytes", "sha256")} for e in exported] + channel.send( + {"type": "result", "attempt": attempt, "outcome": "completed", "released": RELEASED, "files": files} + ) + return 0 + + +def main() -> int: + fd = next(int(a.split("=", 1)[1]) for a in sys.argv[1:] if a.startswith("--ipc-fd=")) + channel = Channel(socket.socket(fileno=fd)) + channel.send({"type": "ready", "worker_version": "fake-1", "sdk_version": None}) + try: + start = channel.read("start") + return run(channel, start) + except Cancelled: + time.sleep(float(SCENARIO.get("slow_cancel", 0))) + channel.status("stopping", []) + channel.send({"type": "result", "attempt": channel.attempt, "outcome": "cancelled", "released": RELEASED}) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/examples/egocap/tests/capture_server/calibration_helpers.py b/examples/egocap/tests/capture_server/calibration_helpers.py new file mode 100644 index 0000000000..a64f9d41ad --- /dev/null +++ b/examples/egocap/tests/capture_server/calibration_helpers.py @@ -0,0 +1,232 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""Shared pieces for the calibration integration tests: a scripted fake worker launched as a real process, raw +websocket helpers, and a driver that confirms prompts through the real CaptureClient API.""" + +from __future__ import annotations + +import asyncio +import contextlib +import dataclasses +import json +import sys +from collections.abc import Awaitable, Callable +from pathlib import Path +from typing import Any + +import websockets + +from capture_server import protocol as p +from capture_server.calibration import CalibrationConfig +from capture_server.client import CalibrationStopped, CaptureClient +from capture_server.server import ServerConfig +from tests.support.capture_server import RunningServer, start_capture_server, stop_capture_server + +FAKE_WORKER = Path(__file__).with_name("calibration_fake_worker.py") +LEFT, RIGHT = p.HandSide.LEFT, p.HandSide.RIGHT +BOTH = (LEFT, RIGHT) +CONTINUOUS_STEPS = [ + { + "title": "Make a fist", + "instruction": "Lift up your hand and make a fist.", + "time": 0, + "continuous": True, + "estimated": 3.0, + }, + { + "title": "Pinch", + "instruction": "Pinch your thumb and index finger.", + "time": 0, + "continuous": True, + "estimated": 2.0, + }, +] + + +@dataclasses.dataclass +class Station: + """A running CaptureServer with the fake calibration worker configured.""" + + url: str + port: int + directory: Path + log: Path + running: RunningServer + worker_path: Path + + @property + def server(self): + return self.running.server + + def commands(self) -> list[str]: + """The command types that reached the fake worker, in order.""" + return self.log.read_text().split() if self.log.exists() else [] + + +def install_fake_worker(tmp_path: Path, monkeypatch, scenario: dict | None) -> tuple[Path, Path]: + """A wrapper executable that runs the fake worker with ``scenario``; returns (executable, command log).""" + log = tmp_path / "worker-commands.log" + monkeypatch.setenv("CAL_FAKE_SCENARIO", json.dumps(scenario or {})) + monkeypatch.setenv("CAL_FAKE_LOG", str(log)) + wrapper = tmp_path / "fake-calibrator" + wrapper.write_text( + f"#!{sys.executable}\nimport runpy\nrunpy.run_path({str(FAKE_WORKER)!r}, run_name='__main__')\n" + ) + wrapper.chmod(0o755) + return wrapper, log + + +StationFactory = Callable[..., Awaitable[Station]] + + +async def start_station( + tmp_path: Path, + monkeypatch, + port: int, + *, + scenario: dict | None = None, + calibration: bool = True, + config: ServerConfig | None = None, +) -> Station: + worker, log = install_fake_worker(tmp_path, monkeypatch, scenario) + directory = tmp_path / "managed-mcal" + base = config or ServerConfig() + if calibration: + base = dataclasses.replace(base, calibration=CalibrationConfig(worker_path=worker, directory=directory)) + running = await start_capture_server(port, config=base) + return Station(f"ws://127.0.0.1:{port}", port, directory, log, running, worker) + + +async def stop_station(station: Station) -> None: + await stop_capture_server(station.running) + + +# --------------------------------------------------------------------------- +# Raw websocket +# --------------------------------------------------------------------------- + + +async def connect(url: str): + return await websockets.connect(url) + + +async def send(ws, message: dict | Any) -> None: + await ws.send(json.dumps(message.to_json() if hasattr(message, "to_json") else message)) + + +async def recv_event(ws, timeout: float = 5.0): + """The next JSON message parsed into its typed event; binary capture frames are skipped.""" + while True: + raw = await asyncio.wait_for(ws.recv(), timeout) + if isinstance(raw, bytes): + continue + return p.parse_server_message(json.loads(raw)) + + +async def recv_until(ws, predicate, timeout: float = 10.0): + """Read events until one matches ``predicate``; returns (match, everything read so far).""" + seen: list = [] + + async def _read(): + while True: + event = await recv_event(ws, timeout) + seen.append(event) + if predicate(event): + return event + + return await asyncio.wait_for(_read(), timeout), seen + + +def is_type(*types): + return lambda event: isinstance(event, types) + + +def prompt_status(event) -> bool: + return ( + isinstance(event, p.CalibrationStatus) + and event.phase is p.CalibrationPhase.AWAITING_STEP + and event.step is not None + ) + + +async def start_over_raw(ws, session_id: str = "cal-1", sides=(LEFT,)): + await send(ws, p.StartCalibrationRequest(session_id, tuple(sides))) + return await recv_event(ws) + + +async def next_prompt(ws, timeout: float = 10.0) -> p.CalibrationStatus: + status, _ = await recv_until(ws, prompt_status, timeout) + return status + + +async def terminal_of(ws, timeout: float = 15.0) -> p.CalibrationStopped: + stopped, _ = await recv_until(ws, is_type(p.CalibrationStopped), timeout) + return stopped + + +# --------------------------------------------------------------------------- +# Client driver +# --------------------------------------------------------------------------- + + +class CalibrationRun: + """Runs one calibration through the real CaptureClient, confirming each prompt unless told not to.""" + + def __init__(self, url: str, *, auto_confirm: bool = True) -> None: + self.client = CaptureClient(url) + self.auto_confirm = auto_confirm + self.started: list[p.CalibrationStarted] = [] + self.statuses: list[p.CalibrationStatus] = [] + self.stopped: list[CalibrationStopped] = [] + self.finished = asyncio.Event() + self.confirmed: list[str] = [] + self.errors: list[BaseException] = [] + self._seen: set[str] = set() + self._tasks: set[asyncio.Task] = set() + + def prompts(self) -> list[p.CalibrationStep]: + """Each distinct offered step, in order.""" + out: dict[str, p.CalibrationStep] = {} + for status in self.statuses: + if status.step is not None and status.phase is p.CalibrationPhase.AWAITING_STEP: + out.setdefault(status.step.token, status.step) + return list(out.values()) + + def _on_status(self, status: p.CalibrationStatus) -> None: + self.statuses.append(status) + step = status.step + if not self.auto_confirm or step is None or status.phase is not p.CalibrationPhase.AWAITING_STEP: + return + if step.token in self._seen: + return + self._seen.add(step.token) + task = asyncio.ensure_future(self._confirm(step.token)) + self._tasks.add(task) + task.add_done_callback(self._tasks.discard) + + async def _confirm(self, token: str) -> None: + try: + await self.client.confirm_calibration_step(step_token=token) + self.confirmed.append(token) + except Exception as exc: # noqa: BLE001 -- recorded for the test to assert on + self.errors.append(exc) + + async def start(self, sides=(LEFT,)) -> None: + await self.client.start_calibration( + sides=sides, + on_started=self.started.append, + on_status=self._on_status, + on_stopped=lambda s: (self.stopped.append(s), self.finished.set()), + ) + + async def wait(self, timeout: float = 20.0) -> CalibrationStopped: + await asyncio.wait_for(self.finished.wait(), timeout) + return self.stopped[0] + + async def aclose(self) -> None: + with contextlib.suppress(Exception): + await self.client.stop_calibration() + for task in list(self._tasks): + task.cancel() diff --git a/examples/egocap/tests/capture_server/conftest.py b/examples/egocap/tests/capture_server/conftest.py index 445eb1ff1f..b0d84cbdb3 100644 --- a/examples/egocap/tests/capture_server/conftest.py +++ b/examples/egocap/tests/capture_server/conftest.py @@ -26,3 +26,29 @@ async def real_mock_server(unused_tcp_port: int) -> AsyncIterator[str]: yield f"ws://127.0.0.1:{unused_tcp_port}" finally: await stop_capture_server(server) + + +@pytest.fixture +async def make_station(tmp_path, monkeypatch, unused_tcp_port_factory): + """Starts a CaptureServer with the fake calibration worker; every station is stopped on teardown.""" + from tests.capture_server.calibration_helpers import start_station, stop_station + + stations = [] + + async def make(*, scenario=None, calibration=True, config=None): + station = await start_station( + tmp_path, + monkeypatch, + unused_tcp_port_factory(), + scenario=scenario, + calibration=calibration, + config=config, + ) + stations.append(station) + return station + + try: + yield make + finally: + for station in stations: + await stop_station(station) diff --git a/examples/egocap/tests/capture_server/test_calibration_cli.py b/examples/egocap/tests/capture_server/test_calibration_cli.py new file mode 100644 index 0000000000..e9822d73e1 --- /dev/null +++ b/examples/egocap/tests/capture_server/test_calibration_cli.py @@ -0,0 +1,195 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""The ``--manus-calibration-dir`` / ``--manus-calibrator`` options and worker discovery. + +The managed directory is always kept when it is given, so saved files stay selectable for capture; only a +missing worker disables generating a calibration. An explicit worker path that cannot run is a startup error. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from capture_server import server as server_module +from capture_server.calibration import WORKER_NAME, CalibrationConfig, find_worker +from capture_server.server import _build_arg_parser, _calibration_config, main + + +def _args(*flags: str): + return _build_arg_parser().parse_args(["--mock", *flags]) + + +def _executable(path: Path) -> Path: + path.write_text("#!/bin/sh\nexit 0\n") + path.chmod(0o755) + return path + + +def _install_worker(isaac_root: Path, *, executable: bool = True) -> Path: + """Where the Manus component installs the worker under ISAAC_ROOT.""" + directory = isaac_root / "install" / "plugins" / "manus" + directory.mkdir(parents=True) + path = directory / WORKER_NAME + path.write_text("#!/bin/sh\nexit 0\n") + path.chmod(0o755 if executable else 0o644) + return path + + +# --------------------------------------------------------------------------- +# Option parsing +# --------------------------------------------------------------------------- + + +def test_the_calibration_options_default_to_off(): + args = _args() + + assert args.manus_calibration_dir is None + assert args.manus_calibrator is None + + +def test_the_calibration_options_round_trip(tmp_path): + args = _args("--manus-calibration-dir", str(tmp_path), "--manus-calibrator", "/opt/worker") + + assert args.manus_calibration_dir == str(tmp_path) + assert args.manus_calibrator == "/opt/worker" + + +# --------------------------------------------------------------------------- +# _calibration_config +# --------------------------------------------------------------------------- + + +def test_no_calibration_options_leave_calibration_and_file_selection_off(): + assert _calibration_config(_args()) is None + + +def test_a_directory_without_a_findable_worker_is_kept_and_only_generation_is_disabled( + tmp_path, monkeypatch, capsys +): + monkeypatch.setattr(server_module, "find_worker", lambda override: None) + + config = _calibration_config(_args("--manus-calibration-dir", str(tmp_path))) + + assert config == CalibrationConfig(directory=tmp_path, worker_path=None) + err = capsys.readouterr().err + assert "generating new calibrations is unavailable" in err + assert "can still be selected for capture" in err + + +def test_a_directory_with_a_found_worker_enables_both_without_a_note(tmp_path, monkeypatch, capsys): + worker = _executable(tmp_path / "found-worker") + monkeypatch.setattr(server_module, "find_worker", lambda override: worker) + + config = _calibration_config(_args("--manus-calibration-dir", str(tmp_path / "mcal"))) + + assert config == CalibrationConfig(directory=tmp_path / "mcal", worker_path=worker) + assert capsys.readouterr().err == "" + + +def test_an_explicit_worker_and_a_directory_set_both(tmp_path): + worker = _executable(tmp_path / "my-worker") + + config = _calibration_config( + _args("--manus-calibration-dir", str(tmp_path / "mcal"), "--manus-calibrator", str(worker)) + ) + + assert config == CalibrationConfig(directory=tmp_path / "mcal", worker_path=worker) + + +def test_an_explicit_worker_that_does_not_exist_is_an_error_not_a_silent_fallback(tmp_path, monkeypatch): + # A worker installed under ISAAC_ROOT must not hide a wrong explicit path. + _install_worker(tmp_path / "root") + monkeypatch.setenv("ISAAC_ROOT", str(tmp_path / "root")) + + with pytest.raises(ValueError, match="is not an executable file"): + _calibration_config( + _args("--manus-calibration-dir", str(tmp_path / "mcal"), "--manus-calibrator", "/nonexistent/worker") + ) + + +def test_an_explicit_worker_that_is_not_executable_is_an_error(tmp_path): + plain = tmp_path / "not-executable" + plain.write_text("x") + plain.chmod(0o644) + + with pytest.raises(ValueError, match="is not an executable file"): + _calibration_config( + _args("--manus-calibration-dir", str(tmp_path / "mcal"), "--manus-calibrator", str(plain)) + ) + + +def test_a_worker_without_a_directory_is_an_error(tmp_path): + worker = _executable(tmp_path / "my-worker") + + with pytest.raises(ValueError, match="needs --manus-calibration-dir"): + _calibration_config(_args("--manus-calibrator", str(worker))) + + +@pytest.mark.parametrize( + "flags", + [ + ["--manus-calibrator", "/nonexistent/worker"], + ["--manus-calibration-dir", "{dir}", "--manus-calibrator", "/nonexistent/worker"], + ], + ids=["worker-without-directory", "missing-explicit-worker"], +) +def test_a_bad_calibration_option_exits_with_status_two_and_the_reason(tmp_path, capsys, flags): + flags = [flag.replace("{dir}", str(tmp_path)) for flag in flags] + + assert main(["--mock", *flags]) == 2 + + err = capsys.readouterr().err + assert err.startswith("capture-server: ") + assert "--manus-calibrator" in err + + +# --------------------------------------------------------------------------- +# find_worker +# --------------------------------------------------------------------------- + + +def test_find_worker_prefers_an_explicit_executable_over_the_install(tmp_path, monkeypatch): + _install_worker(tmp_path / "root") + monkeypatch.setenv("ISAAC_ROOT", str(tmp_path / "root")) + explicit = _executable(tmp_path / "explicit-worker") + + assert find_worker(explicit) == explicit + + +def test_find_worker_does_not_fall_back_when_the_explicit_path_is_unusable(tmp_path, monkeypatch): + _install_worker(tmp_path / "root") + monkeypatch.setenv("ISAAC_ROOT", str(tmp_path / "root")) + plain = tmp_path / "not-executable" + plain.write_text("x") + plain.chmod(0o644) + + assert find_worker(plain) is None + assert find_worker(tmp_path / "missing") is None + + +def test_find_worker_falls_back_to_the_manus_install_under_isaac_root(tmp_path, monkeypatch): + installed = _install_worker(tmp_path / "root") + monkeypatch.setenv("ISAAC_ROOT", str(tmp_path / "root")) + + assert find_worker(None) == installed + + +def test_find_worker_ignores_a_non_executable_file_in_the_install(tmp_path, monkeypatch): + _install_worker(tmp_path / "root", executable=False) + monkeypatch.setenv("ISAAC_ROOT", str(tmp_path / "root")) + + assert find_worker(None) is None + + +def test_find_worker_without_isaac_root_finds_nothing_even_if_the_worker_is_on_path(tmp_path, monkeypatch): + path_dir = tmp_path / "bin" + path_dir.mkdir() + _executable(path_dir / WORKER_NAME) + monkeypatch.delenv("ISAAC_ROOT", raising=False) + monkeypatch.setenv("PATH", str(path_dir)) + + assert find_worker(None) is None diff --git a/examples/egocap/tests/capture_server/test_calibration_client.py b/examples/egocap/tests/capture_server/test_calibration_client.py new file mode 100644 index 0000000000..51c074d157 --- /dev/null +++ b/examples/egocap/tests/capture_server/test_calibration_client.py @@ -0,0 +1,707 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""CaptureClient's calibration API against small scripted servers, so each test controls exactly what the +server sends: acceptance, rejection, statuses, acknowledgements, terminal messages, transport loss and +protocol violations.""" + +from __future__ import annotations + +import asyncio +import contextlib +import json +from collections.abc import AsyncIterator, Awaitable, Callable +from typing import Any + +import pytest +import websockets + +from capture_server import protocol as p +from capture_server.client import ( + CalibrationEndedError, + CalibrationStopped, + CaptureClient, + RequestRejectedError, + StoppedBeforeAcceptedError, +) +from tests.capture_server.calibration_helpers import BOTH, LEFT, RIGHT, CalibrationRun +from tests.support.capture_server import poll_until + +Script = Callable[[Any, p.StartCalibrationRequest], Awaitable[None]] + + +@contextlib.asynccontextmanager +async def scripted_server(port: int, script: Script) -> AsyncIterator[str]: + """A server that reads one StartCalibration and hands the connection to ``script``.""" + + async def handler(ws) -> None: + request = p.parse_start_calibration(json.loads(await ws.recv())) + await script(ws, request) + + server = await websockets.serve(handler, "127.0.0.1", port) + try: + yield f"ws://127.0.0.1:{port}" + finally: + server.close() + await server.wait_closed() + + +async def push(ws, event) -> None: + await ws.send(json.dumps(event.to_json())) + + +def step(token: str = "left-0", **overrides) -> p.CalibrationStep: + fields = dict( + token=token, + side=LEFT, + index=0, + count=2, + title="Make a fist", + instruction="Lift up your hand and make a fist.", + mode=p.CalibrationStepMode.HOLD, + duration_seconds=0.0, + continuous=False, + estimated_seconds=0.0, + ) + fields.update(overrides) + return p.CalibrationStep(**fields) + + +def status(session_id: str, offered: p.CalibrationStep | None = None, phase=None) -> p.CalibrationStatus: + phase = phase or (p.CalibrationPhase.AWAITING_STEP if offered else p.CalibrationPhase.CONNECTING) + return p.CalibrationStatus(session_id, phase, (p.CalibrationGlove(LEFT, 11),), offered, None) + + +def stopped(session_id: str, reason=p.CalibrationStopReason.CANCELLED, result=None, message=None): + return p.CalibrationStopped(session_id, reason, result, message) + + +class Recorder: + """Collects callback values and whether callbacks ever overlapped.""" + + def __init__(self) -> None: + self.started: list[p.CalibrationStarted] = [] + self.statuses: list[p.CalibrationStatus] = [] + self.stopped: list[CalibrationStopped] = [] + self.done = asyncio.Event() + self.running = 0 + self.max_running = 0 + + def on_started(self, event) -> None: + self.started.append(event) + + async def on_status(self, event) -> None: + self.running += 1 + self.max_running = max(self.max_running, self.running) + await asyncio.sleep(0.01) + self.statuses.append(event) + self.running -= 1 + + def on_stopped(self, event: CalibrationStopped) -> None: + self.stopped.append(event) + self.done.set() + + +async def start(client: CaptureClient, recorder: Recorder, sides=(LEFT,)) -> None: + await client.start_calibration( + sides=sides, on_started=recorder.on_started, on_status=recorder.on_status, on_stopped=recorder.on_stopped + ) + + +# --------------------------------------------------------------------------- +# Acceptance and rejection +# --------------------------------------------------------------------------- + + +async def test_start_calibration_returns_after_acceptance_not_after_completion(unused_tcp_port): + release = asyncio.Event() + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await release.wait() + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + assert len(recorder.started) == 1 + assert recorder.stopped == [] + release.set() + await asyncio.wait_for(recorder.done.wait(), 5) + + +async def test_a_rejected_start_raises_and_calls_no_callback_and_leaves_the_client_free(unused_tcp_port): + async def script(ws, request): + await push( + ws, + p.RequestRejected(request.session_id, p.MessageType.START_CALIBRATION, p.RejectionReason.BUSY, "busy"), + ) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + with pytest.raises(RequestRejectedError) as excinfo: + await start(client, recorder) + assert excinfo.value.reason is p.RejectionReason.BUSY + assert str(excinfo.value) == "busy" + assert (recorder.started, recorder.statuses, recorder.stopped) == ([], [], []) + assert client._calibration is None + with pytest.raises(RequestRejectedError): + await start(client, recorder) + + +async def test_a_calibration_started_for_other_sides_is_a_protocol_error_before_acceptance(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, (RIGHT,))) + + async with scripted_server(unused_tcp_port, script) as url: + with pytest.raises(p.ProtocolError): + await start(CaptureClient(url), Recorder()) + + +async def test_a_wrong_first_message_is_a_protocol_error_before_acceptance(unused_tcp_port): + async def script(ws, request): + await push(ws, status(request.session_id)) + + async with scripted_server(unused_tcp_port, script) as url: + recorder = Recorder() + with pytest.raises(p.ProtocolError): + await start(CaptureClient(url), recorder) + assert recorder.stopped == [] + + +async def test_a_stop_before_acceptance_raises_stopped_before_accepted_and_calls_no_stopped_callback( + unused_tcp_port, +): + never = asyncio.Event() + + async def script(ws, request): + await never.wait() + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + pending = asyncio.ensure_future(start(client, recorder)) + await asyncio.sleep(0.1) + await client.stop_calibration() + with pytest.raises(StoppedBeforeAcceptedError): + await asyncio.wait_for(pending, 5) + assert recorder.stopped == [] + never.set() + + +# --------------------------------------------------------------------------- +# Argument validation and local exclusivity +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("sides", [(), (LEFT, LEFT), (1,), ("left",)]) +async def test_start_calibration_rejects_invalid_sides_before_connecting(sides): + client = CaptureClient("ws://127.0.0.1:1") + with pytest.raises(ValueError, match="start_calibration"): + await client.start_calibration(sides=sides) # type: ignore[arg-type] + + +async def test_start_calibration_rejects_a_non_callable_callback(): + client = CaptureClient("ws://127.0.0.1:1") + with pytest.raises(ValueError, match="on_status"): + await client.start_calibration(sides=(LEFT,), on_status="nope") # type: ignore[arg-type] + + +async def test_a_second_start_calibration_while_one_is_active_raises_runtime_error(unused_tcp_port): + hold = asyncio.Event() + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await hold.wait() + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + with pytest.raises(RuntimeError, match="active"): + await start(client, Recorder()) + with pytest.raises(RuntimeError, match="active"): + await client.start(sources=(p.SourceType.HEAD,)) + hold.set() + await asyncio.wait_for(recorder.done.wait(), 5) + assert client._calibration is None + + +async def test_start_calibration_while_a_capture_is_active_raises_runtime_error(make_station): + station = await make_station() + done = asyncio.Event() + client = CaptureClient(station.url) + await client.start(sources=(p.SourceType.HEAD,), on_stopped=lambda _r: done.set()) + try: + with pytest.raises(RuntimeError, match="active"): + await client.start_calibration(sides=(LEFT,)) + finally: + await client.stop() + await asyncio.wait_for(done.wait(), 10) + + +async def test_confirm_without_an_accepted_calibration_raises_runtime_error(): + client = CaptureClient("ws://127.0.0.1:1") + with pytest.raises(RuntimeError, match="no accepted calibration"): + await client.confirm_calibration_step(step_token="left-0") + + +async def test_confirm_rejects_an_empty_token(unused_tcp_port): + hold = asyncio.Event() + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await hold.wait() + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + with pytest.raises(ValueError): + await client.confirm_calibration_step(step_token="") + hold.set() + await asyncio.wait_for(recorder.done.wait(), 5) + + +async def test_stop_calibration_without_an_attempt_is_a_no_op(): + await CaptureClient("ws://127.0.0.1:1").stop_calibration() + + +# --------------------------------------------------------------------------- +# Statuses, acknowledgements and rejections of confirmations +# --------------------------------------------------------------------------- + + +async def test_statuses_are_delivered_in_order_and_callbacks_never_overlap(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + for index in range(4): + await push(ws, status(request.session_id, step(f"left-{index}", index=index, count=4))) + await asyncio.sleep(0.3) + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + await start(CaptureClient(url), recorder) + await asyncio.wait_for(recorder.done.wait(), 5) + assert [s.step.token for s in recorder.statuses] == ["left-0", "left-1", "left-2", "left-3"] + assert recorder.max_running == 1 + + +async def test_confirm_returns_only_after_the_acceptance_and_a_retry_sends_no_second_request(unused_tcp_port): + received: list[p.ConfirmCalibrationStepRequest] = [] + accept = asyncio.Event() + finish = asyncio.Event() + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, status(request.session_id, step("left-0"))) + async for raw in ws: + message = json.loads(raw) + if message["type"] == int(p.MessageType.CONFIRM_CALIBRATION_STEP): + received.append(p.parse_confirm_calibration_step(message)) + await accept.wait() + await push(ws, p.CalibrationStepAccepted(request.session_id, "left-0")) + await finish.wait() + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + confirming = asyncio.ensure_future(client.confirm_calibration_step(step_token="left-0")) + await poll_until(lambda: received, timeout_s=5) + assert not confirming.done() + accept.set() + await asyncio.wait_for(confirming, 5) + await client.confirm_calibration_step(step_token="left-0") + await asyncio.sleep(0.1) + assert len(received) == 1 + finish.set() + await asyncio.wait_for(recorder.done.wait(), 5) + + +async def test_two_concurrent_confirms_of_one_token_send_one_request_and_both_return(unused_tcp_port): + received: list[str] = [] + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + async for raw in ws: + message = json.loads(raw) + if message["type"] == int(p.MessageType.CONFIRM_CALIBRATION_STEP): + received.append(message["step_token"]) + await asyncio.sleep(0.1) + await push(ws, p.CalibrationStepAccepted(request.session_id, message["step_token"])) + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + await asyncio.wait_for( + asyncio.gather( + client.confirm_calibration_step(step_token="left-0"), + client.confirm_calibration_step(step_token="left-0"), + ), + 5, + ) + assert received == ["left-0"] + await asyncio.wait_for(recorder.done.wait(), 5) + + +async def test_a_rejected_confirm_raises_with_the_reason_and_the_attempt_continues(unused_tcp_port): + finish = asyncio.Event() + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + async for raw in ws: + message = json.loads(raw) + if message["type"] == int(p.MessageType.CONFIRM_CALIBRATION_STEP): + await push( + ws, + p.RequestRejected( + request.session_id, + p.MessageType.CONFIRM_CALIBRATION_STEP, + p.RejectionReason.INVALID_CONFIG, + "that step is not awaiting confirmation", + step_token=message["step_token"], + ), + ) + await finish.wait() + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + with pytest.raises(RequestRejectedError) as excinfo: + await client.confirm_calibration_step(step_token="stale") + assert excinfo.value.reason is p.RejectionReason.INVALID_CONFIG + assert recorder.stopped == [] + assert client._calibration is not None + finish.set() + await asyncio.wait_for(recorder.done.wait(), 5) + assert len(recorder.stopped) == 1 + + +# --------------------------------------------------------------------------- +# Terminal outcomes +# --------------------------------------------------------------------------- + + +async def test_a_completed_terminal_delivers_the_files_and_exactly_one_stopped_callback(unused_tcp_port): + result = p.CalibrationResult((p.McalFile(LEFT, "set_left.mcal"), p.McalFile(RIGHT, "set_right.mcal"))) + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, stopped(request.session_id, p.CalibrationStopReason.COMPLETED, result, None)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + await start(CaptureClient(url), recorder, BOTH) + await asyncio.wait_for(recorder.done.wait(), 5) + await asyncio.sleep(0.1) + assert len(recorder.stopped) == 1 + assert recorder.stopped[0].reason is p.CalibrationStopReason.COMPLETED + assert recorder.stopped[0].result == result + assert recorder.stopped[0].error_message is None + + +@pytest.mark.parametrize( + "reason", + [p.CalibrationStopReason.CANCELLED, p.CalibrationStopReason.FAILED, p.CalibrationStopReason.UNCERTAIN], +) +async def test_non_completed_terminals_carry_the_servers_reason_and_no_files(unused_tcp_port, reason): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, stopped(request.session_id, reason, None, "the server says why")) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + await start(CaptureClient(url), recorder) + await asyncio.wait_for(recorder.done.wait(), 5) + assert recorder.stopped[0].reason is reason + assert recorder.stopped[0].result is None + assert recorder.stopped[0].error_message == "the server says why" + + +async def test_the_client_is_free_again_after_the_terminal(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, stopped(request.session_id)) + + first, second = Recorder(), Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, first) + await asyncio.wait_for(first.done.wait(), 5) + await poll_until(lambda: client._calibration is None, timeout_s=5) + await start(client, second) + await asyncio.wait_for(second.done.wait(), 5) + + +# --------------------------------------------------------------------------- +# Transport loss and protocol violations: an honest, unconfirmed ending +# --------------------------------------------------------------------------- + + +async def test_transport_loss_ends_with_one_unconfirmed_stopped_and_no_files(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, status(request.session_id, step("left-0"))) + await ws.close() + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + await start(CaptureClient(url), recorder) + await asyncio.wait_for(recorder.done.wait(), 5) + await asyncio.sleep(0.1) + assert len(recorder.stopped) == 1 + ending = recorder.stopped[0] + assert ending.reason is None + assert ending.result is None + assert "outcome is unknown" in ending.error_message + + +async def test_a_pending_confirm_is_released_with_calibration_ended_when_the_transport_is_lost(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + async for raw in ws: + if json.loads(raw)["type"] == int(p.MessageType.CONFIRM_CALIBRATION_STEP): + await ws.close() + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + with pytest.raises(CalibrationEndedError): + await asyncio.wait_for(client.confirm_calibration_step(step_token="left-0"), 5) + await asyncio.wait_for(recorder.done.wait(), 5) + assert recorder.stopped[0].reason is None + + +async def test_a_pending_confirm_is_released_when_the_server_reports_the_terminal_first(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + async for raw in ws: + if json.loads(raw)["type"] == int(p.MessageType.CONFIRM_CALIBRATION_STEP): + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + with pytest.raises(CalibrationEndedError): + await asyncio.wait_for(client.confirm_calibration_step(step_token="left-0"), 5) + await asyncio.wait_for(recorder.done.wait(), 5) + assert recorder.stopped[0].reason is p.CalibrationStopReason.CANCELLED + + +@pytest.mark.parametrize( + "bad", + [ + pytest.param(lambda sid: b"binary", id="binary"), + pytest.param(lambda sid: "not json", id="not-json"), + pytest.param(lambda sid: json.dumps({"type": 99, "version": 1}), id="unknown-type"), + pytest.param( + lambda sid: json.dumps( + p.CalibrationStatus("other-id", p.CalibrationPhase.CONNECTING, (), None, None).to_json() + ), + id="other-session", + ), + pytest.param( + lambda sid: json.dumps({"type": int(p.MessageType.SESSION_STATUS), "version": 1, "session_id": sid}), + id="malformed-capture-message", + ), + ], +) +async def test_a_protocol_violation_ends_with_an_unconfirmed_stopped(unused_tcp_port, bad): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await ws.send(bad(request.session_id)) + await asyncio.sleep(1) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + await start(CaptureClient(url), recorder) + await asyncio.wait_for(recorder.done.wait(), 5) + assert len(recorder.stopped) == 1 + assert recorder.stopped[0].reason is None + assert "protocol error" in recorder.stopped[0].error_message + assert recorder.stopped[0].result is None + + +# --------------------------------------------------------------------------- +# Callbacks and stop +# --------------------------------------------------------------------------- + + +async def test_a_callback_exception_requests_cancellation_and_appears_in_the_terminal_diagnostic(unused_tcp_port): + stops = 0 + + async def script(ws, request): + nonlocal stops + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, status(request.session_id)) + async for raw in ws: + if json.loads(raw)["type"] == int(p.MessageType.STOP_CALIBRATION): + stops += 1 + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + + def explode(_event) -> None: + raise ValueError("boom") + + async with scripted_server(unused_tcp_port, script) as url: + await CaptureClient(url).start_calibration( + sides=(LEFT,), on_status=explode, on_stopped=recorder.on_stopped + ) + await asyncio.wait_for(recorder.done.wait(), 5) + assert stops == 1 + assert recorder.stopped[0].reason is p.CalibrationStopReason.CANCELLED + assert "a callback raised" in recorder.stopped[0].error_message + assert "boom" in recorder.stopped[0].error_message + + +async def test_an_on_started_exception_also_cancels_and_is_reported_once(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + async for raw in ws: + if json.loads(raw)["type"] == int(p.MessageType.STOP_CALIBRATION): + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + + def explode(_event) -> None: + raise RuntimeError("bad started handler") + + async with scripted_server(unused_tcp_port, script) as url: + await CaptureClient(url).start_calibration( + sides=(LEFT,), on_started=explode, on_stopped=recorder.on_stopped + ) + await asyncio.wait_for(recorder.done.wait(), 5) + assert len(recorder.stopped) == 1 + assert "bad started handler" in recorder.stopped[0].error_message + + +async def test_an_on_stopped_exception_is_reported_to_the_loop_once_and_not_repeated(unused_tcp_port): + reports: list[dict] = [] + loop = asyncio.get_running_loop() + loop.set_exception_handler(lambda _loop, context: reports.append(context)) + calls = 0 + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, stopped(request.session_id)) + + def explode(_event) -> None: + nonlocal calls + calls += 1 + raise RuntimeError("terminal handler failed") + + try: + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await client.start_calibration(sides=(LEFT,), on_stopped=explode) + await poll_until(lambda: reports, timeout_s=5) + await poll_until(lambda: client._calibration is None, timeout_s=5) + finally: + loop.set_exception_handler(None) + assert calls == 1 + assert len(reports) == 1 + assert "on_stopped" in reports[0]["message"] + + +async def test_stop_calibration_is_idempotent_and_safe_to_call_from_a_callback(unused_tcp_port): + stops = 0 + + async def script(ws, request): + nonlocal stops + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, status(request.session_id)) + async for raw in ws: + if json.loads(raw)["type"] == int(p.MessageType.STOP_CALIBRATION): + stops += 1 + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + client = CaptureClient("ws://placeholder") + + async def stopping_status(event) -> None: + await client.stop_calibration() + await client.stop_calibration() + + async with scripted_server(unused_tcp_port, script) as url: + client._url = url + await client.start_calibration(sides=(LEFT,), on_status=stopping_status, on_stopped=recorder.on_stopped) + await asyncio.wait_for(recorder.done.wait(), 5) + await client.stop_calibration() + assert stops == 1 + assert recorder.stopped[0].reason is p.CalibrationStopReason.CANCELLED + + +async def test_confirm_can_be_awaited_from_inside_a_status_callback_without_deadlock(unused_tcp_port): + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await push(ws, status(request.session_id, step("left-0"))) + async for raw in ws: + message = json.loads(raw) + if message["type"] == int(p.MessageType.CONFIRM_CALIBRATION_STEP): + await push(ws, p.CalibrationStepAccepted(request.session_id, message["step_token"])) + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + confirmed: list[str] = [] + client = CaptureClient("ws://placeholder") + + async def confirming_status(event) -> None: + if event.step is not None: + await client.confirm_calibration_step(step_token=event.step.token) + confirmed.append(event.step.token) + + async with scripted_server(unused_tcp_port, script) as url: + client._url = url + await client.start_calibration(sides=(LEFT,), on_status=confirming_status, on_stopped=recorder.on_stopped) + await asyncio.wait_for(recorder.done.wait(), 5) + assert confirmed == ["left-0"] + + +async def test_an_accepted_attempt_stays_active_until_the_server_ends_it(unused_tcp_port): + hold = asyncio.Event() + + async def script(ws, request): + await push(ws, p.CalibrationStarted(request.session_id, request.sides)) + await hold.wait() + await push(ws, stopped(request.session_id)) + + recorder = Recorder() + async with scripted_server(unused_tcp_port, script) as url: + client = CaptureClient(url) + await start(client, recorder) + await asyncio.sleep(0.1) + assert client._calibration is not None + assert recorder.stopped == [] + hold.set() + await asyncio.wait_for(recorder.done.wait(), 5) + + +# --------------------------------------------------------------------------- +# Through the real server +# --------------------------------------------------------------------------- + + +async def test_the_real_client_and_server_complete_a_calibration_end_to_end(make_station): + station = await make_station() + run = CalibrationRun(station.url) + await run.start(BOTH) + ending = await run.wait() + assert ending.reason is p.CalibrationStopReason.COMPLETED + assert sorted(f.filename for f in ending.result.files) == sorted( + path.name for path in station.directory.glob("*.mcal") + ) diff --git a/examples/egocap/tests/capture_server/test_calibration_module.py b/examples/egocap/tests/capture_server/test_calibration_module.py new file mode 100644 index 0000000000..45cf4b5698 --- /dev/null +++ b/examples/egocap/tests/capture_server/test_calibration_module.py @@ -0,0 +1,480 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""capture_server.calibration without processes: a stub worker and outbox stand in for the supervisor and +the connection's single writer, so the translation of worker events into calibration messages, the terminal +ordering and the file rules are checked directly.""" + +from __future__ import annotations + +import asyncio +from pathlib import Path +from typing import Any + +import pytest +from isaaccapture_manus_calibration import protocol as wp +from isaaccapture_manus_calibration.worker import AttemptError, CalibrationOutcome, OutcomeKind + +from capture_server import calibration as cal +from capture_server import protocol as p + +ATTEMPT = "attempt-1" +LEFT, RIGHT = wp.Side.LEFT, wp.Side.RIGHT + + +# --------------------------------------------------------------------------- +# find_worker +# --------------------------------------------------------------------------- + + +def _executable(path: Path) -> Path: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text("#!/bin/sh\nexit 0\n") + path.chmod(0o755) + return path + + +def test_find_worker_returns_an_executable_override(tmp_path): + worker = _executable(tmp_path / "bin" / "custom-calibrator") + assert cal.find_worker(worker) == worker + + +def test_find_worker_rejects_a_non_executable_or_missing_override(tmp_path): + plain = tmp_path / "plain" + plain.write_text("x") + assert cal.find_worker(plain) is None + assert cal.find_worker(tmp_path / "absent") is None + + +def test_find_worker_looks_under_the_manus_install_of_isaac_root(tmp_path, monkeypatch): + worker = _executable(tmp_path / "install" / "plugins" / "manus" / cal.WORKER_NAME) + monkeypatch.setenv("ISAAC_ROOT", str(tmp_path)) + assert cal.find_worker(None) == worker + + +def test_find_worker_is_none_without_isaac_root_or_override(monkeypatch): + monkeypatch.delenv("ISAAC_ROOT", raising=False) + assert cal.find_worker(None) is None + + +def test_find_worker_is_none_when_the_install_has_no_worker(tmp_path, monkeypatch): + (tmp_path / "install" / "plugins" / "manus").mkdir(parents=True) + monkeypatch.setenv("ISAAC_ROOT", str(tmp_path)) + assert cal.find_worker(None) is None + + +# --------------------------------------------------------------------------- +# CalibrationSession with a stub worker and outbox +# --------------------------------------------------------------------------- + + +class StubOutbox: + """Records every call in one ordered log so cross-channel ordering can be asserted.""" + + def __init__(self) -> None: + self.closed = False + self.log: list[tuple[str, Any]] = [] + + def bind_session(self, stream_ids, *, on_sent, on_fatal) -> None: + self.log.append(("bind", tuple(stream_ids))) + self.on_fatal = on_fatal + + def publish_control(self, data) -> bool: + self.log.append(("control", data)) + return True + + def publish_status(self, data) -> None: + self.log.append(("status", data)) + + async def quiesce(self) -> None: + self.log.append(("quiesce", None)) + + def controls(self, message_type: p.MessageType) -> list[dict]: + return [d for kind, d in self.log if kind == "control" and d["type"] == int(message_type)] + + def statuses(self) -> list[dict]: + return [d for kind, d in self.log if kind == "status"] + + +class StubAttempt: + def __init__(self, on_event, script) -> None: + self.on_event = on_event + self.script = script + self.prompt: wp.Prompt | None = None + self.started_with = None + self.cancel_calls = 0 + self.confirm_calls: list[str] = [] + self.confirm_error: Exception | None = None + self._outcome: asyncio.Future[CalibrationOutcome] = asyncio.get_running_loop().create_future() + + async def start(self, sides) -> None: + self.started_with = tuple(sides) + if self.script is not None: + await self.script(self) + + async def outcome(self) -> CalibrationOutcome: + return await asyncio.shield(self._outcome) + + def finish(self, outcome: CalibrationOutcome) -> None: + if not self._outcome.done(): + self._outcome.set_result(outcome) + + async def cancel(self) -> None: + self.cancel_calls += 1 + self.finish(_outcome(OutcomeKind.CANCELLED, "cancelled")) + + async def confirm(self, token: str) -> None: + self.confirm_calls.append(token) + if self.confirm_error is not None: + raise self.confirm_error + + +class StubWorker: + def __init__(self, script=None, unreleased: str | None = None) -> None: + self.script = script + self.unreleased = unreleased + self.attempts: list[StubAttempt] = [] + + def new_attempt(self, on_event) -> StubAttempt: + attempt = StubAttempt(on_event, self.script) + self.attempts.append(attempt) + return attempt + + +def _outcome(kind: OutcomeKind, message: str = "", files: tuple[wp.ExportedFile, ...] = ()) -> CalibrationOutcome: + return CalibrationOutcome(kind, ATTEMPT, (LEFT,), message, files=files) + + +def _files(*sides: wp.Side) -> tuple[wp.ExportedFile, ...]: + return tuple(wp.ExportedFile(side, f"set_{side.value}.mcal", 10, "0" * 64) for side in sides) + + +def _gloves(): + return (wp.Glove(LEFT, 11), wp.Glove(RIGHT, 22)) + + +def _prompt(token="left-0", *, side=LEFT, mode=wp.StepMode.HOLD, duration=0.0, continuous=False, estimated=0.0): + return wp.Prompt( + ATTEMPT, + token, + side, + 11 if side is LEFT else 22, + 0, + 8, + "Make a fist", + "Lift up your hand and make a fist.", + mode, + duration, + continuous, + estimated, + ) + + +def _session(worker: StubWorker, outbox: StubOutbox | None = None, sides=(p.HandSide.LEFT,)): + outbox = outbox or StubOutbox() + request = p.StartCalibrationRequest("cal-1", tuple(sides)) + return cal.CalibrationSession(request, worker, outbox, owner_ws=object()), outbox + + +async def _run_with(script, *, outcome: CalibrationOutcome, unreleased=None, sides=(p.HandSide.LEFT,)): + async def scripted(attempt: StubAttempt) -> None: + await script(attempt) + attempt.finish(outcome) + + session, outbox = _session(StubWorker(scripted, unreleased), sides=sides) + await session.run() + return session, outbox + + +async def _no_events(_attempt) -> None: + return None + + +async def test_run_publishes_started_and_an_initial_connecting_snapshot_first(): + session, outbox = await _run_with(_no_events, outcome=_outcome(OutcomeKind.CANCELLED, "cancelled")) + kinds = [kind for kind, _ in outbox.log] + assert kinds[0] == "bind" + assert outbox.log[0][1] == () + assert outbox.log[1][1]["type"] == int(p.MessageType.CALIBRATION_STARTED) + assert outbox.log[2][0] == "status" + assert outbox.statuses()[0]["phase"] == int(p.CalibrationPhase.CONNECTING) + + +async def test_run_starts_the_attempt_with_the_requested_worker_sides(): + worker = StubWorker(None) + session, outbox = _session(worker, sides=(p.HandSide.LEFT, p.HandSide.RIGHT)) + worker_attempt_ready = asyncio.ensure_future(session.run()) + await asyncio.sleep(0) + await asyncio.sleep(0) + worker.attempts[0].finish(_outcome(OutcomeKind.CANCELLED)) + await worker_attempt_ready + assert worker.attempts[0].started_with == (LEFT, RIGHT) + + +async def test_a_continuous_prompt_carries_its_estimate_distinct_from_the_timed_duration(): + async def script(attempt: StubAttempt) -> None: + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.AWAITING_STEP, _gloves())) + attempt.on_event(_prompt(continuous=True, estimated=3.0)) + + _, outbox = await _run_with(script, outcome=_outcome(OutcomeKind.CANCELLED)) + step = outbox.statuses()[-1]["step"] + assert step["continuous"] is True + assert step["estimated_seconds"] == 3.0 + assert step["duration_seconds"] == 0.0 + assert step["mode"] == int(p.CalibrationStepMode.HOLD) + assert (step["title"], step["instruction"]) == ("Make a fist", "Lift up your hand and make a fist.") + assert (step["index"], step["count"], step["token"]) == (0, 8, "left-0") + + +async def test_a_motion_prompt_keeps_its_timed_window_and_no_estimate(): + async def script(attempt: StubAttempt) -> None: + attempt.on_event(_prompt("m-0", mode=wp.StepMode.MOTION, duration=2.0)) + + _, outbox = await _run_with(script, outcome=_outcome(OutcomeKind.CANCELLED)) + step = outbox.statuses()[-1]["step"] + assert step["mode"] == int(p.CalibrationStepMode.MOTION) + assert (step["duration_seconds"], step["estimated_seconds"], step["continuous"]) == (2.0, 0.0, False) + + +async def test_an_awaiting_status_alone_publishes_no_snapshot(): + async def script(attempt: StubAttempt) -> None: + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.AWAITING_STEP, _gloves())) + + _, outbox = await _run_with(script, outcome=_outcome(OutcomeKind.CANCELLED)) + # Only the initial CONNECTING snapshot: the prompt that follows carries the step, so no prompt-less wait shows. + assert [s["phase"] for s in outbox.statuses()] == [int(p.CalibrationPhase.CONNECTING)] + + +async def test_step_acceptance_is_published_in_worker_order_before_the_next_prompt_snapshot(): + async def script(attempt: StubAttempt) -> None: + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.AWAITING_STEP, _gloves())) + attempt.on_event(_prompt("left-0", continuous=True, estimated=2.0)) + attempt.on_event(wp.StepAccepted(ATTEMPT, "left-0")) + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.AWAITING_STEP, _gloves())) + attempt.on_event(_prompt("left-1", continuous=True, estimated=3.0)) + + _, outbox = await _run_with(script, outcome=_outcome(OutcomeKind.CANCELLED)) + accepted_at = next( + i + for i, (kind, data) in enumerate(outbox.log) + if kind == "control" and data["type"] == int(p.MessageType.CALIBRATION_STEP_ACCEPTED) + ) + next_prompt_at = next( + i + for i, (kind, data) in enumerate(outbox.log) + if kind == "status" and data["step"] and data["step"]["token"] == "left-1" + ) + assert accepted_at < next_prompt_at + assert outbox.log[accepted_at][1]["step_token"] == "left-0" + + +async def test_every_snapshot_is_a_complete_replacement(): + async def script(attempt: StubAttempt) -> None: + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.DISCOVERING, ())) + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.DISCOVERING, _gloves())) + attempt.on_event(_prompt("m-0", mode=wp.StepMode.MOTION, duration=2.0)) + attempt.on_event(wp.StepAccepted(ATTEMPT, "m-0")) + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.CAPTURING_STEP, _gloves())) + attempt.on_event(wp.StepProgress(ATTEMPT, "m-0", 0.5)) + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.SAVING, _gloves())) + + _, outbox = await _run_with(script, outcome=_outcome(OutcomeKind.CANCELLED)) + keys = {"type", "version", "session_id", "phase", "gloves", "step", "step_elapsed_seconds"} + assert all(set(status) == keys for status in outbox.statuses()) + capturing = [s for s in outbox.statuses() if s["phase"] == int(p.CalibrationPhase.CAPTURING_STEP)][-1] + assert capturing["step"]["token"] == "m-0" + progress = [s for s in outbox.statuses() if s["step_elapsed_seconds"] == 0.5][-1] + assert progress["phase"] == int(p.CalibrationPhase.CAPTURING_STEP) + saving = outbox.statuses()[-1] + assert saving["phase"] == int(p.CalibrationPhase.SAVING) + assert saving["step"] is None and saving["step_elapsed_seconds"] is None + + +async def test_gloves_are_reported_with_wire_sides(): + async def script(attempt: StubAttempt) -> None: + attempt.on_event(wp.Status(ATTEMPT, wp.Phase.DISCOVERING, _gloves())) + + _, outbox = await _run_with(script, outcome=_outcome(OutcomeKind.CANCELLED)) + assert outbox.statuses()[-1]["gloves"] == [{"side": 1, "glove_id": 11}, {"side": 2, "glove_id": 22}] + + +async def test_the_terminal_message_is_published_last_and_only_after_quiesce(): + session, outbox = await _run_with(_no_events, outcome=_outcome(OutcomeKind.CANCELLED)) + names = [kind for kind, _ in outbox.log] + assert names.index("quiesce") < len(names) - 1 + kind, last = outbox.log[-1] + assert kind == "control" and last["type"] == int(p.MessageType.CALIBRATION_STOPPED) + assert outbox.controls(p.MessageType.CALIBRATION_STOPPED) == [last] + assert session.terminal is not None and session.terminal.to_json() == last + assert session.done.is_set() + + +async def test_a_completed_outcome_offers_one_basename_per_published_side(tmp_path): + published = _files(LEFT, RIGHT) + session, _ = await _run_with( + _no_events, + outcome=_outcome(OutcomeKind.COMPLETED, "ok", published), + sides=(p.HandSide.LEFT, p.HandSide.RIGHT), + ) + terminal = session.terminal + assert terminal.reason is p.CalibrationStopReason.COMPLETED + assert terminal.error_message is None + assert [(f.side, f.filename) for f in terminal.result.files] == [ + (p.HandSide.LEFT, "set_left.mcal"), + (p.HandSide.RIGHT, "set_right.mcal"), + ] + + +@pytest.mark.parametrize( + "kind,reason", + [ + (OutcomeKind.FAILED, p.CalibrationStopReason.FAILED), + (OutcomeKind.UNCERTAIN, p.CalibrationStopReason.UNCERTAIN), + (OutcomeKind.CANCELLED, p.CalibrationStopReason.CANCELLED), + ], +) +async def test_files_are_never_offered_for_a_non_completed_outcome(tmp_path, kind, reason): + session, _ = await _run_with(_no_events, outcome=_outcome(kind, "no", _files(LEFT))) + assert session.terminal.reason is reason + assert session.terminal.result is None + + +async def test_failed_and_uncertain_outcomes_carry_a_diagnostic_and_cancelled_does_not(): + failed, _ = await _run_with(_no_events, outcome=_outcome(OutcomeKind.FAILED, "the SDK refused")) + uncertain, _ = await _run_with( + _no_events, outcome=_outcome(OutcomeKind.UNCERTAIN, "no response; state unknown") + ) + cancelled, _ = await _run_with(_no_events, outcome=_outcome(OutcomeKind.CANCELLED, "cancelled")) + assert failed.terminal.error_message == "the SDK refused" + assert uncertain.terminal.error_message == "no response; state unknown" + assert cancelled.terminal.error_message is None + + +async def test_a_completed_outcome_without_published_files_degrades_instead_of_raising(): + session, _ = await _run_with(_no_events, outcome=_outcome(OutcomeKind.COMPLETED, "ok", None)) + assert session.terminal.reason is not p.CalibrationStopReason.COMPLETED + assert session.terminal.result is None + + +async def test_cleanup_ok_follows_the_supervisors_unreleased_latch(): + ok, _ = await _run_with(_no_events, outcome=_outcome(OutcomeKind.CANCELLED), unreleased=None) + stuck, _ = await _run_with( + _no_events, outcome=_outcome(OutcomeKind.UNCERTAIN, "x"), unreleased="worker still running" + ) + assert ok.cleanup_ok is True + assert stuck.cleanup_ok is False + + +async def test_a_worker_that_cannot_be_admitted_raises_before_the_session_exists(): + class Busy: + unreleased = "a worker may still hold the gloves" + + def new_attempt(self, on_event): + raise AttemptError(self.unreleased) + + with pytest.raises(AttemptError): + cal.CalibrationSession( + p.StartCalibrationRequest("cal-1", (p.HandSide.LEFT,)), Busy(), StubOutbox(), object() + ) + + +async def test_request_stop_cancels_the_attempt_once(): + worker = StubWorker(None) + session, outbox = _session(worker) + run = asyncio.ensure_future(session.run()) + await asyncio.sleep(0) + await asyncio.sleep(0) + session.request_stop(p.StopReason.OWNER_DISCONNECTED) + session.request_stop(p.StopReason.SERVER_SHUTDOWN) + session.request_stop() + await run + assert worker.attempts[0].cancel_calls == 1 + assert session.terminal.reason is p.CalibrationStopReason.CANCELLED + + +async def test_a_stop_before_the_worker_starts_is_settled_quietly(): + class AlreadyCancelled(StubAttempt): + async def start(self, sides) -> None: + raise AttemptError("an attempt can be started only once") + + worker = StubWorker(None) + session, _ = _session(worker) + attempt = AlreadyCancelled(session._on_event, None) + session._attempt = attempt + attempt.finish(_outcome(OutcomeKind.CANCELLED, "cancelled before start")) + await session.run() + assert session.terminal.reason is p.CalibrationStopReason.CANCELLED + + +async def test_the_connection_dying_cancels_the_attempt(): + worker = StubWorker(None) + session, outbox = _session(worker) + run = asyncio.ensure_future(session.run()) + await asyncio.sleep(0) + await asyncio.sleep(0) + outbox.on_fatal("control queue overflow") + await run + assert worker.attempts[0].cancel_calls == 1 + + +async def test_confirming_an_accepted_token_acknowledges_again_without_a_second_step(): + worker = StubWorker(None) + session, outbox = _session(worker) + session._accepted.add("left-0") + await session.handle_confirm("left-0") + assert worker.attempts[0].confirm_calls == [] + assert outbox.controls(p.MessageType.CALIBRATION_STEP_ACCEPTED)[0]["step_token"] == "left-0" + + +async def test_confirming_a_token_that_is_not_offered_is_refused_without_reaching_the_worker(): + worker = StubWorker(None) + session, _ = _session(worker) + with pytest.raises(cal.ConfirmRefused): + await session.handle_confirm("never-offered") + worker.attempts[0].prompt = _prompt("left-0") + with pytest.raises(cal.ConfirmRefused): + await session.handle_confirm("left-1") + assert worker.attempts[0].confirm_calls == [] + + +async def test_confirming_the_offered_token_reaches_the_worker_once(): + worker = StubWorker(None) + session, _ = _session(worker) + worker.attempts[0].prompt = _prompt("left-0") + await session.handle_confirm("left-0") + assert worker.attempts[0].confirm_calls == ["left-0"] + + +async def test_a_worker_side_confirm_refusal_becomes_confirm_refused_with_its_message(): + worker = StubWorker(None) + session, _ = _session(worker) + worker.attempts[0].prompt = _prompt("left-0") + worker.attempts[0].confirm_error = AttemptError("the attempt is not running") + with pytest.raises(cal.ConfirmRefused, match="not running"): + await session.handle_confirm("left-0") + + +async def test_spawn_confirm_reports_a_refusal_through_the_callback_without_raising(): + worker = StubWorker(None) + session, _ = _session(worker) + refusals: list[str] = [] + session.spawn_confirm("nope", refusals.append) + await asyncio.sleep(0.01) + assert refusals and "not awaiting confirmation" in refusals[0] + + +async def test_replay_resends_started_and_the_current_snapshot(): + worker = StubWorker(None) + session, outbox = _session(worker) + session.replay() + assert outbox.controls(p.MessageType.CALIBRATION_STARTED) + assert outbox.statuses() + assert outbox.controls(p.MessageType.CALIBRATION_STARTED)[0]["session_id"] == "cal-1" + + +async def test_a_missing_outcome_is_reported_as_uncertain(): + session, _ = _session(StubWorker(None)) + terminal = session._terminal_of(None) + assert terminal.reason is p.CalibrationStopReason.UNCERTAIN + assert terminal.result is None diff --git a/examples/egocap/tests/capture_server/test_calibration_protocol.py b/examples/egocap/tests/capture_server/test_calibration_protocol.py new file mode 100644 index 0000000000..976a248ffa --- /dev/null +++ b/examples/egocap/tests/capture_server/test_calibration_protocol.py @@ -0,0 +1,417 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +import math + +import pytest + +from capture_server import protocol as p + +LEFT, RIGHT = p.HandSide.LEFT, p.HandSide.RIGHT + + +def _step(**overrides) -> p.CalibrationStep: + fields = dict( + token="tok-1", + side=LEFT, + index=0, + count=8, + title="Make a fist", + instruction="Lift up your hand and make a fist.", + mode=p.CalibrationStepMode.HOLD, + duration_seconds=0.0, + continuous=False, + estimated_seconds=0.0, + ) + fields.update(overrides) + return p.CalibrationStep(**fields) + + +def _stopped(reason=p.CalibrationStopReason.CANCELLED, result=None, error_message=None) -> p.CalibrationStopped: + return p.CalibrationStopped("cal-1", reason, result, error_message) + + +def _begin_json(plugins) -> dict: + return { + "type": 1, + "version": 1, + "session_id": "session-1", + "tracking": {"sources": [1, 3], "rate_hz": 60, "headset": {"device_index": 0}}, + "plugins": plugins, + "cameras": [], + } + + +# --------------------------------------------------------------------------- +# Numeric values and enums +# --------------------------------------------------------------------------- + + +def test_existing_capture_message_values_are_unchanged(): + assert [int(t) for t in (p.MessageType.BEGIN, p.MessageType.STOP, p.MessageType.SESSION_INIT)] == [1, 2, 9] + assert int(p.MessageType.REQUEST_REJECTED) == 10 + assert int(p.MessageType.CAPTURE_FRAME) == 7 + assert int(p.MessageType.CAPTURE_IMAGE) == 8 + + +def test_calibration_message_types_take_the_next_numeric_values(): + names = ( + "START_CALIBRATION", + "CONFIRM_CALIBRATION_STEP", + "STOP_CALIBRATION", + "CALIBRATION_STARTED", + "CALIBRATION_STATUS", + "CALIBRATION_STEP_ACCEPTED", + "CALIBRATION_STOPPED", + ) + assert [int(p.MessageType[name]) for name in names] == [13, 14, 15, 16, 17, 18, 19] + + +def test_calibration_enums_have_their_documented_values(): + assert (int(p.HandSide.LEFT), int(p.HandSide.RIGHT)) == (1, 2) + assert [int(x) for x in p.CalibrationPhase] == [1, 2, 3, 4, 5] + assert (int(p.CalibrationStepMode.HOLD), int(p.CalibrationStepMode.MOTION)) == (0, 1) + assert [int(x) for x in p.CalibrationStopReason] == [0, 1, 2, 3] + assert p.CalibrationStopReason.UNCERTAIN == 3 + + +# --------------------------------------------------------------------------- +# Client-to-server requests +# --------------------------------------------------------------------------- + + +def test_start_calibration_round_trips(): + request = p.StartCalibrationRequest(session_id="cal-1", sides=(LEFT, RIGHT)) + assert request.to_json() == {"type": 13, "version": 1, "session_id": "cal-1", "sides": [1, 2]} + assert p.parse_start_calibration(request.to_json()) == request + + +@pytest.mark.parametrize("sides", [(), (LEFT, LEFT)]) +def test_start_calibration_request_rejects_empty_or_duplicate_sides(sides): + with pytest.raises(ValueError): + p.StartCalibrationRequest(session_id="cal-1", sides=sides) + + +def test_start_calibration_request_rejects_non_hand_side_members(): + with pytest.raises(ValueError): + p.StartCalibrationRequest(session_id="cal-1", sides=(1,)) # type: ignore[arg-type] + + +@pytest.mark.parametrize( + "mutation", + [ + {"sides": []}, + {"sides": [1, 1]}, + {"sides": [3]}, + {"sides": [True]}, + {"sides": "left"}, + {"sides": None}, + {"extra": 1}, + ], +) +def test_parse_start_calibration_rejects_bad_fields(mutation): + golden = p.StartCalibrationRequest("cal-1", (LEFT,)).to_json() + with pytest.raises(p.ProtocolError): + p.parse_start_calibration({**golden, **mutation}) + + +def test_parse_start_calibration_rejects_a_missing_sides_key_and_the_wrong_envelope_type(): + golden = p.StartCalibrationRequest("cal-1", (LEFT,)).to_json() + without_sides = {k: v for k, v in golden.items() if k != "sides"} + with pytest.raises(p.ProtocolError): + p.parse_start_calibration(without_sides) + with pytest.raises(p.ProtocolError): + p.parse_start_calibration({**golden, "type": int(p.MessageType.STOP_CALIBRATION)}) + + +def test_confirm_calibration_step_round_trips_and_rejects_bad_tokens(): + request = p.ConfirmCalibrationStepRequest(session_id="cal-1", step_token="left-0") + assert request.to_json() == {"type": 14, "version": 1, "session_id": "cal-1", "step_token": "left-0"} + assert p.parse_confirm_calibration_step(request.to_json()) == request + for bad in ("", None, 5, "x" * 129): + with pytest.raises(p.ProtocolError): + p.parse_confirm_calibration_step({**request.to_json(), "step_token": bad}) + with pytest.raises(p.ProtocolError): + p.parse_confirm_calibration_step({**request.to_json(), "extra": 1}) + + +def test_stop_calibration_round_trips_and_rejects_extra_keys(): + request = p.StopCalibrationRequest(session_id="cal-1") + assert request.to_json() == {"type": 15, "version": 1, "session_id": "cal-1"} + assert p.parse_stop_calibration(request.to_json()) == request + with pytest.raises(p.ProtocolError): + p.parse_stop_calibration({**request.to_json(), "sides": [1]}) + + +# --------------------------------------------------------------------------- +# Step model +# --------------------------------------------------------------------------- + + +def test_step_round_trips_for_hold_motion_and_continuous(): + hold = _step() + motion = _step(mode=p.CalibrationStepMode.MOTION, duration_seconds=2.5) + continuous = _step(continuous=True, estimated_seconds=3.0) + for step in (hold, motion, continuous): + assert p.CalibrationStep.from_json(step.to_json()) == step + + +def test_continuous_step_keeps_actual_duration_distinct_from_the_estimate(): + step = _step(continuous=True, estimated_seconds=10.0) + data = step.to_json() + assert data["duration_seconds"] == 0.0 + assert data["estimated_seconds"] == 10.0 + assert data["continuous"] is True + + +@pytest.mark.parametrize( + "overrides", + [ + {"mode": p.CalibrationStepMode.HOLD, "duration_seconds": 1.0}, + {"mode": p.CalibrationStepMode.MOTION, "duration_seconds": 0.0}, + {"continuous": True, "estimated_seconds": 0.0}, + { + "continuous": True, + "estimated_seconds": 2.0, + "mode": p.CalibrationStepMode.MOTION, + "duration_seconds": 1.0, + }, + {"continuous": False, "estimated_seconds": 2.0}, + {"index": 8}, + {"index": -1}, + {"count": 0}, + {"token": ""}, + ], +) +def test_step_constructor_rejects_inconsistent_invariants(overrides): + with pytest.raises(ValueError): + _step(**overrides) + + +@pytest.mark.parametrize("bad", [float("nan"), float("inf"), -1.0, True, "1", None]) +@pytest.mark.parametrize("key", ["duration_seconds", "estimated_seconds"]) +def test_step_from_json_rejects_non_finite_negative_or_mistyped_seconds(key, bad): + data = _step().to_json() + data[key] = bad + with pytest.raises(p.ProtocolError): + p.CalibrationStep.from_json(data) + + +def test_step_from_json_rejects_missing_extra_and_mistyped_fields(): + golden = _step().to_json() + for key in golden: + with pytest.raises(p.ProtocolError): + p.CalibrationStep.from_json({k: v for k, v in golden.items() if k != key}) + with pytest.raises(p.ProtocolError): + p.CalibrationStep.from_json({**golden, "extra": 1}) + for key, bad in ( + ("side", 3), + ("mode", 2), + ("index", 1.5), + ("count", "8"), + ("instruction", ""), + ("title", None), + ("continuous", 1), + ): + with pytest.raises(p.ProtocolError): + p.CalibrationStep.from_json({**golden, key: bad}) + + +def test_step_from_json_reports_a_broken_invariant_as_a_protocol_error(): + golden = _step().to_json() + with pytest.raises(p.ProtocolError): + p.CalibrationStep.from_json({**golden, "continuous": True, "estimated_seconds": 0.0}) + + +def test_step_has_no_hardcoded_count_and_carries_the_sdk_text_unchanged(): + step = _step(count=3, title="SDK title", instruction="SDK instruction") + data = step.to_json() + assert (data["count"], data["title"], data["instruction"]) == (3, "SDK title", "SDK instruction") + + +# --------------------------------------------------------------------------- +# Server-to-client messages +# --------------------------------------------------------------------------- + + +def test_started_round_trips_and_rejects_bad_sides(): + event = p.CalibrationStarted("cal-1", (LEFT, RIGHT)) + assert event.to_json() == {"type": 16, "version": 1, "session_id": "cal-1", "sides": [1, 2]} + assert p.CalibrationStarted.from_json(event.to_json()) == event + for bad in ([], [1, 1], [9], None): + with pytest.raises(p.ProtocolError): + p.CalibrationStarted.from_json({**event.to_json(), "sides": bad}) + + +def test_status_round_trips_with_and_without_a_step(): + gloves = (p.CalibrationGlove(LEFT, 11), p.CalibrationGlove(RIGHT, 22)) + with_step = p.CalibrationStatus("cal-1", p.CalibrationPhase.AWAITING_STEP, gloves, _step(), None) + capturing = p.CalibrationStatus( + "cal-1", + p.CalibrationPhase.CAPTURING_STEP, + gloves, + _step(mode=p.CalibrationStepMode.MOTION, duration_seconds=2.0), + 0.5, + ) + connecting = p.CalibrationStatus("cal-1", p.CalibrationPhase.CONNECTING, (), None, None) + for status in (with_step, capturing, connecting): + assert p.CalibrationStatus.from_json(status.to_json()) == status + + +def test_status_rejects_bad_fields(): + golden = p.CalibrationStatus("cal-1", p.CalibrationPhase.CONNECTING, (), None, None).to_json() + for key, bad in ( + ("phase", 0), + ("phase", 6), + ("gloves", None), + ("gloves", [{"side": 1}]), + ("gloves", [{"side": 1, "glove_id": -1}]), + ("step", {"token": "x"}), + ("step_elapsed_seconds", -1.0), + ("step_elapsed_seconds", math.nan), + ): + with pytest.raises(p.ProtocolError): + p.CalibrationStatus.from_json({**golden, key: bad}) + with pytest.raises(p.ProtocolError): + p.CalibrationStatus.from_json({**golden, "extra": 1}) + + +def test_step_accepted_round_trips_and_requires_a_token(): + event = p.CalibrationStepAccepted("cal-1", "left-0") + assert event.to_json() == {"type": 18, "version": 1, "session_id": "cal-1", "step_token": "left-0"} + assert p.CalibrationStepAccepted.from_json(event.to_json()) == event + with pytest.raises(p.ProtocolError): + p.CalibrationStepAccepted.from_json({**event.to_json(), "step_token": ""}) + + +def test_stopped_completed_requires_a_result_with_basenames(): + result = p.CalibrationResult((p.McalFile(LEFT, "set_left.mcal"), p.McalFile(RIGHT, "set_right.mcal"))) + event = _stopped(p.CalibrationStopReason.COMPLETED, result, None) + assert p.CalibrationStopped.from_json(event.to_json()) == event + assert event.to_json()["result"]["files"][0] == {"side": 1, "filename": "set_left.mcal"} + + +@pytest.mark.parametrize( + "reason", + [p.CalibrationStopReason.CANCELLED, p.CalibrationStopReason.FAILED, p.CalibrationStopReason.UNCERTAIN], +) +def test_stopped_other_reasons_forbid_a_result(reason): + result = p.CalibrationResult((p.McalFile(LEFT, "set_left.mcal"),)) + with pytest.raises(ValueError): + _stopped(reason, result, "oops") + event = _stopped(reason, None, "something went wrong") + assert p.CalibrationStopped.from_json(event.to_json()) == event + + +def test_stopped_completed_without_a_result_or_with_an_error_is_invalid(): + result = p.CalibrationResult((p.McalFile(LEFT, "set_left.mcal"),)) + with pytest.raises(ValueError): + _stopped(p.CalibrationStopReason.COMPLETED, None, None) + with pytest.raises(ValueError): + _stopped(p.CalibrationStopReason.COMPLETED, result, "late warning") + + +def test_stopped_from_json_reports_broken_invariants_as_protocol_errors(): + golden = _stopped(p.CalibrationStopReason.FAILED, None, "failed").to_json() + with pytest.raises(p.ProtocolError): + p.CalibrationStopped.from_json({**golden, "result": {"files": [{"side": 1, "filename": "a.mcal"}]}}) + with pytest.raises(p.ProtocolError): + p.CalibrationStopped.from_json({**golden, "error_message": ""}) + with pytest.raises(p.ProtocolError): + p.CalibrationStopped.from_json({**golden, "reason": 4}) + + +@pytest.mark.parametrize( + "filename", ["../x.mcal", "/abs/x.mcal", "dir/x.mcal", ".hidden.mcal", "x.txt", "x", "", "a" * 130 + ".mcal"] +) +def test_mcal_file_rejects_anything_but_a_bare_mcal_basename(filename): + with pytest.raises(p.ProtocolError): + p.McalFile.from_json({"side": 1, "filename": filename}) + + +def test_calibration_result_rejects_empty_and_two_files_for_one_side(): + with pytest.raises(p.ProtocolError): + p.CalibrationResult.from_json({"files": []}) + one = {"side": 1, "filename": "a.mcal"} + with pytest.raises(p.ProtocolError): + p.CalibrationResult.from_json({"files": [one, {"side": 1, "filename": "b.mcal"}]}) + + +def test_parse_server_message_dispatches_the_four_calibration_events(): + events = ( + p.CalibrationStarted("cal-1", (LEFT,)), + p.CalibrationStatus("cal-1", p.CalibrationPhase.CONNECTING, (), None, None), + p.CalibrationStepAccepted("cal-1", "tok"), + _stopped(p.CalibrationStopReason.CANCELLED, None, None), + ) + for event in events: + assert p.parse_server_message(event.to_json()) == event + + +def test_parse_server_message_rejects_client_to_server_calibration_requests(): + for request in ( + p.StartCalibrationRequest("cal-1", (LEFT,)), + p.ConfirmCalibrationStepRequest("cal-1", "tok"), + p.StopCalibrationRequest("cal-1"), + ): + with pytest.raises(p.ProtocolError): + p.parse_server_message(request.to_json()) + + +# --------------------------------------------------------------------------- +# RequestRejected +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "request_type", + [ + p.MessageType.BEGIN, + p.MessageType.STOP, + p.MessageType.START_CALIBRATION, + p.MessageType.CONFIRM_CALIBRATION_STEP, + p.MessageType.STOP_CALIBRATION, + ], +) +def test_request_rejected_answers_every_client_request_type(request_type): + event = p.RequestRejected("s", request_type, p.RejectionReason.BUSY, "busy") + assert p.RequestRejected.from_json(event.to_json()) == event + + +@pytest.mark.parametrize( + "request_type", + [p.MessageType.SESSION_INIT, p.MessageType.CALIBRATION_STATUS, p.MessageType.CALIBRATION_STOPPED], +) +def test_request_rejected_refuses_server_message_types(request_type): + with pytest.raises(ValueError, match="client request type"): + p.RequestRejected("s", request_type, p.RejectionReason.BUSY, "busy") + + +def test_request_rejected_echoes_a_step_token_only_for_confirm(): + event = p.RequestRejected( + "cal-1", + p.MessageType.CONFIRM_CALIBRATION_STEP, + p.RejectionReason.INVALID_CONFIG, + "stale", + step_token="left-0", + ) + assert event.to_json()["step_token"] == "left-0" + assert p.RequestRejected.from_json(event.to_json()) == event + with pytest.raises(ValueError): + p.RequestRejected("s", p.MessageType.BEGIN, p.RejectionReason.BUSY, "busy", step_token="x") + golden = p.RequestRejected("s", p.MessageType.BEGIN, p.RejectionReason.BUSY, "busy").to_json() + with pytest.raises(p.ProtocolError): + p.RequestRejected.from_json({**golden, "step_token": "x"}) + + +def test_begin_and_stop_rejections_keep_their_exact_earlier_wire_form(): + golden = p.RequestRejected("s", p.MessageType.BEGIN, p.RejectionReason.BUSY, "busy").to_json() + assert golden == { + "type": 10, + "version": 1, + "session_id": "s", + "request_type": 1, + "reason": 2, + "error_message": "busy", + } diff --git a/examples/egocap/tests/capture_server/test_calibration_server.py b/examples/egocap/tests/capture_server/test_calibration_server.py new file mode 100644 index 0000000000..ef56daaf25 --- /dev/null +++ b/examples/egocap/tests/capture_server/test_calibration_server.py @@ -0,0 +1,683 @@ +# SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# +# SPDX-License-Identifier: Apache-2.0 + +"""CaptureServer calibration sessions over real sockets, with the scripted fake worker as a real process. + +Admission, ownership and message-ordering rules are asserted at the wire level on one raw connection (the +server admits a single owner connection, so capture/calibration exclusivity is observed on that one socket); +the typed client API is exercised end to end through CalibrationRun. +""" + +from __future__ import annotations + +import asyncio +import json +import time +from pathlib import Path + +import pytest + +from capture_server import protocol as p +from capture_server.calibration import CalibrationConfig, CalibrationSession +from capture_server.client import CaptureClient +from capture_server.server import ServerConfig +from tests.capture_server.calibration_helpers import ( + BOTH, + CONTINUOUS_STEPS, + LEFT, + RIGHT, + CalibrationRun, + connect, + is_type, + next_prompt, + recv_event, + recv_until, + send, + start_over_raw, + terminal_of, +) +from tests.support.capture_server import poll_until + + +def _begin(session_id: str = "cap-1") -> dict: + return { + "type": int(p.MessageType.BEGIN), + "version": 1, + "session_id": session_id, + "tracking": {"sources": [1], "rate_hz": 60, "headset": {"device_index": 0}}, + "plugins": [], + "cameras": [], + } + + +def _capture_stop(session_id: str) -> dict: + return {"type": int(p.MessageType.STOP), "version": 1, "session_id": session_id} + + +async def _stop_calibration_and_wait(ws, session_id: str = "cal-1") -> p.CalibrationStopped: + await send(ws, p.StopCalibrationRequest(session_id)) + return await terminal_of(ws) + + +async def _rejection(ws) -> p.RequestRejected: + event, _ = await recv_until(ws, is_type(p.RequestRejected)) + return event + + +# --------------------------------------------------------------------------- +# Happy paths through the typed client +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("sides", [(LEFT,), (RIGHT,), BOTH], ids=["left", "right", "both"]) +async def test_a_calibration_completes_with_one_saved_file_per_requested_side(make_station, sides): + station = await make_station() + run = CalibrationRun(station.url) + await run.start(sides) + stopped = await run.wait() + + assert run.started[0].sides == sides + assert stopped.reason is p.CalibrationStopReason.COMPLETED + assert stopped.error_message is None + assert [f.side for f in stopped.result.files] == list(sides) + for file in stopped.result.files: + assert (station.directory / file.filename).is_file() + assert "/" not in file.filename + assert len(list(station.directory.glob("*.mcal"))) == len(sides) + assert not run.errors + + +async def test_every_prompt_needs_its_own_confirmation_including_the_second_gloves_first(make_station): + station = await make_station() + run = CalibrationRun(station.url) + await run.start(BOTH) + await run.wait() + + assert [step.token for step in run.prompts()] == ["left-0", "left-1", "right-0", "right-1"] + assert run.confirmed == ["left-0", "left-1", "right-0", "right-1"] + assert station.commands().count("start") == 1 + assert station.commands().count("confirm") == 4 + assert all(step.count == 2 for step in run.prompts()) + + +async def test_a_timed_step_reports_elapsed_progress_in_snapshots(make_station): + station = await make_station() + run = CalibrationRun(station.url) + await run.start((LEFT,)) + await run.wait() + + capturing = [s for s in run.statuses if s.phase is p.CalibrationPhase.CAPTURING_STEP] + assert capturing + assert all(s.step.mode is p.CalibrationStepMode.MOTION for s in capturing) + assert any(s.step_elapsed_seconds for s in capturing) + assert all( + s.step_elapsed_seconds <= s.step.duration_seconds + 1e-6 for s in capturing if s.step_elapsed_seconds + ) + + +async def test_continuous_step_metadata_survives_worker_to_public_status_to_client_callback(make_station): + station = await make_station(scenario={"steps": CONTINUOUS_STEPS}) + run = CalibrationRun(station.url) + await run.start((LEFT,)) + stopped = await run.wait() + + first, second = run.prompts() + assert (first.continuous, first.estimated_seconds, first.duration_seconds) == (True, 3.0, 0.0) + assert (second.continuous, second.estimated_seconds, second.duration_seconds) == (True, 2.0, 0.0) + assert first.mode is p.CalibrationStepMode.HOLD + assert (first.title, first.instruction) == ("Make a fist", "Lift up your hand and make a fist.") + assert stopped.reason is p.CalibrationStopReason.COMPLETED + + +async def test_the_station_is_free_again_after_a_calibration(make_station): + station = await make_station() + first = CalibrationRun(station.url) + await first.start((LEFT,)) + await first.wait() + second = CalibrationRun(station.url) + await second.start((RIGHT,)) + assert (await second.wait()).reason is p.CalibrationStopReason.COMPLETED + + +# --------------------------------------------------------------------------- +# Exclusive admission +# --------------------------------------------------------------------------- + + +async def test_a_capture_begin_during_a_calibration_is_rejected_busy(make_station): + station = await make_station() + ws = await connect(station.url) + try: + assert isinstance(await start_over_raw(ws), p.CalibrationStarted) + await send(ws, _begin()) + rejected = await _rejection(ws) + assert rejected.request_type is p.MessageType.BEGIN + assert rejected.reason is p.RejectionReason.BUSY + assert (await _stop_calibration_and_wait(ws)).reason is p.CalibrationStopReason.CANCELLED + finally: + await ws.close() + + +async def test_a_calibration_start_during_a_capture_is_rejected_busy(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await send(ws, _begin()) + await recv_until(ws, is_type(p.SessionInit)) + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + rejected = await _rejection(ws) + assert rejected.request_type is p.MessageType.START_CALIBRATION + assert rejected.reason is p.RejectionReason.BUSY + await send(ws, _capture_stop("cap-1")) + await recv_until(ws, is_type(p.SessionStopped)) + finally: + await ws.close() + + +async def test_two_near_simultaneous_starts_admit_exactly_one(make_station): + station = await make_station() + ws = await connect(station.url) + try: + # Sent back to back: the first accepted request must already occupy the station when the second is read. + await send(ws, _begin()) + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + rejected, seen = await recv_until(ws, is_type(p.RequestRejected)) + assert rejected.request_type is p.MessageType.START_CALIBRATION + assert rejected.reason is p.RejectionReason.BUSY + # The accepted session's own first message may be written after the rejection of the later request. + if not any(isinstance(event, p.SessionInit) for event in seen): + _, later = await recv_until(ws, is_type(p.SessionInit)) + seen += later + assert any(isinstance(event, p.SessionInit) for event in seen) + assert not any(isinstance(event, p.CalibrationStarted) for event in seen) + await send(ws, _capture_stop("cap-1")) + await recv_until(ws, is_type(p.SessionStopped)) + finally: + await ws.close() + + +async def test_a_calibration_first_then_a_capture_admits_only_the_calibration(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + await send(ws, _begin()) + rejected, seen = await recv_until(ws, is_type(p.RequestRejected)) + assert rejected.request_type is p.MessageType.BEGIN + # The accepted session's own first message may be written after the rejection of the later request. + if not any(isinstance(event, p.CalibrationStarted) for event in seen): + _, later = await recv_until(ws, is_type(p.CalibrationStarted)) + seen += later + assert any(isinstance(event, p.CalibrationStarted) for event in seen) + assert not any(isinstance(event, p.SessionInit) for event in seen) + assert (await _stop_calibration_and_wait(ws)).reason is p.CalibrationStopReason.CANCELLED + finally: + await ws.close() + + +async def test_a_capture_stop_addressed_at_a_calibration_id_does_not_stop_it(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + await send(ws, _capture_stop("cal-1")) + rejected = await _rejection(ws) + assert rejected.request_type is p.MessageType.STOP + assert rejected.reason is p.RejectionReason.UNKNOWN_SESSION + # Still alive: the worker keeps offering its steps. + assert (await next_prompt(ws)).step.token == "left-0" + assert (await _stop_calibration_and_wait(ws)).reason is p.CalibrationStopReason.CANCELLED + finally: + await ws.close() + + +async def test_a_stop_calibration_addressed_at_a_capture_id_does_not_stop_the_capture(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await send(ws, _begin("cap-1")) + await recv_until(ws, is_type(p.SessionInit)) + await send(ws, p.StopCalibrationRequest("cap-1")) + rejected = await _rejection(ws) + assert rejected.request_type is p.MessageType.STOP_CALIBRATION + assert rejected.reason is p.RejectionReason.UNKNOWN_SESSION + # The capture is still running: its own Stop ends it with the requested reason. + await send(ws, _capture_stop("cap-1")) + stopped, _ = await recv_until(ws, is_type(p.SessionStopped)) + assert stopped.reason is p.StopReason.REQUESTED + finally: + await ws.close() + + +async def test_a_session_id_cannot_be_reused_after_a_calibration_finished(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + await _stop_calibration_and_wait(ws, "cal-1") + await poll_until(lambda: station.server._active_session is None, timeout_s=10) + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + rejected = await _rejection(ws) + assert rejected.reason is p.RejectionReason.INVALID_CONFIG + assert "already used" in rejected.error_message + finally: + await ws.close() + + +async def test_a_stop_retry_after_the_terminal_replays_it(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + first = await _stop_calibration_and_wait(ws, "cal-1") + await poll_until(lambda: station.server._active_session is None, timeout_s=10) + await send(ws, p.StopCalibrationRequest("cal-1")) + assert await terminal_of(ws) == first + finally: + await ws.close() + + +async def test_a_stop_for_an_unknown_calibration_is_rejected(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await send(ws, p.StopCalibrationRequest("never-started")) + rejected = await _rejection(ws) + assert rejected.request_type is p.MessageType.STOP_CALIBRATION + assert rejected.reason is p.RejectionReason.UNKNOWN_SESSION + finally: + await ws.close() + + +# --------------------------------------------------------------------------- +# Failed cleanup keeps the station unavailable +# --------------------------------------------------------------------------- + + +async def test_failed_cleanup_keeps_both_capture_and_calibration_rejected(make_station, monkeypatch): + station = await make_station() + # The supervisor's unreleased latch is modelled directly: a worker that could not be reaped. + monkeypatch.setattr(CalibrationSession, "cleanup_ok", property(lambda self: False)) + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + terminal = await _stop_calibration_and_wait(ws, "cal-1") + await asyncio.sleep(0.3) + assert station.server._active_session is not None + + await send(ws, _begin("cap-2")) + capture = await _rejection(ws) + assert (capture.request_type, capture.reason) == (p.MessageType.BEGIN, p.RejectionReason.BUSY) + + await send(ws, p.StartCalibrationRequest("cal-2", (LEFT,))) + calibration = await _rejection(ws) + assert (calibration.request_type, calibration.reason) == ( + p.MessageType.START_CALIBRATION, + p.RejectionReason.BUSY, + ) + + # The held session answers a Stop retry with the terminal it already produced. + await send(ws, p.StopCalibrationRequest("cal-1")) + assert await terminal_of(ws) == terminal + finally: + await ws.close() + + +async def test_a_worker_that_reports_the_sdk_unreleased_ends_uncertain_and_keeps_the_station_unavailable( + make_station, +): + station = await make_station(scenario={"unreleased": True}) + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + # Stopping before the worker has started never touches the SDK and is a plain cancellation. + await next_prompt(ws) + terminal = await _stop_calibration_and_wait(ws, "cal-1") + assert terminal.reason is p.CalibrationStopReason.UNCERTAIN + assert terminal.result is None + assert "release" in terminal.error_message + assert not list(station.directory.glob("*")) + + await send(ws, p.StartCalibrationRequest("cal-2", (LEFT,))) + calibration = await _rejection(ws) + assert (calibration.request_type, calibration.reason) == ( + p.MessageType.START_CALIBRATION, + p.RejectionReason.BUSY, + ) + + await send(ws, _begin("cap-2")) + capture = await _rejection(ws) + assert (capture.request_type, capture.reason) == (p.MessageType.BEGIN, p.RejectionReason.BUSY) + + assert station.commands().count("start") == 1 + finally: + await ws.close() + + +async def test_a_worker_that_may_still_hold_the_gloves_refuses_a_new_calibration_with_its_reason(make_station): + station = await make_station() + station.server._calibration_worker.unreleased = "the worker (pid 1) could not be stopped" + ws = await connect(station.url) + try: + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + rejected = await _rejection(ws) + assert rejected.reason is p.RejectionReason.BUSY + assert "could not be stopped" in rejected.error_message + assert station.server._active_session is None + finally: + await ws.close() + + +# --------------------------------------------------------------------------- +# Confirmations +# --------------------------------------------------------------------------- + + +async def test_a_duplicate_confirm_of_an_accepted_token_starts_only_one_step(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + token = (await next_prompt(ws)).step.token + await send(ws, p.ConfirmCalibrationStepRequest("cal-1", token)) + first, _ = await recv_until(ws, is_type(p.CalibrationStepAccepted)) + await send(ws, p.ConfirmCalibrationStepRequest("cal-1", token)) + second, _ = await recv_until(ws, is_type(p.CalibrationStepAccepted)) + assert first.step_token == second.step_token == token + assert station.commands().count("confirm") == 1 + await _stop_calibration_and_wait(ws) + finally: + await ws.close() + + +async def test_a_stale_token_is_rejected_with_the_token_echoed_and_the_attempt_unchanged(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + offered = (await next_prompt(ws)).step.token + await send(ws, p.ConfirmCalibrationStepRequest("cal-1", "left-7")) + rejected = await _rejection(ws) + assert rejected.request_type is p.MessageType.CONFIRM_CALIBRATION_STEP + assert rejected.step_token == "left-7" + assert rejected.reason is p.RejectionReason.INVALID_CONFIG + # The worker never saw it, and the offered prompt can still be confirmed. + assert station.commands().count("confirm") == 0 + await send(ws, p.ConfirmCalibrationStepRequest("cal-1", offered)) + accepted, _ = await recv_until(ws, is_type(p.CalibrationStepAccepted)) + assert accepted.step_token == offered + await _stop_calibration_and_wait(ws) + finally: + await ws.close() + + +async def test_a_confirm_for_an_unknown_calibration_is_rejected_with_the_token_echoed(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await send(ws, p.ConfirmCalibrationStepRequest("nope", "left-0")) + rejected = await _rejection(ws) + assert rejected.reason is p.RejectionReason.UNKNOWN_SESSION + assert rejected.step_token == "left-0" + finally: + await ws.close() + + +async def test_a_slow_confirm_does_not_hold_up_a_stop_calibration_behind_it(make_station): + station = await make_station(scenario={"slow_confirm": 5.0}) + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + token = (await next_prompt(ws)).step.token + await send(ws, p.ConfirmCalibrationStepRequest("cal-1", token)) + began = time.monotonic() + await send(ws, p.StopCalibrationRequest("cal-1")) + terminal = await terminal_of(ws) + assert terminal.reason is p.CalibrationStopReason.CANCELLED + assert time.monotonic() - began < 4.0 + finally: + await ws.close() + + +async def test_a_confirm_after_the_calibration_ended_is_rejected(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + token = (await next_prompt(ws)).step.token + await _stop_calibration_and_wait(ws) + await poll_until(lambda: station.server._active_session is None, timeout_s=10) + await send(ws, p.ConfirmCalibrationStepRequest("cal-1", token)) + rejected = await _rejection(ws) + assert rejected.reason is p.RejectionReason.UNKNOWN_SESSION + finally: + await ws.close() + + +# --------------------------------------------------------------------------- +# Start replay and conflicts +# --------------------------------------------------------------------------- + + +async def test_an_identical_repeated_start_replays_acceptance_and_never_starts_a_second_worker(make_station): + station = await make_station() + ws = await connect(station.url) + try: + first = await start_over_raw(ws, "cal-1", (LEFT,)) + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + replay, _ = await recv_until(ws, is_type(p.CalibrationStarted)) + assert replay == first + await next_prompt(ws) + assert station.commands().count("start") == 1 + await _stop_calibration_and_wait(ws) + finally: + await ws.close() + + +async def test_a_start_with_the_same_id_but_different_sides_is_rejected(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1", (LEFT,)) + await send(ws, p.StartCalibrationRequest("cal-1", BOTH)) + rejected = await _rejection(ws) + assert rejected.reason is p.RejectionReason.INVALID_CONFIG + assert "differs" in rejected.error_message + await _stop_calibration_and_wait(ws) + finally: + await ws.close() + + +# --------------------------------------------------------------------------- +# Cancellation and cleanup +# --------------------------------------------------------------------------- + + +async def test_a_stop_calibration_cancels_and_reports_cancelled_without_files(make_station): + station = await make_station() + run = CalibrationRun(station.url, auto_confirm=False) + await run.start((LEFT,)) + await poll_until(lambda: run.prompts(), timeout_s=10) + await run.client.stop_calibration() + await run.client.stop_calibration() # idempotent + stopped = await run.wait() + assert stopped.reason is p.CalibrationStopReason.CANCELLED + assert stopped.result is None + assert not station.directory.exists() or list(station.directory.iterdir()) == [] + + +async def test_the_owner_disconnecting_cancels_the_calibration_and_reaps_the_worker(make_station): + station = await make_station() + ws = await connect(station.url) + await start_over_raw(ws, "cal-1") + await next_prompt(ws) + await ws.close() + + await poll_until(lambda: station.server._active_session is None, timeout_s=20) + assert station.server._calibration_worker.unreleased is None + staging = station.directory / ".staging" + assert not staging.exists() or list(staging.iterdir()) == [] + + +async def test_closing_the_server_cancels_an_active_calibration(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + await next_prompt(ws) + closing = asyncio.ensure_future(station.server.close()) + terminal = await terminal_of(ws) + assert terminal.reason is p.CalibrationStopReason.CANCELLED + await asyncio.wait_for(closing, timeout=15) + await poll_until(lambda: station.server._active_session is None, timeout_s=10) + finally: + await ws.close() + + +async def test_a_calibration_never_publishes_files_when_the_worker_fails(make_station): + station = await make_station(scenario={"fail_at_step": ["left", 1]}) + run = CalibrationRun(station.url) + await run.start((LEFT,)) + stopped = await run.wait() + assert stopped.reason is p.CalibrationStopReason.FAILED + assert stopped.result is None + assert "refused" in stopped.error_message + assert not station.directory.exists() or list(station.directory.iterdir()) == [] + + +# --------------------------------------------------------------------------- +# Unavailable calibration leaves capture alone +# --------------------------------------------------------------------------- + + +async def test_without_a_worker_and_directory_a_start_is_rejected_with_a_useful_message(make_station): + station = await make_station(calibration=False) + ws = await connect(station.url) + try: + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + rejected = await _rejection(ws) + assert rejected.request_type is p.MessageType.START_CALIBRATION + assert rejected.reason is p.RejectionReason.INVALID_CONFIG + assert "not available" in rejected.error_message + assert "worker" in rejected.error_message and "directory" in rejected.error_message + finally: + await ws.close() + + +async def test_ordinary_capture_still_works_when_calibration_is_unavailable(make_station): + station = await make_station(calibration=False) + stopped: list = [] + done = asyncio.Event() + client = CaptureClient(station.url) + await client.start( + sources=(p.SourceType.HEAD,), + on_stopped=lambda r: (stopped.append(r), done.set()), + ) + await asyncio.sleep(0.2) + await client.stop() + await asyncio.wait_for(done.wait(), timeout=10) + assert stopped[0].reason is p.StopReason.REQUESTED + + +async def test_ordinary_capture_still_works_when_calibration_is_configured(make_station): + station = await make_station() + stopped: list = [] + done = asyncio.Event() + client = CaptureClient(station.url) + await client.start( + sources=(p.SourceType.HEAD,), + on_stopped=lambda r: (stopped.append(r), done.set()), + ) + await asyncio.sleep(0.2) + await client.stop() + await asyncio.wait_for(done.wait(), timeout=10) + assert stopped[0].reason is p.StopReason.REQUESTED + + +async def test_a_configured_worker_that_cannot_be_launched_ends_the_calibration_failed(make_station, tmp_path): + config = ServerConfig( + calibration=CalibrationConfig( + worker_path=Path("/nonexistent/manus_glove_calibrator"), directory=tmp_path / "m" + ) + ) + station = await make_station(calibration=False, config=config) + ws = await connect(station.url) + try: + assert isinstance(await start_over_raw(ws, "cal-1"), p.CalibrationStarted) + terminal = await terminal_of(ws) + assert terminal.reason is p.CalibrationStopReason.FAILED + assert terminal.result is None + assert "cannot launch" in terminal.error_message + await poll_until(lambda: station.server._active_session is None, timeout_s=10) + finally: + await ws.close() + + +# --------------------------------------------------------------------------- +# Wire-level message ordering +# --------------------------------------------------------------------------- + + +async def test_started_precedes_every_status_and_the_terminal_message_is_last(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await send(ws, p.StartCalibrationRequest("cal-1", (LEFT,))) + events = [] + stop_sent = False + while True: + event = await recv_event(ws, timeout=15) + events.append(event) + if prompt_ready(event) and not stop_sent: + stop_sent = True + await send(ws, p.StopCalibrationRequest("cal-1")) + if isinstance(event, p.CalibrationStopped): + break + assert isinstance(events[0], p.CalibrationStarted) + assert sum(isinstance(e, p.CalibrationStopped) for e in events) == 1 + assert isinstance(events[-1], p.CalibrationStopped) + assert all(e.session_id == "cal-1" for e in events) + finally: + await ws.close() + + +def prompt_ready(event) -> bool: + return ( + isinstance(event, p.CalibrationStatus) + and event.phase is p.CalibrationPhase.AWAITING_STEP + and event.step is not None + ) + + +async def test_the_accepted_acknowledgement_arrives_before_the_next_prompt(make_station): + station = await make_station(scenario={"steps": CONTINUOUS_STEPS}) + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + first = (await next_prompt(ws)).step.token + await send(ws, p.ConfirmCalibrationStepRequest("cal-1", first)) + _, seen = await recv_until(ws, lambda e: prompt_ready(e) and e.step.token != first) + accepted_at = next(i for i, e in enumerate(seen) if isinstance(e, p.CalibrationStepAccepted)) + assert accepted_at < len(seen) - 1 + await _stop_calibration_and_wait(ws) + finally: + await ws.close() + + +async def test_calibration_messages_are_json_text_only_and_carry_no_capture_frames(make_station): + station = await make_station() + ws = await connect(station.url) + try: + await start_over_raw(ws, "cal-1") + await next_prompt(ws) + await send(ws, p.StopCalibrationRequest("cal-1")) + while True: + raw = await asyncio.wait_for(ws.recv(), timeout=15) + assert isinstance(raw, str) + message = json.loads(raw) + assert message["type"] not in (int(p.MessageType.CAPTURE_FRAME), int(p.MessageType.CAPTURE_IMAGE)) + if message["type"] == int(p.MessageType.CALIBRATION_STOPPED): + break + finally: + await ws.close() diff --git a/examples/egocap/tests/capture_server/test_protocol.py b/examples/egocap/tests/capture_server/test_protocol.py index b721e2cd35..3c1410fbfd 100644 --- a/examples/egocap/tests/capture_server/test_protocol.py +++ b/examples/egocap/tests/capture_server/test_protocol.py @@ -917,7 +917,7 @@ def test_request_rejected_for_stop(): def test_request_rejected_rejects_non_begin_stop_request_type(): - with pytest.raises(ValueError, match="BEGIN or STOP"): + with pytest.raises(ValueError, match="client request type"): p.RequestRejected( session_id="session-1", request_type=p.MessageType.SESSION_INIT, @@ -942,7 +942,7 @@ def test_request_rejected_rejects_empty_error_message(): def test_message_type_wire_values(): - assert [int(m) for m in p.MessageType] == [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] + assert [int(m) for m in p.MessageType] == [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17, 18, 19] def test_status_reason_wire_values_are_0_through_23(): @@ -1256,7 +1256,7 @@ def test_request_rejected_from_json_rejects_bad_values(): error_message="bad config", ).to_json() mutated = dict(golden, request_type=int(p.MessageType.SESSION_INIT)) - with pytest.raises(p.ProtocolError, match="BEGIN or STOP"): + with pytest.raises(p.ProtocolError, match="client request type"): p.RequestRejected.from_json(mutated) _assert_rejects(p.RequestRejected.from_json, golden, {"error_message": ""}) diff --git a/examples/egocap/uv.lock b/examples/egocap/uv.lock index b4a29b14fa..3513850af9 100644 --- a/examples/egocap/uv.lock +++ b/examples/egocap/uv.lock @@ -202,6 +202,7 @@ version = "0.1.0" source = { editable = "." } dependencies = [ { name = "fastapi" }, + { name = "isaaccapture-manus-calibration" }, { name = "loguru" }, { name = "numpy" }, { name = "opencv-python" }, @@ -236,6 +237,7 @@ requires-dist = [ { name = "depthai", marker = "extra == 'oak'", specifier = "==3.6.1" }, { name = "fastapi", specifier = ">=0.115" }, { name = "httpx", marker = "extra == 'dev'" }, + { name = "isaaccapture-manus-calibration", directory = "../../src/plugins/manus/calibration/python" }, { name = "loguru", specifier = ">=0.7.3" }, { name = "numpy", specifier = ">=1.26.4" }, { name = "opencv-python", specifier = "==4.12.0.88" }, @@ -409,6 +411,11 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/56/43/4ca9e49d27a1fcf6bece6f6aec0ea46bb9112489b93d4b688fb415457bdb/iniconfig-2.3.1-py3-none-any.whl", hash = "sha256:9121e2c1fdb355232495be3194c8dfe87ccc2d5dee45947b78e68f499790d7a7", size = 7552, upload-time = "2026-10-06T22:48:36.959Z" }, ] +[[package]] +name = "isaaccapture-manus-calibration" +version = "0.0.0" +source = { directory = "../../src/plugins/manus/calibration/python" } + [[package]] name = "jsonschema" version = "4.26.0" diff --git a/examples/manus_calibration/README.md b/examples/manus_calibration/README.md index 292f0fdb83..f5410ae6d4 100644 --- a/examples/manus_calibration/README.md +++ b/examples/manus_calibration/README.md @@ -27,11 +27,13 @@ offsets are separate operations. ## Run -The page needs only Python 3.11 or newer and the standard library: +The page needs only Python 3.11 or newer and the standard library. The worker supervisor and file storage +come from the `isaaccapture-manus-calibration` library in `src/plugins/manus/calibration/python`; install it +(`pip install ../../src/plugins/manus/calibration/python`) or put it on `PYTHONPATH` as below: ```bash cd examples/manus_calibration -PYTHONPATH=python python3 -m isaaccapture_examples.manus_calibration \ +PYTHONPATH=python:../../src/plugins/manus/calibration/python python3 -m isaaccapture_examples.manus_calibration \ --worker /path/to/manus_glove_calibrator \ --out-dir ~/manus-calibration ``` @@ -99,7 +101,7 @@ so the operator can save them to the machine the browser runs on. Loading a file ```bash cd examples/manus_calibration python3 -m pip install pytest -PYTHONPATH=python python3 -m pytest +PYTHONPATH=python:../../src/plugins/manus/calibration/python python3 -m pytest ``` The tests drive the adapter, controller and HTTP server against a scripted stand-in for the worker, so diff --git a/examples/manus_calibration/pyproject.toml b/examples/manus_calibration/pyproject.toml index 6bb337f907..88d9cc3623 100644 --- a/examples/manus_calibration/pyproject.toml +++ b/examples/manus_calibration/pyproject.toml @@ -12,8 +12,9 @@ name = "isaaccapture-examples-manus-calibration" version = "0.0.0" # Internal example - not versioned description = "Standalone Manus glove calibration with a browser page" requires-python = ">=3.11,<3.14" -# Standard library only: calibration needs no OpenXR, CUDA, capture session or UI toolkit. -dependencies = [] +# The worker supervisor and MCAL storage live in a standard-library-only library beside the native worker; +# calibration needs no OpenXR, CUDA, capture session or UI toolkit. +dependencies = ["isaaccapture-manus-calibration"] [project.optional-dependencies] dev = [ @@ -31,4 +32,7 @@ sources = ["python"] [tool.pytest.ini_options] testpaths = ["tests"] # Run from a source tree without installing the example. -pythonpath = ["python"] +pythonpath = ["python", "../../src/plugins/manus/calibration/python"] + +[tool.uv.sources] +isaaccapture-manus-calibration = { path = "../../src/plugins/manus/calibration/python" } diff --git a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/__main__.py b/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/__main__.py index 635051a037..5b3f96af62 100644 --- a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/__main__.py +++ b/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/__main__.py @@ -14,7 +14,7 @@ from pathlib import Path from .server import PageServer -from .worker import CalibrationWorker +from isaaccapture_manus_calibration.worker import CalibrationWorker logger = logging.getLogger("isaaccapture.examples.manus_calibration") diff --git a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/controller.py b/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/controller.py index 0b362aeee3..eed5aa0082 100644 --- a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/controller.py +++ b/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/controller.py @@ -15,8 +15,8 @@ from dataclasses import dataclass, replace from enum import Enum -from . import protocol as proto -from .worker import ( +from isaaccapture_manus_calibration import protocol as proto +from isaaccapture_manus_calibration.worker import ( AttemptError, CalibrationAttempt, CalibrationOutcome, diff --git a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/server.py b/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/server.py index e090419037..8e562f3d1f 100644 --- a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/server.py +++ b/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/server.py @@ -20,10 +20,14 @@ from typing import Any from urllib.parse import parse_qs, urlsplit -from . import protocol as proto +from isaaccapture_manus_calibration import protocol as proto from .controller import CalibrationController, NotOwner, View from .wire import view_to_dict -from .worker import AttemptError, CalibrationWorker, OutcomeKind +from isaaccapture_manus_calibration.worker import ( + AttemptError, + CalibrationWorker, + OutcomeKind, +) logger = logging.getLogger("isaaccapture.examples.manus_calibration.server") diff --git a/examples/manus_calibration/tests/conftest.py b/examples/manus_calibration/tests/conftest.py index d4bdaf88f2..a61e7eb973 100644 --- a/examples/manus_calibration/tests/conftest.py +++ b/examples/manus_calibration/tests/conftest.py @@ -9,8 +9,8 @@ from pathlib import Path import pytest -from isaaccapture_examples.manus_calibration import protocol as proto -from isaaccapture_examples.manus_calibration.worker import ( +from isaaccapture_manus_calibration import protocol as proto +from isaaccapture_manus_calibration.worker import ( AttemptState, CalibrationAttempt, CalibrationOutcome, diff --git a/examples/manus_calibration/tests/test_controller.py b/examples/manus_calibration/tests/test_controller.py index cc9555b761..987b5df246 100644 --- a/examples/manus_calibration/tests/test_controller.py +++ b/examples/manus_calibration/tests/test_controller.py @@ -9,13 +9,13 @@ import pytest from conftest import commands, wait_for -from isaaccapture_examples.manus_calibration import protocol as proto from isaaccapture_examples.manus_calibration.controller import ( CalibrationController, NotOwner, Stage, ) -from isaaccapture_examples.manus_calibration.worker import AttemptError, OutcomeKind +from isaaccapture_manus_calibration import protocol as proto +from isaaccapture_manus_calibration.worker import AttemptError, OutcomeKind HOLD = { "steps": [ diff --git a/examples/manus_calibration/tests/test_protocol.py b/examples/manus_calibration/tests/test_protocol.py index e479073f77..ab783b0662 100644 --- a/examples/manus_calibration/tests/test_protocol.py +++ b/examples/manus_calibration/tests/test_protocol.py @@ -6,7 +6,7 @@ import json import pytest -from isaaccapture_examples.manus_calibration import protocol as proto +from isaaccapture_manus_calibration import protocol as proto SHA = "a" * 64 ATTEMPT = "cal-abc" diff --git a/examples/manus_calibration/tests/test_server.py b/examples/manus_calibration/tests/test_server.py index 46a0291f6e..f8f69296ea 100644 --- a/examples/manus_calibration/tests/test_server.py +++ b/examples/manus_calibration/tests/test_server.py @@ -11,7 +11,7 @@ import pytest from conftest import commands from isaaccapture_examples.manus_calibration import server as server_module -from isaaccapture_examples.manus_calibration.worker import OutcomeKind +from isaaccapture_manus_calibration.worker import OutcomeKind from web_helpers import ( CLIENT_A, CLIENT_B, diff --git a/examples/manus_calibration/tests/test_watchdog.py b/examples/manus_calibration/tests/test_watchdog.py index d758588f99..cc2c8194d7 100644 --- a/examples/manus_calibration/tests/test_watchdog.py +++ b/examples/manus_calibration/tests/test_watchdog.py @@ -12,8 +12,8 @@ import pytest from conftest import BOTH, commands, wait_for -from isaaccapture_examples.manus_calibration import protocol as proto -from isaaccapture_examples.manus_calibration.worker import ( +from isaaccapture_manus_calibration import protocol as proto +from isaaccapture_manus_calibration.worker import ( AttemptEnded, AttemptState, ConfirmError, diff --git a/examples/manus_calibration/tests/test_wire.py b/examples/manus_calibration/tests/test_wire.py index 5c5e4e0a9d..2150bca42d 100644 --- a/examples/manus_calibration/tests/test_wire.py +++ b/examples/manus_calibration/tests/test_wire.py @@ -8,10 +8,10 @@ import json import pytest -from isaaccapture_examples.manus_calibration import protocol as proto from isaaccapture_examples.manus_calibration.controller import Stage, View from isaaccapture_examples.manus_calibration.wire import view_to_dict -from isaaccapture_examples.manus_calibration.worker import ( +from isaaccapture_manus_calibration import protocol as proto +from isaaccapture_manus_calibration.worker import ( CalibrationOutcome, OutcomeKind, ) diff --git a/examples/manus_calibration/tests/test_worker.py b/examples/manus_calibration/tests/test_worker.py index 7a14e74539..c652839fbe 100644 --- a/examples/manus_calibration/tests/test_worker.py +++ b/examples/manus_calibration/tests/test_worker.py @@ -13,8 +13,8 @@ import pytest from conftest import BOTH, Folder, commands, drive, wait_for -from isaaccapture_examples.manus_calibration import protocol as proto -from isaaccapture_examples.manus_calibration.worker import ( +from isaaccapture_manus_calibration import protocol as proto +from isaaccapture_manus_calibration.worker import ( AttemptEnded, AttemptError, AttemptState, diff --git a/src/plugins/manus/calibration/python/isaaccapture_manus_calibration/__init__.py b/src/plugins/manus/calibration/python/isaaccapture_manus_calibration/__init__.py new file mode 100644 index 0000000000..99f5acce51 --- /dev/null +++ b/src/plugins/manus/calibration/python/isaaccapture_manus_calibration/__init__.py @@ -0,0 +1,8 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Supervision of the Manus glove calibration worker, and storage of the MCAL files it exports. + +The worker executable is ``manus_glove_calibrator`` (see ``src/plugins/manus/calibration``). This package +has no dependencies, so a UI or a server can embed it without a vendor SDK being installed. +""" diff --git a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/protocol.py b/src/plugins/manus/calibration/python/isaaccapture_manus_calibration/protocol.py similarity index 100% rename from examples/manus_calibration/python/isaaccapture_examples/manus_calibration/protocol.py rename to src/plugins/manus/calibration/python/isaaccapture_manus_calibration/protocol.py diff --git a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/worker.py b/src/plugins/manus/calibration/python/isaaccapture_manus_calibration/worker.py similarity index 99% rename from examples/manus_calibration/python/isaaccapture_examples/manus_calibration/worker.py rename to src/plugins/manus/calibration/python/isaaccapture_manus_calibration/worker.py index 1c098a9031..ce774a2756 100644 --- a/examples/manus_calibration/python/isaaccapture_examples/manus_calibration/worker.py +++ b/src/plugins/manus/calibration/python/isaaccapture_manus_calibration/worker.py @@ -29,7 +29,7 @@ from . import protocol as proto -logger = logging.getLogger("isaaccapture.examples.manus_calibration.worker") +logger = logging.getLogger("isaaccapture.manus_calibration.worker") _LOG_LINE_CHARS = 1000 _DRAIN_CHUNK = 4096 diff --git a/src/plugins/manus/calibration/python/pyproject.toml b/src/plugins/manus/calibration/python/pyproject.toml new file mode 100644 index 0000000000..02d64af6fb --- /dev/null +++ b/src/plugins/manus/calibration/python/pyproject.toml @@ -0,0 +1,17 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "isaaccapture-manus-calibration" +version = "0.0.0" # Internal library - not versioned +description = "Supervisor, wire protocol and MCAL storage for the Manus glove calibration worker" +requires-python = ">=3.11" +# Standard library only, so any UI or server can use it without the Manus SDK. +dependencies = [] + +[tool.hatch.build.targets.wheel] +packages = ["isaaccapture_manus_calibration"]