feat: Add multi-camera support with LiveController camera switching #484

hongquanli · 2026-01-28T11:54:17Z

Summary

Phase 2 of multi-camera support implementation, building on the schema/models from Phase 1.

Camera config factory: Generate per-camera configs from registry + base template
Multi-camera Microscope: Store cameras as Dict[int, AbstractCamera] with backward compatibility
LiveController camera switching: Switch active camera based on channel.camera field
Channel Group UI: Master-detail editor dialog for configuring channel groups
Validation: Duplicate channel detection in channel groups

Key Implementation Details

LiveController Multi-Camera Switching

Tracks _active_camera_id for current camera
set_microscope_mode() checks channel.camera and switches if different
Race condition protection: Sets is_live=False during switch to prevent frame callback conflicts
Early validation: Checks target camera exists before any state changes (prevents stuck state)
Explicit initial_camera_id parameter removes fragile initialization order dependency

Microscope Changes

Cameras stored in Dict[int, AbstractCamera] with DEFAULT_SINGLE_CAMERA_ID = 1 constant
Backward compatible: single camera wrapped in dict automatically
get_camera(id), get_camera_ids(), get_camera_count() API
Cleanup: stop_streaming() before close() in Microscope.close()

Channel Group Editor UI

ChannelGroupEditorDialog: Master-detail dialog for managing groups
ChannelGroupDetailWidget: Edit sync mode, channels, timing offsets
Dirty state tracking with unsaved changes warning on close
Deep copy of config to prevent accidental mutation

Test Plan

28 unit tests passing
Manual test: Run main_hcs.py --simulation
Manual test: Settings > Advanced > Channel Group Configuration
Manual test: Create group, add channels, save, verify YAML persists
Manual test: Camera selector shows camera column when 2+ cameras configured

🤖 Generated with Claude Code

Copilot

Pull request overview

Implements Phase 2 of multi-camera support (per-camera config generation, multi-camera storage in Microscope, and live camera switching in LiveController) and adds a new Channel Group configuration UI with validation—while also introducing a substantial multi-port illumination protocol + firmware/versioning update.

Changes:

Add multi-camera infrastructure: camera config factory, Microscope multi-camera API, and LiveController camera switching keyed by channel.camera.
Add Channel Group configuration UI (master-detail editor) and validation (including duplicate-channel detection).
Add multi-port illumination support across Python + firmware (new command IDs 34–39, SimSerial behavior, tests, and documentation).

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
software/control/_def.py	Adds multi-port illumination command IDs and port/source mapping helpers.
software/control/core/live_controller.py	Adds active camera tracking and camera switching during mode changes.
software/control/firmware_sim_serial.py	Exposes multi-port illumination commands to firmware simulator wrapper.
software/control/gui_hcs.py	Adds Channel Group Configuration entry to Advanced menu (multi-camera only).
software/control/lighting.py	Adds multi-port illumination API to `IlluminationController` and port state tracking.
software/control/microcontroller.py	Adds multi-port illumination commands, firmware version parsing, and SimSerial multi-port behavior.
software/control/microscope.py	Stores cameras as dict + adds camera APIs; initializes/closes all cameras; wires `LiveController` initial camera ID.
software/control/models/acquisition_config.py	Adds duplicate channel detection to `validate_channel_group`.
software/control/widgets.py	Adds camera-ID-based camera combo population and Channel Group editor dialog UI.
software/docs/illumination-control.md	Documents legacy vs multi-port illumination APIs and mapping details.
software/squid/camera/config_factory.py	Adds per-camera `CameraConfig` factory and primary camera selection helper.
software/tests/control/test_microscope_cameras.py	Adds unit tests for multi-camera config factory, `Microscope` camera API, `LiveController` switching, and group validation.
software/tests/test_multiport_illumination.py	Adds core tests for multi-port illumination simulation and controller behavior.
software/tests/test_multiport_illumination_bugs.py	Adds targeted tests for common multi-port illumination failure modes.
software/tests/test_multiport_illumination_edge_cases.py	Adds edge case + integration-style tests for multi-port illumination and mappings.
software/tests/test_multiport_illumination_protocol.py	Adds protocol byte agreement tests for multi-port illumination commands.
firmware/controller/src/commands/commands.cpp	Registers multi-port illumination command callbacks.
firmware/controller/src/commands/light_commands.cpp	Implements multi-port illumination command callbacks.
firmware/controller/src/commands/light_commands.h	Declares multi-port illumination callbacks.
firmware/controller/src/constants.h	Adds firmware version constants.
firmware/controller/src/constants_protocol.h	Adds protocol constants for multi-port illumination commands.
firmware/controller/src/functions.cpp	Adds per-port tracking updates and implements multi-port port control helpers.
firmware/controller/src/functions.h	Declares multi-port illumination control functions and includes mapping util.
firmware/controller/src/globals.cpp	Adds global arrays for per-port illumination state/intensity.
firmware/controller/src/globals.h	Declares globals for multi-port illumination state/intensity.
firmware/controller/src/serial_communication.cpp	Encodes firmware version in response byte 22.
firmware/controller/src/utils/illumination_mapping.h	Adds pure mapping utilities for legacy source ↔ port index.
firmware/controller/src/utils/illumination_utils.h	Adds pure utilities for intensity conversion, version encoding, mask helpers.
firmware/controller/test/test_command_layout/test_command_layout.cpp	Adds tests for command/response byte layouts (including version byte).
firmware/controller/test/test_illumination_mapping/test_illumination_mapping.cpp	Adds tests for legacy source ↔ port mapping correctness (incl D3/D4 swap).
firmware/controller/test/test_illumination_utils/test_illumination_utils.cpp	Adds tests for intensity conversion, version encoding, and mask helpers.
firmware/controller/test/test_protocol/test_protocol.cpp	Updates protocol tests to include new commands and W2 axis.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-28T12:22:21Z

firmware/controller/src/functions.cpp

 void turn_off_illumination()
 {
+  // Update per-port state for D1-D5 sources (backward compatibility with new multi-port tracking)
+  int port_index = illumination_source_to_port_index(illumination_source);
+  if (port_index >= 0)
+    illumination_port_is_on[port_index] = false;


Firmware legacy turn_off_illumination() only turns off the currently selected legacy source pin, but with multi-port illumination it’s now possible for other ports to remain ON (e.g., after SET_MULTI_PORT_MASK / TURN_ON_PORT). This conflicts with the Python SimSerial/tests which assume TURN_OFF_ILLUMINATION is a global shutdown. Consider making legacy TURN_OFF_ILLUMINATION call turn_off_all_ports() (or otherwise guarantee all ports are OFF) to avoid leaving illumination outputs active unintentionally and to keep firmware/simulation behavior consistent.

Copilot · 2026-01-28T12:22:22Z

software/control/microcontroller.py

+        self.log.debug(f"[MCU] set_multi_port_mask: port_mask=0x{port_mask:04X}, on_mask=0x{on_mask:04X}")
+        cmd = bytearray(self.tx_buffer_length)
+        cmd[1] = CMD_SET.SET_MULTI_PORT_MASK
+        cmd[2] = (port_mask >> 8) & 0xFF
+        cmd[3] = port_mask & 0xFF


set_multi_port_mask() doesn’t validate port_mask/on_mask bounds. Negative values or values > 0xFFFF will be silently truncated by the shifts/masks, potentially sending an unintended pattern (e.g., -1 becomes 0xFFFF). Add explicit validation (type, 0..0xFFFF) or clamp/mask inputs before logging/sending.

Copilot · 2026-01-28T12:22:22Z

software/control/microcontroller.py

    CMD_SET.SET_STROBE_DELAY: "SET_STROBE_DELAY",
    CMD_SET.SET_AXIS_DISABLE_ENABLE: "SET_AXIS_DISABLE_ENABLE",
    CMD_SET.SET_PIN_LEVEL: "SET_PIN_LEVEL",
+    # Multi-port illumination commands (firmware v1.0+)
+    CMD_SET.SET_PORT_INTENSITY: "SET_PORT_INTENSITY",
+    CMD_SET.TURN_ON_PORT: "TURN_ON_PORT",
+    CMD_SET.TURN_OFF_PORT: "TURN_OFF_PORT",
+    CMD_SET.SET_PORT_ILLUMINATION: "SET_PORT_ILLUMINATION",
+    CMD_SET.SET_MULTI_PORT_MASK: "SET_MULTI_PORT_MASK",
+    CMD_SET.TURN_OFF_ALL_PORTS: "TURN_OFF_ALL_PORTS",


This PR introduces substantial multi-port illumination changes (new MCU commands 34–39, firmware version encoding, SimSerial behavior, extensive illumination tests/docs) that aren’t described in the PR title/description (which focuses on multi-camera + channel group UI). Please either update the PR description to cover the illumination/firmware scope, or split these illumination changes into a separate PR for clearer review and rollout.

Copilot · 2026-01-28T12:22:22Z

software/control/widgets.py

+        layout = QHBoxLayout(self)
+
+        # Left panel: group list
+        left_panel = QWidget()
+        left_layout = QVBoxLayout(left_panel)


ChannelGroupEditorDialog._setup_ui() creates a QHBoxLayout with self as parent, then later adds that same layout into another layout and calls setLayout() again. In Qt, reparenting layouts like this triggers warnings and can prevent the bottom Save/Cancel buttons (or the whole dialog layout) from rendering correctly. Create the top-level layout only once (e.g., a single QVBoxLayout on self) and add the horizontal splitter layout as a child layout without giving it an initial parent.

Copilot · 2026-01-28T12:22:23Z

software/control/microcontroller.py

+            # Turn off all ports (matches firmware behavior)
+            for i in range(SimSerial.NUM_ILLUMINATION_PORTS):
+                self.port_is_on[i] = False


SimSerial treats legacy TURN_OFF_ILLUMINATION as turning off all illumination ports and the comment says this matches firmware behavior, but the firmware turn_off_illumination() implementation only turns off the currently selected legacy source (D1-D5) and leaves any ports enabled via multi-port commands untouched. This makes simulation/tests disagree with real hardware behavior; either update firmware to make TURN_OFF_ILLUMINATION shut down all ports, or change SimSerial/tests to match the firmware semantics.

Suggested change

# Turn off all ports (matches firmware behavior)

for i in range(SimSerial.NUM_ILLUMINATION_PORTS):

self.port_is_on[i] = False

# Turn off only the currently selected legacy source's port

if self._illumination_source is not None:

port_index = source_code_to_port_index(self._illumination_source)

if 0 <= port_index < SimSerial.NUM_ILLUMINATION_PORTS:

self.port_is_on[port_index] = False

Copilot · 2026-01-28T12:22:23Z

software/squid/camera/config_factory.py

+import logging
+from typing import Dict, Optional
+
+from control.models import CameraRegistryConfig, CameraDefinition


Import of 'CameraDefinition' is not used.

Suggested change

from control.models import CameraRegistryConfig, CameraDefinition

from control.models import CameraRegistryConfig

Phase 2 of multi-camera support implementation: - Add camera config factory for generating per-camera configs from registry - Support multiple cameras in Microscope class with Dict[int, AbstractCamera] - Add channel-driven camera switching in LiveController.set_microscope_mode() - Add Channel Group editor UI (ChannelGroupEditorDialog) for multi-camera sync - Add duplicate channel validation in validate_channel_group() Key changes: - LiveController tracks active camera ID and switches based on channel.camera - Race condition protection: is_live=False during camera switch - Early validation: check target camera exists before any state changes - Explicit camera ID passing to LiveController (removes initialization fragility) - Proper cleanup: stop_streaming before close in Microscope.close() Includes 28 unit tests covering camera config factory, Microscope camera API, LiveController multi-camera switching, and channel group validation. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Change Add buttons to not use "+" prefix - Move Remove button next to Add (removes selected row) - Remove per-row remove buttons from table - Use ChannelPickerDialog for adding channels (no empty rows) - Simplify table to 3 columns (Channel, Camera, Offset) - Hide row numbers in channel table - Improve overall dialog layout with proper spacing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

When switching between channels assigned to different cameras, the frame callback must be moved from the old camera to the new camera for the live view to display frames from the correct camera. Changes: - LiveController: Add on_camera_switched callback for camera switch notifications - LiveController: Filter out channels without camera assignment in multi-camera systems - gui_hcs: Register callback handler to move frame callback on camera switch - gui_hcs: Store frame callback ID for proper removal - AbstractCamera: Add thread-safe locking for _frame_callbacks list Thread safety rationale: - Camera SDK threads call _propogate_frame() which iterates callbacks - GUI/TCP threads may add/remove callbacks during camera switches - Without locking, concurrent access could cause RuntimeError or missed callbacks - Lock is acquired for list modification; iteration copies list under lock then releases lock before calling callbacks (avoids deadlock) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add Phase 1.5 of multi-camera support following the filter wheel pattern: INI Settings (control/_def.py): - USE_MULTI_CAMERA: Enable/disable multi-camera mode - MULTI_CAMERA_IDS: List of camera IDs to instantiate - MULTI_CAMERA_SNS: Camera ID to serial number mapping (required) All cameras must be the same type (specified by CAMERA_TYPE). Camera Configuration UI (control/widgets.py): - New CameraConfiguratorDialog for editing camera names - Serial numbers shown read-only (from INI) - Names saved to machine_configs/cameras.yaml - Menu item under Settings > Advanced (when USE_MULTI_CAMERA=True) Config Factory (squid/camera/config_factory.py): - Updated create_camera_configs() to use INI serial numbers - Validates all camera IDs have serial numbers in MULTI_CAMERA_SNS - Specific error message for empty MULTI_CAMERA_SNS dict - Registry (cameras.yaml) optional, provides friendly names only Tests: - Added test_empty_sns_dict_raises for empty SNS validation - Updated existing tests with explicit USE_MULTI_CAMERA mocking Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Previously, ToupcamCamera.__init__ hardcoded index=0 when opening the camera, ignoring the serial number from the config. This caused the second camera to fail initialization because index=0 was already open. Changes: - Pass serial number from config to _open() when available - In _open(), use the matched device index when serial number is provided - Fall back to index=0 for backward compatibility with single-camera systems Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Multi-camera trigger channel support: - Add trigger_channel field to CameraDefinition model (cameras.yaml) - Add get_trigger_channel() method to CameraRegistryConfig - Create per-camera hardware trigger functions with correct channel - Update send_hardware_trigger calls in microscope.py and auto_focus_worker.py - Add trigger channel column to CameraConfiguratorDialog UI Trigger mode UI synchronization: - When switching cameras, read the new camera's acquisition mode - Update LiveController.trigger_mode to match - Add on_trigger_mode_changed callback to notify UI - Add update_trigger_mode_display() to update dropdown without recursion This ensures each camera uses its configured trigger output channel and the UI reflects the actual trigger mode when switching cameras. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Tests hardware trigger wiring for multi-camera Toupcam systems. Enumerates cameras directly and tests both channel mappings to determine correct camera-to-trigger-channel assignment. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Detects unselected channel entries and reports which rows need attention. Also excludes empty names from duplicate checking to avoid false positives. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

1. Fix ChannelPickerDialog call - pass channel names (not objects) and include existing_names argument with correct parent position 2. Prevent creating channel groups when no acquisition channels exist - show warning dialog instead of creating invalid group 3. Add thread safety to camera switch callback - use QTimer.singleShot for consistency with other cross-thread GUI updates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add try-except to _on_live_camera_switched callback to log errors instead of silently failing when camera frame callback switch fails - Convert _on_trigger_mode_changed lambda to function with error handling to prevent silent UI update failures - Add debug logging to save_last_used_saving_path for diagnosing repeated cache write failures Qt's event loop silently swallows exceptions in QTimer.singleShot callbacks, making debugging extremely difficult without explicit handling. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

1. Use early-return pattern for is_switching_mode guard in LiveControlWidget - Consistent with NapariLiveWidget pattern - Reduces nesting and follows Python conventions 2. Add get_trigger_channel_for_camera() helper to Microscope - Consolidates duplicated trigger channel lookup pattern - Used in acquire_image() and auto_focus_worker.py 3. Use dict mapping for camera-to-trigger mode conversion - Replaces verbose if-elif chain in live_controller._switch_camera() - More concise and maintainable Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

1. Use proper Callable type annotations for camera switch callbacks - Replaces `callable` (runtime check) with `Callable` (type annotation) - Adds type aliases for better documentation 2. Add warning log when get_trigger_channel falls back to default - Logs when camera_id not found in registry - Helps diagnose misconfiguration issues 3. Return bool from set_microscope_mode to indicate success/failure - Returns False for None config or invalid camera ID - Returns True on success - Callers can optionally check return value Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Prevents accidentally committing worktree contents to repository. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

hongquanli requested a review from Copilot January 28, 2026 12:10

Copilot started reviewing on behalf of hongquanli January 28, 2026 12:11 View session

Copilot AI reviewed Jan 28, 2026

View reviewed changes

hongquanli and others added 13 commits February 5, 2026 09:45

fix: Add validation for empty channel names in channel groups

1367fe3

Detects unselected channel entries and reports which rows need attention. Also excludes empty names from duplicate checking to avoid false positives. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add worktrees/ to .gitignore

d7ec251

Prevents accidentally committing worktree contents to repository. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

hongquanli force-pushed the refactor/laser-port-naming branch from c0c14c3 to d7ec251 Compare February 5, 2026 17:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add multi-camera support with LiveController camera switching #484

feat: Add multi-camera support with LiveController camera switching #484

Uh oh!

hongquanli commented Jan 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

-            # Turn off all ports (matches firmware behavior)
-            for i in range(SimSerial.NUM_ILLUMINATION_PORTS):
-                self.port_is_on[i] = False
+            # Turn off only the currently selected legacy source's port
+            if self._illumination_source is not None:
+                port_index = source_code_to_port_index(self._illumination_source)
+                if 0 <= port_index < SimSerial.NUM_ILLUMINATION_PORTS:
+                    self.port_is_on[port_index] = False

	from control.models import CameraRegistryConfig, CameraDefinition
	from control.models import CameraRegistryConfig

feat: Add multi-camera support with LiveController camera switching #484

Are you sure you want to change the base?

feat: Add multi-camera support with LiveController camera switching #484

Uh oh!

Conversation

hongquanli commented Jan 28, 2026

Summary

Key Implementation Details

LiveController Multi-Camera Switching

Microscope Changes

Channel Group Editor UI

Test Plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant