An overhead camera system that tracks and autonomously steers robots around a tabletop arena. A ceiling-mounted camera watches the playing field, computer vision identifies robots by their AprilTag markers and detects obstacles with YOLOv8, an A* path planner charts a course around them, and serial commands drive each robot to its destination.
Built for Vorpal the Hexapod but works with any robot that accepts single-character serial commands over an Arduino.
- AprilTag identification — each robot wears a unique tag; the system tracks multiple robots simultaneously
- YOLOv8 obstacle detection — real-time detection of objects on the playing field
- A* path planning — global paths with velocity-obstacle local avoidance and multi-robot coordination
- Vue 3 web UI — live MJPEG video with canvas overlay showing tracks, paths, and obstacle boundaries; click anywhere to send a robot there
- Web-based calibration — four-point homography mapping from pixel coordinates to real-world positions
- Arduino serial control — single ASCII commands (F/B/L/R/S) at 9600 baud
- Demo mode — run the full UI with simulated robots, no camera or hardware required
Prerequisites: Go 1.24+, Node.js 18+, OpenCV 4.x
# Install OpenCV (Linux or macOS)
./scripts/install-dependencies.sh
# Build the Vue UI and Go backend
./scripts/build.sh
# Run in demo mode (no camera or Arduino needed)
./scripts/run.sh --demoOpen http://localhost:9086 to see the web UI.
OpenCV 4.x is required (via GoCV bindings).
Linux (Ubuntu/Debian):
./scripts/install-dependencies.shmacOS (Homebrew):
brew install opencvThe wrapper scripts (build.sh, run.sh, test.sh) automatically set the OpenCV environment variables (OPENCV_DIR, LD_LIBRARY_PATH, PKG_CONFIG_PATH). Always use the wrapper scripts rather than running go build directly.
A USB webcam is strongly recommended over WiFi/IP cameras. WiFi cameras introduce 1--1.5 second frame delivery gaps due to WiFi jitter, causing steering pauses. USB cameras deliver frames at a consistent ~33 ms interval.
In config/tracking_config.yaml:
cameras:
- id: 0
name: "usb_camera"
width: 1280
height: 720
fps: 30Connect the Arduino over USB. The tracker sends single ASCII characters at 9600 baud:
| Command | Char | Description |
|---|---|---|
| FORWARD | F | Move forward |
| BACKWARD | B | Move backward |
| LEFT | L | Rotate CCW |
| RIGHT | R | Rotate CW |
| STOP | S | Stop immediately |
Format: {char}\r\n — for example F\r\n.
# Auto-detect serial port
./scripts/run.sh
# Specify a port
./scripts/run.sh --port /dev/ttyUSB0 # Linux
./scripts/run.sh --port /dev/cu.usbserial-0001 # macOS
# List available ports
./scripts/run.sh --list-portsA pre-trained YOLOv8n ONNX model is included at assets/yolov8n.onnx. No export step is needed.
macOS blocks camera access for processes started via SSH. Use the LaunchAgent service manager instead:
./scripts/service.sh install # one-time setup
./scripts/service.sh start # start/stop from any terminal, including SSH
./scripts/service.sh stop
./scripts/service.sh status
./scripts/service.sh log # tail the log file
./scripts/service.sh uninstallAfter a reboot, someone must log in to the Mac GUI before the LaunchAgent is available. Enable automatic login in System Settings > Users & Groups to make this hands-off.
| Option | Price | Resolution | FOV | Notes |
|---|---|---|---|---|
| Logitech C920s Pro | ~$50 | 1080p/30 fps | 78° | Recommended. Manual focus via UVC, excellent OpenCV compatibility. |
| Logitech Brio 100 | ~$25 | 1080p/30 fps | 58° | Budget pick. Fixed focus works well overhead. Narrower FOV limits arena size. |
| ELP USB (wide-angle) | ~$25--40 | 1080p/30 fps | 100--120° | Good for larger arenas. Avoid >150° — distortion degrades AprilTag detection. |
The AprilTag should be at least ~40 px across in the image for reliable detection. Avoid 4K cameras — the detection pipeline can't use the extra pixels at ~5 fps.
| Platform | AprilTag | YOLOv8 nano | Track + Plan | Effective FPS |
|---|---|---|---|---|
| Mac Mini M4 | ~40--70 ms | ~20--35 ms | ~15 ms | ~8--12 fps |
| x86 desktop (4+ cores) | ~100--150 ms | ~50--70 ms | ~33 ms | ~5 fps |
| Raspberry Pi 5 | ~150--250 ms | ~100--200 ms | ~50 ms | ~2--3 fps |
Mac Mini M4 is the fastest option (roughly 2--3x faster than a typical x86 desktop). Install OpenCV via Homebrew. Serial ports are /dev/cu.usbserial-* or /dev/tty.usbmodem-*.
Raspberry Pi 5 works but navigation is noticeably more sluggish. If using Pi 5:
- Disable YOLO (
conf_thres: 1.0) if you don't need dynamic obstacle detection — saves ~100--200 ms per frame - Increase
quad_decimatefrom 2.0 to 3.0 for faster AprilTag detection at slight accuracy cost - Use the 8 GB RAM model (4 GB is tight with YOLO loaded)
Budget alternative: Intel N100-based mini PCs (~$100--150) deliver near-desktop performance in a small form factor and run the standard Linux build without modification.
Data flows through a pipeline:
Camera → Detection → Tracking → Planning → Controller
The RobotSystem struct in cmd/main.go owns and orchestrates all subsystems.
robot_tracker_go/
├── cmd/ # Application entry point
├── internal/
│ ├── camera/ # Camera abstraction
│ ├── config/ # YAML configuration loading
│ ├── controller/ # Arduino serial communication
│ ├── detection/ # AprilTag + YOLOv8 detection
│ ├── planning/ # A* path planning + velocity obstacles
│ ├── position/ # Homography calibration (pixel ↔ world)
│ ├── tracking/ # ByteTrack multi-object tracker + Kalman filter
│ ├── ui/ # Gin HTTP server, MJPEG stream, WebSocket, REST API
│ └── utils/ # Logging utilities
├── ui/ # Vue 3 + TypeScript frontend (Pinia stores, canvas overlay)
├── Arduino/ # Arduino gamepad firmware
├── assets/ # YOLOv8 ONNX model
├── config/ # Configuration YAML files
└── scripts/ # Build, run, test, and install wrappers
Edit config/tracking_config.yaml to configure:
- Robot definitions (tag IDs, physical sizes, speeds)
- Detection parameters (confidence thresholds, decimation)
- Planning settings (step size, safety margins)
- Camera settings (resolution, FPS, source)
# Run all tests (Go + Vue)
./scripts/test.sh --verbose
# Run a specific Go package
go test -v ./internal/planning/
# Run a single Go test
go test -v -run TestFunctionName ./internal/planning/
# Vue tests
cd ui && npm run test:run # CI mode
cd ui && npm run test:coverage # with coverage