BFMC SimulatorROS2 — Complete Setup Guide

This repository is the updated ROS 2 version of the UniPD DriveOps hardware stack.

Complete guide for installing, configuring and running the BFMC simulator with ROS2 Jazzy and Gazebo Harmonic, including the simulated OAK-D camera integration.

Quick hardware-free localization test

The simulator includes the separate sim2real_mapping adapter, so no physical encoder, GPS, IMU, or camera is needed. It starts with the simulator in one command and publishes the real localization topic contract, including a approximately 1-second artificial GPS/tag delay (randomized from 0.9 to 1.1 s):

source /opt/ros/jazzy/setup.bash
cd ~/BFMC_2027/Simulator_UniPD-DriveOps
colcon build --symlink-install
source install/setup.bash
ros2 launch sim_pkg map_with_car.launch camera_view:=false

The simulator, Brain, and Localization use one canonical UniPD map pair:

Brain_UniPD-DriveOps/.../2024_VerySmall.png
Brain_UniPD-DriveOps/.../final_graph.graphml

Exact copies are installed with the simulator track and localization map packages. The shared frame is 22.612256424738774 m × 15.0 m; the normal car spawn is GraphML node 472 at (12.9154898505, 2.0165327404).

Optional extra GPS Gaussian XY noise, in metres:

ros2 launch sim_pkg map_with_car.launch camera_view:=false \
  gps_delay_s:=1.0 gps_delay_jitter_s:=0.1 gps_xy_noise_stddev_m:=0.05

1. System Requirements

OS: Ubuntu 24.04 LTS (Noble)
ROS2: Jazzy Jalisco
RAM: 8GB minimum (16GB recommended)
GPU: any (Intel Iris Xe or better supported)

2. Installing Dependencies

2.1 ROS2 Jazzy

Follow the official guide: https://docs.ros.org/en/jazzy/Installation/Ubuntu-Install-Debians.html

# Add the ROS2 repository
sudo apt update && sudo apt install curl -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key \
  -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] \
  http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | \
  sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

sudo apt update
sudo apt install ros-jazzy-desktop -y

2.2 Gazebo Harmonic (via ROS2 Jazzy vendor packages)

Gazebo Harmonic is available through the ROS2 Jazzy vendor packages.

sudo apt install ros-jazzy-ros-gz -y

2.3 Additional ROS2 dependencies

sudo apt install \
  ros-jazzy-depth-image-proc \
  ros-jazzy-rqt-image-view \
  ros-jazzy-rqt \
  python3-colcon-common-extensions \
  python3-rosdep \
  -y

2.4 Just (task runner)

curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh \
  | bash -s -- --to ~/.local/bin

# Add to PATH
echo 'export PATH=$PATH:~/.local/bin' >> ~/.bashrc
source ~/.bashrc

# Verify
just --version

3. Workspace Setup

# Create the working directory
mkdir -p ~/BFMC_2027/
cd ~/BFMC_2027/

# Clone the repository
git clone https://github.com/UniPD-DriveOps-BFMC/SimulatorROS2.git Simulator_UniPD-DriveOps

4. Gazebo Configuration

4.1 Configuration directory

mkdir -p ~/.gz/sim/8

4.2 gui.config file

The gui.config file is already included in the cloned repository:

cp ~/BFMC_2027/Simulator_UniPD-DriveOps/"additional files"/gui.config ~/.gz/sim/8/gui.config

4.3 Fix permissions and Ogre cache

sudo chown -R $USER:$USER ~/.gz
rm -rf ~/.gz/rendering/ogre-rtshader/*
mkdir -p ~/.gz/rendering/ogre-rtshader/hisham-rtshaderlibcache

5. Building the Workspace

Build from the repository root:

cd ~/BFMC_2027/Simulator_UniPD-DriveOps
just build

The build creates the repository-local build/, install/, and log/ directories.

6. Launching the Simulation

Simulator command list

Run these commands from ~/BFMC_2027/Simulator_UniPD-DriveOps.

# Build the complete ROS 2 Jazzy workspace
just build

# Run the simulator, car, and center OAK-D camera viewer
just car

# Run with a custom camera height (metres) and downward tilt (degrees)
just car 0.22 15

# Run the map, car, signs, traffic lights, pedestrians, and obstacle cars
just all-objects

# Run all objects with a custom OAK-D pose
just all-objects 0.22 15

# Run the simulator without the Gazebo GUI or camera viewer
just headless

# Run the simulator and keyboard controller together
just run

# Open the center-camera viewer manually
just camera-view

# Run without the separate center-camera window
source /opt/ros/jazzy/setup.bash
source install/setup.bash
ros2 launch sim_pkg map_with_car.launch camera_view:=false

# Stop a foreground simulator cleanly
# Press Ctrl+C in the terminal that launched it

From the repository root, launch the simulator with:

cd ~/BFMC_2027/Simulator_UniPD-DriveOps
just car

6.1 Map and robot only

Starts: Gazebo, robot, ROS2↔Gazebo bridge, OAK-D mono converter, PointCloud2. It also opens an rqt_image_view window showing the center OAK-D RGB stream from /oak/rgb/image_rect.

To launch without the separate camera window, use:

source /opt/ros/jazzy/setup.bash
source install/setup.bash
ros2 launch sim_pkg map_with_car.launch camera_view:=false

11. Hardware-free localization testing

The simulator includes a separate sim2real_mapping package that mimics the topics normally produced by the vehicle hardware. It does not modify the simulator pose/localization source topics and does not require an encoder, GPS, IMU, or OAK-D device.

Build and source the workspace, then start the complete simulator and mapping layer together:

source /opt/ros/jazzy/setup.bash
cd ~/BFMC_2027/Simulator_UniPD-DriveOps
colcon build --symlink-install
source install/setup.bash
ros2 launch sim_pkg map_with_car.launch camera_view:=false

The launch provides the localization stack's hardware-compatible inputs:

Simulator source	Hardware-compatible output
`/encoder_odom`	`/automobile/encoder/speed`, `/automobile/encoder/distance`
`/automobile/localisation_raw`	`/automobile/localisation` JSON containing `x`, `y`, `z`, `quality`
chassis Gazebo IMU	`/automobile/imu/data`
simulated OAK-D	`/oak/rgb/image_raw`, `/oak/rgb/camera_info`

GPS/tag messages use a randomized delay from 0.9 to 1.1 seconds by default. To add extra Gaussian position noise in metres:

ros2 launch sim_pkg map_with_car.launch camera_view:=false \
  gps_delay_s:=1.0 gps_delay_jitter_s:=0.1 gps_xy_noise_stddev_m:=0.05

With the simulator still running, open a second terminal and start the hardware-free localization stack. Build its Docker image once:

cd ~/BFMC_2027/Localization_UniPD-DriveOps
./docker/build.sh

Then run localization:

./docker/run.sh --normal-start

If it reports Localization image is missing, run ./docker/build.sh first. On later simulator runs, only ./docker/run.sh --normal-start is needed. This uses Localization's restored test-1 GPS quality covariance profile and does not add simulator noise or modify the GPS coordinates.

Verify the mapped inputs and fused outputs:

source /opt/ros/jazzy/setup.bash
ros2 topic list | grep -E \
'automobile/(encoder|localisation|imu)|encoder_odom|odometry'

ros2 topic echo /automobile/localisation
ros2 topic echo /automobile/encoder/speed
ros2 topic echo /automobile/imu/data
ros2 topic echo /odometry/global

Expected outputs include /odometry/local, /odometry/global, and /automobile/gps/base_pose. For a live monitor, run from the localization workspace:

cd ~/BFMC_2027/Localization_UniPD-DriveOps
./monitor_localization.sh --no-bag --topic-publish

The raw simulator topics remain available for comparison, especially /automobile/localisation_raw and /encoder_odom.

At normal start, /encoder_odom is the simulator pose in this same map frame and node 472 is the expected nearest graph node. GPS adds its built-in sensor error before the approximately one-second mapping delay, so individual GPS and fused samples need not equal the spawn coordinate exactly.

After the simulator is running, start the localization container and Brain from their respective workspaces. The Brain simulator launch is:

cd ~/BFMC_2027/Brain_UniPD-DriveOps
source /opt/ros/jazzy/setup.bash
colcon build --symlink-install
source ~/BFMC_2027/Simulator_UniPD-DriveOps/install/setup.bash
source install/setup.bash
ros2 launch brain_bringup simulation.launch.py

For checkpoint-only path following, use ros2 launch brain_bringup checkpoint_test.launch.py. There is no separate lane_following.launch.py file.

Test 2 image-free recording and line chart

The clean Test 1 rerun is stored as Test 2:

bags/checkpoint_localization_test2_20260826_0957

Its rosbag records only localization, GPS, odometry, IMU, control, checkpoint/node state, diagnostics, and TF. RGB, stereo, depth, camera-info, and lane-mask image topics are excluded. Run the controller after starting the recorder:

ros2 launch brain_bringup checkpoint_test.launch.py

The car moved from node 472 and reached controller transitions 472 → 393 → 306. The generated Test 2 line chart compares simulator ground truth with /odometry/global. The full report contains the metrics and notes that BEV produced zero accepted poses because the synchronized stereo depth stream had zero valid lane points.

6.2 OAK-D mounting height and tilt

The default OAK-D height is 0.17551 m and its default downward tilt is 18.0 degrees. Pass alternative values when launching:

# Arguments: camera height in metres, then downward tilt in degrees
just car 0.20 18

The equivalent ROS 2 command is:

source /opt/ros/jazzy/setup.bash
source install/setup.bash
ros2 launch sim_pkg map_with_car.launch \
  camera_height:=0.20 camera_tilt_deg:=18

Valid tilt values range from -90 to 90 degrees. Positive values tilt the camera downward; negative values tilt it upward. The selected pose applies to the RGB, stereo, depth, and OAK-D IMU sensors as one rigid assembly.

6.3 Map + all objects (signs, traffic lights, pedestrians, obstacle cars)

just all-objects

Note: all the objects are in the wrong positions, since they are referred to the old map

6.3 Monitor RAM usage

In a second terminal:

watch -n 2 free -h

6.4 Free RAM if needed

pkill -9 gz; pkill -9 ruby; pkill -9 ros2; pkill -9 python3
sudo sync && sudo sysctl vm.drop_caches=3
free -h

7. Vehicle Control

With the simulation running, open a second terminal:

source /opt/ros/jazzy/setup.bash
source ~/BFMC_2027/Simulator_UniPD-DriveOps/install/setup.bash

7.1 Steering

# Steer left (positive angle in degrees, range -25/+25)
ros2 topic pub /automobile/command std_msgs/msg/String \
  "data: '{\"action\": \"2\", \"steerAngle\": 15.0}'" --once

# Steer right
ros2 topic pub /automobile/command std_msgs/msg/String \
  "data: '{\"action\": \"2\", \"steerAngle\": -15.0}'" --once

# Center steering
ros2 topic pub /automobile/command std_msgs/msg/String \
  "data: '{\"action\": \"2\", \"steerAngle\": 0.0}'" --once

7.2 Speed

# Move forward (speed in m/s)
ros2 topic pub /automobile/command std_msgs/msg/String \
  "data: '{\"action\": \"1\", \"speed\": 0.05}'" --once

# Brake / stop
ros2 topic pub /automobile/command std_msgs/msg/String \
  "data: '{\"action\": \"1\", \"speed\": 0.0}'" --once

# Reverse
ros2 topic pub /automobile/command std_msgs/msg/String \
  "data: '{\"action\": \"1\", \"speed\": -0.05}'" --once

7.3 Keyboard control (built-in example)

just control_example
# W/A/S/D to move, ESC to exit

7.4 Vehicle topics reference

Topic	Type	Direction	Description
`/automobile/command`	`std_msgs/String`	ROS2 → Gazebo	Steering / speed commands
`/automobile/feedback`	`std_msgs/String`	Gazebo → ROS2	Vehicle feedback
`/automobile/IMU`	`geometry_msgs/Vector3`	Gazebo → ROS2	Roll, pitch, yaw
`/automobile/localisation`	`geometry_msgs/Pose`	Gazebo → ROS2	GPS position

8. Simulated OAK-D Camera

The simulation includes a full OAK-D camera with the following sensors:

Topic	Encoding	Resolution	Hz	Description
`/automobile/image_raw`	rgb8	640×480	~19	Original camera (backward compat.)
`/oak/rgb/image_raw`	rgb8	1920×1080	~19	RGB (simulated IMX378)
`/oak/rgb/image_rect`	rgb8	1920×1080	~19	Rectified RGB; identical to raw in the ideal pinhole simulation
`/oak/rgb/camera_info`	CameraInfo	—	~19	RGB calibration
`/oak/stereo/image_raw`	32FC1	1280×800	~20	Depth in meters (float32)
`/oak/stereo/image_fixed`	32FC1	1280×800	~20	Depth with corrected frame_id
`/oak/stereo/camera_info`	CameraInfo	—	~20	Depth calibration
`/oak/left/image_raw`	rgb8	640×400	~23	Left mono raw
`/oak/left/image_mono`	mono8	640×400	~23	Left mono grayscale (OV9282)
`/oak/left/image_rect`	mono8	640×400	~23	Rectified left mono image
`/oak/left/camera_info`	CameraInfo	—	~23	Left mono calibration
`/oak/right/image_raw`	rgb8	640×400	~23	Right mono raw
`/oak/right/image_mono`	mono8	640×400	~23	Right mono grayscale (OV9282)
`/oak/right/image_rect`	mono8	640×400	~23	Rectified right mono image
`/oak/right/camera_info`	CameraInfo	—	~23	Right mono calibration
`/oak/imu/data`	Imu	—	sensor rate	Accelerometer and gyroscope
`/oak/points`	PointCloud2	—	~12	RGB-coloured point cloud from aligned depth

These are the simulated equivalents of the OAK-D ROS RGBD, stereo, IMU, and point-cloud outputs. Image transport suffixes such as /compressed are created on demand by ROS image transport plugins. Neural-network detections, feature tracking, magnetometer, thermal, and VIO topics are not advertised because the simulated OAK-D does not currently model those producers.

8.1 Check active topics

ros2 topic list | grep oak

8.2 Check encoding and resolution

# RGB
ros2 topic echo /oak/rgb/image_raw --once | grep -E "height|width|encoding"

# Depth
ros2 topic echo /oak/stereo/image_raw --once | grep -E "height|width|encoding"

# Left mono (should be mono8)
ros2 topic echo /oak/left/image_mono --once | grep -E "height|width|encoding"

8.3 Check frame rates

ros2 topic hz /oak/rgb/image_raw &
ros2 topic hz /oak/stereo/image_raw &
ros2 topic hz /oak/left/image_mono &
wait

9. PointCloud2

The XYZRGB PointCloud2 is generated automatically by combining depth + RGB using depth_image_proc.

Topic	Type	Hz	Description
`/oak/points`	`sensor_msgs/PointCloud2`	~12	XYZRGB PointCloud

9.1 Verify

ros2 topic info /oak/points
ros2 topic hz /oak/points

9.2 Visualize in RViz2

rviz2

Inside RViz2:

Click Add → By topic → /oak/points → PointCloud2 → OK
Set Fixed Frame by typing manually: automobile/camera/link_camera/oak_rgb
In the PointCloud2 panel set Color Transformer → RGB8

10. Visualization with RViz2 and rqt

10.1 View all cameras with rqt

ros2 run rqt_image_view rqt_image_view

Select the desired topic from the dropdown:

/oak/rgb/image_raw — color RGB image
/oak/stereo/image_raw — depth map (grayscale)
/oak/left/image_mono — left grayscale mono
/oak/right/image_mono — right grayscale mono

10.2 Gazebo navigation controls

Action	Control
Zoom	Mouse scroll wheel
Rotate view	Left click + drag
Pan	Middle click + drag / Shift + left click + drag
Focus on object	Click object in Entity Tree → press F
Reset view	Home key

Package Structure

~/BFMC_2027/Simulator_UniPD-DriveOps/src/
├── sim_pkg/                  # Launch files and world files
│   ├── launch/
│   │   ├── map_with_car.launch          # just car
│   │   ├── map_with_car_light.launch    # lightweight version
│   │   ├── all_objects.launch           # all objects
│   │   └── bridge.launch                # ROS2↔Gazebo bridge
│   └── worlds/
│       └── world_with_separators.world  # main world
├── models_pkg/               # 3D models (signs, vehicle, traffic lights...)
├── car_plugin/               # C++ vehicle control plugin
├── bno055_plugin/            # IMU plugin
├── gps_plugin/               # GPS plugin
├── traffic_light_plugin/     # Traffic light plugin
├── traffic_light_pkg/        # ROS2 node for traffic light logic
├── oak_mono_converter/       # Python node: RGB→mono8 + depth frame fix
├── example/                  # Control examples (keyboard, camera)
└── utils/                    # Utilities

Quick Reference

# Launch simulation (map + robot only)
cd ~/BFMC_2027/Simulator_UniPD-DriveOps && just car

# Launch full simulation (all objects, lightweight)
just all-objects-light

# Move robot forward
ros2 topic pub /automobile/command std_msgs/msg/String "data: '{\"action\": \"1\", \"speed\": 0.05}'" --once

# Steer left
ros2 topic pub /automobile/command std_msgs/msg/String "data: '{\"action\": \"2\", \"steerAngle\": 15.0}'" --once

# Stop
ros2 topic pub /automobile/command std_msgs/msg/String "data: '{\"action\": \"1\", \"speed\": 0.0}'" --once

# View camera feed
ros2 run rqt_image_view rqt_image_view

# List all active topics
ros2 topic list

# Check camera frame rate
ros2 topic hz /oak/rgb/image_raw

# Free RAM and restart
pkill -9 gz; pkill -9 ruby; pkill -9 ros2; pkill -9 python3
sudo sync && sudo sysctl vm.drop_caches=3

Name		Name	Last commit message	Last commit date
additional files		additional files
reference_materials/public_bfmc_car_materials		reference_materials/public_bfmc_car_materials
src		src
.envrc		.envrc
.gitignore		.gitignore
LICENSE		LICENSE
Picture1.png		Picture1.png
README.md		README.md
SIMULATOR_LOCALIZATION_RUN.md		SIMULATOR_LOCALIZATION_RUN.md
justfile		justfile
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation