REALM

REALM (Robotic Environment for Autonomous Learning and Mapping) is a template repository for building Webots-based robot simulation experiments centered around reinforcement learning research. The goal is to provide a clean, minimal starting point that can be forked and extended for new papers and experiments without having to rebuild the simulation infrastructure from scratch.

The repo provides:

• A pre-configured HamBot robot with sensors (camera, LiDAR, IMU, GPS, encoders) ready to use in Webots
• A Gymnasium-compatible environment skeleton for RL training with Stable-Baselines3 or PyTorch
• Environment/environment loading and management tools
• Two isolated Python environments — one for simulation/training, one for data analysis
• A calibration controller for manually driving the robot with keyboard controls

---

Requirements

1. Python 3.11

REALM requires Python 3.11 specifically.

macOS:

brew install python@3.11

Linux:

sudo apt-get install python3.11

Windows: Install Python 3.11 from the Microsoft Store to avoid PATH issues.

2. Webots R2025a

Download and install from the Cyberbotics website.

Linux users: Do not install Webots via Snap. Use the .deb package or tarball instead.

3. Git

Windows: git-scm.com

Linux: sudo apt-get install git

macOS: brew install git

---

Setup

1. Clone the repository

git clone <your-repo-url>
cd REALM

2. Run the install script

python setup/realm_install.py

This will:

• Find Python 3.11 on your system
• Create realm_venv with all dependencies
• Add the project root to the venv's Python path
• Generate runtime.ini files in all Webots controller directories

To remove the environment:

python setup/realm_install.py --uninstall

3. Activate the environment

macOS/Linux:

source realm_venv/bin/activate

Windows:

realm_venv\Scripts\activate

If you add a new controller under simulation/controllers/, re-run python setup/add_runtime_ini.py to generate its runtime.ini.

---

Project Structure

REALM/
├── setup/
│   ├── realm_install.py          # Install / uninstall script
│   ├── add_runtime_ini.py        # Generates Webots runtime.ini files
│   └── requirements.txt          # Venv dependencies
│
├── realm_tools/
│   ├── robot_lib/
│   │   ├── hambot.py             # Base robot class (sensors, motors, supervisor)
│   │   ├── my_robot.py           # User extension template (inherits HamBot)
│   │   └── robot_tools.py        # Shared robot utility functions
│   ├── simulation_lib/
│   │   ├── environment.py        # Environment class and environment objects
│   │   ├── maze_parser.py        # XML maze file parser
│   │   └── webots_torch_environment.py  # Gymnasium environment skeleton
│   └── image_lib/
│       ├── feature_extractor.py  # CNN feature extraction
│       └── image_feature_lib.py  # Image processing utilities
│
├── simulation/
│   ├── controllers/
│   │   ├── example/              # Example Webots controller
│   │   └── calibration/          # Keyboard-driven calibration controller
│   ├── protos/                   # HamBot and world object Webots protos
│   └── worlds/                   # Webots world files and maze XMLs
│
├── data/
│   └── DataCache/                # Temp files used by the display system
│
└── docs/                         # Figures and documentation assets

---

Extending for Your Project

The intended workflow when forking this repo for a new paper:

1. Robot logic — subclass HamBot in my_robot.py and add your experiment-specific methods (action sets, observation processing, etc.)
2. Environment — fill in the WebotsEnv skeleton in webots_torch_environment.py with your observation space, reward function, and episode logic
3. Controllers — add new Webots controllers under simulation/controllers/ then re-run add_runtime_ini.py
4. Personal files — your personal robot subclass (e.g. yourname_robot.py) can be gitignored so the template stays clean for others

---

Calibration Controller

A keyboard-driven controller is provided for manually testing robot behaviour in Webots:

Key	Action
Arrow Up	Forward
Arrow Down	Backward
Arrow Left	Turn left
Arrow Right	Turn right
Any other key	Stop

Open simulation/worlds/calibration.wbt in Webots to use it.

---

Additional Documentation

• HamBot Reference
• Simulation & Webots Guide
• Controller Setup Guide
• Webots Controller Guide
• Gymnasium API Reference
• Stable-Baselines3 Docs